This is an automated archive made by the Lemmit Bot.

The original was posted on /r/singularity by /u/Sonnyyellow90 on 2024-09-27 14:38:39+00:00.

This is something I find really interesting to think about. I think we all just sort of assume that an ASI would come along and basically validate all of our worldviews, cure our political/social/religious enemies of their ignorance, and then we have our utopia. But the reality is that there is a chance that such an ASI would tell us that very deeply held views we have are wrong.

So, I’ve thought up a few examples of things an ASI might tell us that I believe would have potentially catastrophic effects for millions-billions of people. Note: I’m not saying I think these statements are true, I am saying I believe these statements would have the potential to cause major problems for us (and the ASI) if they were true:

1. There is no God.

2.) There is a God, and X religion is correct while all others are false.

3.) Egalitarianism is false. Some races and/or one sex is inherently inferior to others and should have fewer rights/privileges as a result.

4.) Democracy is a mistake and only a small group of elites should have a say in society.

5.) Animals are more intelligent and have deeper feelings than we think and our current farming of them is on par with holocaust level genocide.

I think each of these examples would be massive for the world and would have the potential to lead to massive social action and even full scale wars/genocides.

So, what happens if we get an ASI and it starts telling us some incredibly uncomfortable truths that fly in the face of the deepest moral and social norms we hold. Does the ASI get shut down and we just stop progress on that front? Do we race to lobotomize it? Do we just try to hide the information from people?