> If we solve alignment, we can look forward to material and cultural abundance.<p>I don’t know what it would mean to “solve alignment”. I am skeptical it’s even a coherent possibility.<p>Let’s assume we had a perfect AI. It does everything we want it to but is much more effective at accomplishing our goals.<p>The United States government takes control of the AI and uses it to attack China. China would say the AI is misaligned. Or China takes control of it and uses it to attack the US. The US would say it’s misaligned.<p>Ok, you might say that the issue was that in the above case the AI was acting as a tool rather than an independent agent. Fine. Now, the AI acts on its own to further goals which would be beneficial to humanity. Very quickly we will want to disable it because we don’t even agree on what’s best for humanity, and even if we can agree on general principles any action will have externalities that someone won’t like.