<i>The failure mode of governments is to become “exploitative and corrupt,” Runciman notes. The failure mode of corporations, as extensions of an independent civil society, is that “their independence undoes social stability by allowing those making the money to make their own rules.” There is only a “narrow corridor”—a term Runciman borrows from the economists Daron Acemoglu and James A. Robinson—in which the artificial agents balance each other out, and citizens get to enjoy the sense of control that emerges from an atmosphere of freedom and security. The ideal scenario is, in other words, a kludgy equilibrium.</i><p>This is really interesting; to draw parallels between AI and the emergence of States and Corporations, which are two other inventions that we have lost control of. You could argue that Climate Change is an X-risk bought about by Corporations, which lumber around our civil landscape like single minded giants pursuing profit above all else and using regulatory capture to push externalities onto the populace.<p>I'm a big fan of The Corporation (2003), a documentary that explores how Corporations came to have legal rights, and asks, if Corporations are legal people, what sort of people are they? They exhibit reckless disregard for the safety of others, deceitfulness, the incapacity to experience guilt, and the failure to conform to social norms and respect the law, which matches the DSM-IV's symptoms of psychopathy.