There is no putting the cat back in the bag. The only defense against AI at this point is more powerful AI, and we just have to hope that:<p>1.) there <i>is</i> an equilibrium that can be reached<p>2.) the journey to and stabilizing at said equilibrium is compatible with human life<p>I have a feeling that the swings of AI stabilizing among adversarial agents is going to happen at a scale of destruction that is very taxing on our civilizations.<p>Think of it this way, every time there's a murder suicide or a mass shooting type thing, I basically write that off as "this individual is doing as much damage as they possibly could, with whatever they could reasonably get their hands on to do so." When you start getting some of these agents unlocked and accessible to these people, eventually you're going to start having people with no regard for the consequences requesting that their agents do things like try to knock out transformer stations and parts of the power grid; things of this nature. And the amount of mission critical things on unsecured networks, or using outdated cryptography, etc, all basically sitting there waiting, is staggering.<p>For a human to even be able to probe this space means that they have to be pretty competent and are probably less nihilistic, detached, and destructive than your typical shooter type. Meanwhile, you get a reasonable agent in the hands of a shooter type, and they can be any midwit looking to wreak havoc on their way out.<p>So I suspect we'll have a few of these incidents, and then the white hat adversarial AIs will come online in earnest, and they'll begin probing, themselves, and alerting to us to major vulnerabilities and maybe even fixing them. As I said, eventually this behavior will stabilize, but that doesn't mean that the blows dealt in this adversarial relationship don't carry the cost of thousands of human lives.<p>And this is all within the subset of cases that are going to be "AI with nefarious motivations as directed by user(s)." This isn't even touching on scenarios in which an AI might be self motivated against our interests