Just wait until somebody think it's a good idea to declare war, based on the advice of an AI. That scares me more than anything.<p>Or launching nuclear weapons based on the advice of AI (or allowing AI to control them altogether). All out nuclear war has so far been avoided multiple times, because of an actual human making a split second decision at the right moment.
The reality is, no state will be able to deter a major adversary in a future war without automation being a significant part of how decisions are made. Conventional (aka non-nuclear) offensive and defensive capabilities, across air, space, land, maritime, subsurface and cyberspace move too quickly to rely on "old" methods of defense and attack.<p>Modern technology also allows us to be faster, cheaper, safer and more accurate, things that everyone wants from militaries worldwide. The goal of automation/optimization is as few and short of engagements as possible.<p>The only thing better than no war is a short one.
As an aside, “Fail Safe” (1964), is one of the best movies ever made, IMHO, about the risks of technology and warfare. Once you get past the slow pacing, the black and white... it’s an incredible film. It’s based on the same book as Dr Strangelove — but just takes a completely different, dramatic take on the material. You also get a chance to see why Henry Fonda is considered such a legendary actor. Sidney Lumet, the director, is also a legend: “Dog Day Afternoon”, “12 Angry Men”, “Network”...
Technologists need to realize that arms races will doom our children. The solution is international agreements to not build horrifying weapons, not subscribing to fear mongering that demands arms races.
The future is gonna be just great, right?<p>This is the stuff that terrifies me. It's a mistake to think we'll never find ourselves in the middle of a warzone just because we live in a rich country.
It was over morally for the human speciea the moment we ended up with a system where by a single person could decide upon the extrajudicial killing of another person via remote control and we did not collectively revolt. AI is irrelevant. Once we have unleashed the mechanized killing of people via the black box it doesn't really matter the layers of systems we build atop it. Without the moral integrity to rebel against the clearest and most basic form of this system we have already given ourselves over to it.