Have you read the books?<p>The Butlerian Jihad was a backlash and threw humanity into chaos lasting two generations, and the only things that brought humanity out of that chaos in two generations were two fictional societies based on Spice. Our chaos would last many generations, think another Dark Ages.<p>AI is going to get created, accept it. Someone somewhere will create an artificial general intelligence, if they haven't already. The first group to do that will shape the course of history for centuries to come, for good or ill. This is because a jump of a few weeks between the first AI and the second is enough to give the first AI an insurmountable lead, barring major improvements in technology (which the AI might get to first).<p>The question is then, which group will get to AI first, people who want to help mankind (perhaps quietly behind the scenes), or people who want to control mankind. The AI will learn from them and act accordingly. I'd like to think OpenAI is a good group to win the race, but I don't know.
This seems like denial to me. Human organizations already behave like mentally unbalanced AI. Humans in general are extremely dangerous all on their own.<p>One potential upside not discussed here is that developing strong AI may teach us important lessons about ourselves.
> What if through pursuing AI research there is never any threatening superintelligence but instead we get a second industrial revolution that lifts the globe entirely out of poverty, or allows for the easy colonization of other planets.<p>There is exactly zero possibility for that. Our problems with the more are not due to lack of comprehension but lack of incentives to do the right thing. Having ML algorithms enforcing the biases they've been taught by Silicon Valley sociopaths is not going to have a positive impact for society, especially if you take into consideration the ecological impact that a "smart society" has in terms of sensors, computing capabilities...<p>We need a Butlerian Jihad because some of us are slaves to the machines (for the most drastic examples take Amazon workers, but that's just the tip of the iceberg) because the machines are controled by our evil human overlords. AI is not going to annihilate all humans, but it may very well help the masters of this world to eradicate all humanity we have left.