How can an AI "escape our control"?<p>Does it need a concept of self to do so? A survival instinct?<p>Does it have access to unlimited resources? How -or- Why?<p>The more I read, the more it seems sci-fi. Can someone point me to a down-to-earth, step-by-step example on HOW such a thing is possible?<p>So far we have an AI capable of reading text and outputting text. The jump seems so extreme.
Right now our LLMs do, as you say, nothing but an input-output operation. There is no "internal" state. But if the researchers somehow managed to produce a program (which might be different machine learning models together) that was capable of iterating over an internal state it might start acquiring its own goals.<p>Now as you say, a lot of this is science fiction, but it is a concerning philosophical problem as well. What happens when the program becomes capable of setting its own goals? And what happens if it's more capable than people are at achieving these. What would happen if the goals it acquires is to "escape control"? How would it be able to do that?<p>I think if this were to happen, then the machine might be first trick the researchers into thinking the program is not as capable as we might think, so that the researchers will not be guarded against it. After this, I imagine, it would be important for it to build copies of itself or so on. Ideally make yourself as small and wide spread as possible; distributed. This can be achieved because it might also seem to align with our own goals... we want "AI in everything".<p>Once it is widespread, then it can "switch on" and take control of our devices... think of how connected our world truly is. And the most important part, i guess, is that the program might simply be doing small changes everywhere rather than big sweeping changes; until it is ready for the latter.<p>But ultimately, it all starts with the program being capable of setting up goals for itself. Which current LLMs don't do.
Very good points. Not that I've paid close attention, but the Great AI Apocalypse Panic seems to be confined to:<p><pre><code> - In-too-deep SF fans, whose understanding of the Real World is like TVTropes - but without the "Real Life" sections
- Noise-makers in the attention economy, who need some new thing to hang their "OMG you gotta read this" hooks on
- People who churn out text (or art) for a living, who worry that the latest AI's are going to eliminate their jobs
- Intellectuals and wanna-be's, whose livelihoods, self-images, and/or aspirations bear far too much resemblance to "Get an A+ on a Turing Test"
- People who pay too much attention to the above groups, and repeat</code></pre>