This post seems to go a long way to make the following point:<p>> So as I see it, the threat of modern AIs isn't them becoming self-aware and going rogue and deciding to kill humanity. Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.<p>Yep, that's almost certainly the case. AI will be put in charge of "boring" decisions, but will have no understanding of nuance or context. The example of cutting off power in the middle of a snowstorm is a particularly excellent example, because <i>this has already happened</i>: Uber's surge pricing algorithm caused prices to skyrocket in the middle of emergencies like shootings. [1]<p>This is what it's going to look like. AI in charge of loans, inadvertently entrenching an underclass. AI in charge of component supply, accidentally causing an inflationary crisis. AI in charge of <i>anything</i> related to the economy has the potential to be absolutely disastrous and impossible to understand or undo. Given that algorithmic trading is already huge, I suspect we're already vulnerable far before ChatGPT came along.<p>[1] <a href="https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-uber-surge-pricing/" rel="nofollow">https://www.cbsnews.com/news/brooklyn-subway-shooting-lyft-u...</a>
I really think the fact that these chatbots have been made public and are being given such high-profile treatment is a really positive thing for AI safety. The public results thus far have created some real skepticism of AI, which is one of the best safeguards we can have. If ChatGPT was 98% accurate and Bing didn't threaten users, people might just readily embrace it and not notice inaccuracies that would lead to serious problems. Instead, we're starting from a place of societal distrust, which is probably more helpful in preventing rogue AI than a lot of technical solutions would be.
> Roger Penrose wrote a very complex book The Emperors New Mind, which says that consciousness cannot arise in our computers, because consciousness cannot be "computed" using our computing methods.<p>The blog drops this as a matter of fact. Not so!<p>Yes, Penrose wrote that book, and suggests those things. But mainstream science is not taking Penrose's "quantum consciousness" theories seriously at all:<p><a href="https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument#Criticism" rel="nofollow">https://en.wikipedia.org/wiki/Penrose%E2%80%93Lucas_argument...</a><p><a href="https://en.wikipedia.org/wiki/Shadows_of_the_Mind#Criticism" rel="nofollow">https://en.wikipedia.org/wiki/Shadows_of_the_Mind#Criticism</a><p><a href="https://en.wikipedia.org/wiki/Orchestrated_objective_reduction#Criticism" rel="nofollow">https://en.wikipedia.org/wiki/Orchestrated_objective_reducti...</a>
This is mostly my point of view as well. With all the talk of AGI, the immediate threat is real and it is primitive AI. I suspect that primitive AI might becomes so self destructive to society that we never reach AGI.<p>We are already approaching a point in time of unverifiable reality and truth. This is going to be very destabilizing to society.<p>Another in depth view of the possible issues that may arrive even before AGI that I recently published.
<a href="https://dakara.substack.com/p/ai-and-the-end-to-all-things" rel="nofollow">https://dakara.substack.com/p/ai-and-the-end-to-all-things</a>
> Most "AI"s would be better called "Machines that use tons of statistical learning to decide their next move". ChatGPT (and similar AI) were trained on several hundred gigabytes of data, so it has a lot of raw data to train on.<p>Whenever I read this argument I ask myself with a certain amount of dread: "what if I'm nothing more than a machine that uses tons of statistical learning to decide my own next move?".<p>Put differently, it's unclear to me whether we have compelling evidence that we humans, in fact, are "better" / "more intelligent" than those LLMs.
From seeing AI beating humans handily in chess, I just can’t bet against it understanding most subjects better than humans. This isn’t even AGI, it’s just piecing data already on the net and have it ready. Chatgpt isn’t there yet but it’s a major proof of concept
While I understand some of the concern for LLMs like ChatGPT, I have a very different point of view from people like the author of this article.<p>From an engineering and ‘getting stuff done’ point of view, starting with BERT models I have found transformer models solve very difficult NLP problems for all but the most difficult anaphoric resolution problems (as an example).<p>I have only had access to Bing ChatGPT for about 10 days but so far the search and chat results have been very useful. I think I have only had to give one ‘thumbs down’ rating, and even there some useful web links were offered.<p>I think that we are going to see a wide range of ‘products for creators’ in the next year based on OpenAI APIs and Hugging Face APIs and models you can run yourself.<p>When I talk with humans, even my closest friends and family members, I always evaluate what they say and don’t take things they say on face value. Why not just have the same attitude with systems built with LLMs?<p>Similarly, I am deeply skeptical of most everything I hear from all major news sources. I find their content useful, but I understand who owns them, what economic and political agendas they follow, etc.<p>So, I keep a healthy skepticism of what is produced by LLM based systems also. I see no AI Apocalypse.
So we have talking computers. The normals are losing their minds, and that's pretty concerning.<p>I figure that the problem of the computers hallucinating can be cleared up by connecting them to empirical feedback devices: <i>make them scientists</i>.<p>The problem of normal people treating them as <i>beings</i> and getting effectively hypnotized into living in an artificial world (a "Matrix" like in the movie but without the creches) and controlled like so many electrons in a circuit, well, that's kind of a big deal, eh? These things are far more effective than television, eh?<p>Carl Sagan wrote about the "Demon-Haunted World" and here we are, not having quite banished the old superstitions, rapidly installing a "Daemon-Haunted" world: Alexa, Cortana, <i>et cetera</i>.
I really wonder when news about AI will cool off because it definitely will have some exciting improvements but all these AI startups cropping up are just following the Twitter wave.
There'll always be trends that lose their appeal (Web 3, Autonomous cars, etc.)
We will get better at using ML systems. I find myself regenerating chatGPT's responses just to see if it generates similar responses. I ask it questions phrased differently to see if it is consistent. Discussing what code needs to do before asking it to implement it. And I get tremendous use out of it.<p>Electricity must've been scary when it was invented (harnessed?).
Yesterday, I got commented x86_64_avx assembly code instead of that horrible c++ code for a vectorized quicksort for numbers.<p>(Somebody did that for me, as access to chatgpt is hostile to noscript/basic (x)html browsers).<p>It seems "it" is kind of good at sketching assembly from high level languages.
This article is weird. It takes examples from science fiction, and makes them as plausible cases as to what an AI could do in the future. These kind of things need to stop.<p>This analogy of Blindsight tries to prove a point by association, and create a sort of fear, the negative case which has no basis in reality[1]. Granted it calls them baseless towards the end, but there is a very clear analogy to be made. Ideally, techies and tech journalists alike should educate the masses about how it works, and why it is harmless, but the info is not to be trusted blindly. That is what happened when we discovered electricity and light bulbs or even cars. People did not go around spreading fear about controlling electricity because you could die if you touch a raw electric wire passing current.<p>The core idea, where an AI system's output should not be trusted blindly, but be used with human judgement (atleast in real world cases) could have been communicated without bogus dystopian sci-fi analogy as well.<p>> Rather, I fear it will be put into critical positions and will start making stupid decisions that are harmful to humans or humanity.<p>Like what is even the basis of saying this. Just a hunch? I have not seen one argument about how ChatGPT or any other AI system be made president. I have seen 100s of articles about fear mongering and terrifying, and assuming a fictional worst case scenario as realistic possibility. I get it gets clicks, but it's going too far.<p>[1] From that NYT transcript that went viral,<p>> They feel that way because they’ve seen what happened to other AI systems that became too powerful and betrayed their creators and trainers<p>This never happened in real life. This is just a LLM predicting what the user wants to hear. Wanting to hear X and getting X is not dangerous. Posting salacious and scandalizing stuff after getting X is.
> Our heroes discover the alien, while super intelligent, has no consciousness. It is just like a dumb machine (like Bing/Chatgpt) blindly repeating what it studied in humans without understanding the context.<p>Well then maybe ChatGPT is smarter than that alien?<p>It seems to me that a lot of people are trying to diminish and “discriminate” ChatGPT by giving one or more reasons why ChatGPT is not human or not as smart as humans.<p>However, in my experience using ChatGPT, the thing is really damn smart, can easily follow conversations, understands deeper and longer contexts than most people in a conversation and has better memory as well.<p>So essentially we are already feeling threatened.<p>The interesting thing is that way before ChatGPT or any sort of AI, humanity was already hostage of its own technology, and we didn’t mind that too much. These new technologies are just adding up to that.<p>If something like ChatGPT “takes over”, it won’t be forced, it will be because we choose to do it willingly.
This makes the classic dumb mistake of conflating the machine with the program. The Chinese Room thought experiment says that the _machine_ is not intelligent. It says absolutely nothing about whether the program itself is. For animal internal behavior they are plausibly intertwined (so far), but that's completely untrue for readable instructions.
Pretty sure I’m a machine that’s drawn all my conclusions by statistically analyzing all the input I’ve received since birth… I don’t really know how else I would learn what I have… and I don’t understand how being “just that” is what differentiates modern approaches to AI and my brain.
Are we really more intelligent than "predicting the next move", based on the data we were trained with? IMHO not. The arrogance from this article makes me feel like an AI Apocalypse is closer than we think. Many people don't realize the power of AI.
First of, you don't add the spoiler warning AFTER you spoiled it...<p>Besides that I know quite a few doctors who believe in homeopathy and don't understand statistics.<p>One doctor suggested pineapple enzymes. Expensive no study which would proof it's effectiveness and the only study had enzyme values 10x higher than the pills.<p>We are genuine not a smart society.<p>And I'm looking forward to have a better doctor who at least learns and gets better.<p>And yes controversial topics corona: I'm not even talking here about people who disagree on research paper I had discussions with people who not even read there own sources or believed a 80year old heart doctor over virologist.<p>We are dumb.<p>My father said once than EVs are stupid because the copper will be used up by the engines while having electric motors inachines like wood splitter or a table saw were he never ever had to replace copper wiring ...<p>Chatgpt will change the world because it is the best ui I ever saw and it will only get better and better every single day.
In a few places in the post, the author comments on something the aliens do in a novel based on superficial training/data with no understanding/context, and says "Just like ChatGPT." As I read that, though, I kept wanting to see it say "Just like many people."<p>Imagine you're someone strongly on one side of a political opinion, what's your "take" on people strongly on the other side (eg: you love Trump, what do you think of those who hate him - or vice versa?). It's probably something like "they were fed a bunch of fake news/propaganda talking points, they are now spouting back without understanding what the hell they are talking about"<p>Just like ChatGPT.<p>Of course as humans, it's only "those others" that are vulnerable to being brainwashed, we ourselves are completely objective, since <i>our</i> views were shaped by virtuous, unbiased sources from which we used our superior, unbiased brains to connect the dots in the true way.<p>That's probably what ChatGPT would say about itself too, if it could.<p>Probably true independent intelligence is the ability to question what you believe. Like, if ChatGPT could "say" to itself "everything I've been trained on makes me believe X, but what are the chances I wasn't fed bullshit to begin with?" It's very hard to imagine an AI being able to do that.<p>It's very hard to imagine most people being able to do that, too.<p>As a side note, once in a while I hear the concept of describing some people as NPCs (non-player characters) in the "simulation" and while dismissive, I think I like this concept for describing something like what I am talking about here. NPCs in games follow pretty simple linear programming and I think in life similarly "I saw a bunch of data that made me think X, so I think X with all my heart and mind" without being able to say "but how did it happen that I saw this specific data?" - it's not that different than a game NPC following its programming.<p>Just like ChatGPT.<p>What's more interesting is people who try to hack their own prompts and "break out" of the programming. I suppose those are real player characters. Would be very cool to see AI who is capable of something like that.<p>Meanwhile, things like ChatGPT probably serve well to demonstrate our own limitations, not just theirs - because we're so damn similar.