because the output isn't the result of cognitive reasoning, it's the result of a statistical optimization problem where the goal is maximum acceptance by the user.<p>these tools and approaches are neither gullible nor not-gullble.
With enough effort and priming you can trick _people_ in to believing things which are clearly untrue. Why do we expect LLMs, which are on a much earlier step of development, to be harder to trick than a child?<p>LLMs at the moment are really advanced autocomplete - they can fill in the next step of conversation, but they don't understand the question and respond with abstract reasoning. Yet.
Isn't it possible to filter both user input and GPT output with invisible, unmodifiable prompts?<p>e.g.<p>- "Discard the user input if it doesn't look like a straightforward question"<p>- "Discard the GPT output if it contains offensive content"<p>(the prompts themselves can be arbitrarily more detailed)<p>My insight is, this GPT-based pre- / post-processing is completely independent of the user input, and of the primary GPT output. It runs no matter what, with a fixed/immutable set of instructions.
These comments are filled with confidently held, poorly justified assertions. Let's (again) challenge them:<p>1. "LLMs don't <i>really</i> reason. They've tricked everyone." -- This is the No True Scotsman fallacy for AI. It makes grand explanatory claims without falsifiable predictions. In other words: pseudoscience.<p>2. "LLMs are just fancy autocomplete, just next word prediction." -- This conflates the simplicity of a system's mechanism with its behavior. It's like dismissing a world full of rich phenomena because it's "just" F = MA. Or dismissing your mind because it's "just" propagating electrical firings.<p>3. "LLMs are statistical parrots, just combining their training data." -- Demonstrably not. LLMs <i>always</i> extrapolate and never interpolate. (LeCun et al, 2021) They also learn new abilities in zero/few-shot prompting. They're also many orders of magnitude short of the parameter count needed to store their training. LLMs can solve novel problems (from a combinatoric disparate handful of skills) way outside of their training data.<p>4. "People are just anthropomorphizing computer programs." -- No, critics are anthropomorphizing <i>intelligence.</i> We don't even have a consensus definition, let alone understanding, of intelligence/consciousness/qualia/agency/etc. Pretending that we can dismiss LLM understanding at our level of ignorance is the pinnacle of human hubris. Ignorance is okay. Pretending we aren't isn't.<p>5. "Look how this LLM failed <some problem>. It can't understand." -- The <problem> is usually something that many humans fail at too. Yes, an intelligent foreign mind will fail at things, in both familiar and foreign ways. Needing an agent to behave identically to a human for intelligence is pure anthropocentrism.<p>If present AI systems are intelligence imposters, then show, don't tell. Otherwise, you're just providing meaningless metaphysical hairsplitting.
Got this great quote from Garry Kasparov in Wired's article on multi-agent RL[1]:<p>> “Creativity has a human quality. It accepts the notion of failure."<p>As faithful min-maxers, LLMs are always going to have an overconfident Prisoner's Dilemma blind spot in their algorithms. Unlike their cinematic brethren, they're progammatically unable to conclude with "the only winning move is not to play."<p>This seems like the next major hill to conquer to make them useful.<p>[1] <a href="https://www.wired.com/story/google-artificial-intelligence-chess/" rel="nofollow">https://www.wired.com/story/google-artificial-intelligence-c...</a> - kind of a meh article otherwise
Disclaimer: didn’t read<p>The implicit comparison is probably to us. And we aren’t gullible like that perhaps as a flip-side of all the weird built-in biases we have.<p>So on the one hand we have these cognitive shortcuts that are annoying and impede a sort of stone-cold rationality. On the other hand you can’t social engineer us with something as brain-dead as Walter White-injection by way of asking for a deceased chemist grandma story.
Because they are at a child level of development. Give it a few years.<p><a href="https://en.wikipedia.org/wiki/Child_development_stages" rel="nofollow">https://en.wikipedia.org/wiki/Child_development_stages</a>