TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

AI hallucinations: Why LLMs make things up (and how to fix it)

196 点作者 emil_sorensen6 个月前

29 条评论

lolinder6 个月前
&gt; While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced...<p>Every article on hallucinations needs to <i>start</i> with this fact until we&#x27;ve hammered that into every &quot;AI Engineer&quot;&#x27;s head. Hallucinations are not a bug—they&#x27;re not a different mode of operation, they&#x27;re not a logic error. They&#x27;re not even really a distinct kind of output.<p>What they <i>are</i> is a value judgement we assign to the output of an LLM program. A &quot;hallucination&quot; is just output from an LLM-based workflow that is not fit for purpose.<p>This means that all techniques for managing hallucinations (such as the ones described in TFA, which are good) are better understood as techniques for constraining and validating the probabilistic output of an LLM to ensure fitness for purpose—it&#x27;s a process of quality control, and it should be approached as such. The trouble is that we software engineers have spent so long working in an artificially deterministic world that we&#x27;re not used to designing and evaluating probabilistic quality control systems for computer output.<p>[0] They link to this paper: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2401.11817" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2401.11817</a>
评论 #42323416 未加载
评论 #42328133 未加载
评论 #42324032 未加载
评论 #42323895 未加载
评论 #42323774 未加载
评论 #42328200 未加载
评论 #42323626 未加载
评论 #42326255 未加载
评论 #42328145 未加载
评论 #42331803 未加载
评论 #42326536 未加载
评论 #42328208 未加载
评论 #42323760 未加载
评论 #42325795 未加载
评论 #42325412 未加载
评论 #42323795 未加载
评论 #42326193 未加载
tokioyoyo6 个月前
To my understanding, the reason why companies don&#x27;t mind the hallucinations is the acceptable error rate for a given system. Let&#x27;s say something hallucinated 25% of the time, but if that&#x27;s ok, then it&#x27;s fine for a certain product. If it only hallucinates 5% of the time, it&#x27;s good enough for even more products and so on. The market will just choose the LLM appropriately depended on the tolerable error rate.
评论 #42323764 未加载
评论 #42326568 未加载
Terr_6 个月前
When people talk about stopping an LLM from &quot;seeing hallucinations instead of the truth&quot;, that&#x27;s like stopping an Ouija-board from &quot;channeling the <i>wrong</i> spirits instead of the right spirits.&quot;<p>It suggests a qualitative difference between desirable and undesirable operation that isn&#x27;t really there. They&#x27;re all hallucinations, we just happen to like some of them more than others.
评论 #42323144 未加载
评论 #42323276 未加载
评论 #42331695 未加载
评论 #42324145 未加载
评论 #42328041 未加载
Loughla6 个月前
I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it&#x27;s plagiarism doesn&#x27;t have an impact, but showing them how it gets even simple things wrong had a HUGE impact.<p>The first problem was a simple numbers problem. It&#x27;s 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or vertically. The numbers must add up to 1000 when you get to the exit. For people it takes about 5 minutes to figure out. The AI couldn&#x27;t get it after all 50 students each spent a full 30 minutes changing the prompt to try to get it done. The AI would just randomly add numbers and either add extra at the end to make 1000, or just say the numbers added to 1000 even if it didn&#x27;t.<p>The second problem was writing a basic one paragraph essay with one citation. The humans got it done, when with researching for a source, in about 10 minutes. After an additional 30 minutes none of the students could get AI to produce the paragraph without logic or citation errors. It would either make up fake sources, or would just flat out lie about what the sources said. My favorite was a citation related to dairy farming in an essay that was supposed to be about the dangers of smoking tobacco.<p>This isn&#x27;t necessarily relevant to the article above, but if there are any teachers here, this is something to do with your students to teach them exactly why not to just use AI for their homework.
评论 #42323137 未加载
评论 #42323466 未加载
评论 #42323556 未加载
mdaniel6 个月前
the only comment on the prior submission 3 days ago summarizes the whole thing: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42285149">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42285149</a><p>Also, I saw any such blog title as &quot;how to make money in the stock market:&quot; friend, if you knew the answer you wouldn&#x27;t blog about it you&#x27;d be infinitely rich
评论 #42323275 未加载
int_19h6 个月前
I&#x27;ve been playing with Qwen&#x27;s QwQ-32b, and watching this thing&#x27;s chain of thought is really interesting. In particular, it&#x27;s pretty good at catching its own mistakes, and at the same time, gives off a &quot;feeling&quot; of someone very uncertain about themselves, trying to verify their answer again and again. Which seems to be the main reason why it can correctly solve puzzles that some much larger models fail. You can still see it occasionally hallucinate things in the CoT, but they are usually quickly caught and discarded.<p>The only downsides of this approach is that it requires a lot of tokens before the model can ascertain the correctness of its answer, and also that sometimes it just gives up and concludes that the puzzle is unsolvable (although that second part can be mitigated by adding something like &quot;There is definitely a solution, keep trying until you solve it&quot; to the prompt).
评论 #42326242 未加载
Der_Einzige6 个月前
Wow, a whole article that didn&#x27;t mention the word &quot;sampler&quot; once. There&#x27;s pretty strong evidence coming out that truncation samplers like min_p and entropix are strictly superior to previous samplers (which everyone uses like top_p) to prevent hallucinations and that LLMs usually &quot;know&quot; when they are &quot;hallucinating&quot; based on their logprobs.<p><a href="https:&#x2F;&#x2F;openreview.net&#x2F;forum?id=FBkpCyujtS" rel="nofollow">https:&#x2F;&#x2F;openreview.net&#x2F;forum?id=FBkpCyujtS</a> (min_p sampling, note extremely high review scores)<p><a href="https:&#x2F;&#x2F;github.com&#x2F;xjdr-alt&#x2F;entropix">https:&#x2F;&#x2F;github.com&#x2F;xjdr-alt&#x2F;entropix</a> (Entropix)<p><a href="https:&#x2F;&#x2F;artefact2.github.io&#x2F;llm-sampling&#x2F;index.xhtml" rel="nofollow">https:&#x2F;&#x2F;artefact2.github.io&#x2F;llm-sampling&#x2F;index.xhtml</a>
LetsGetTechnicl6 个月前
Why do LLMs make things up? Because that is all that LLMs do, sometimes what it outputs is correct though.
PLenz6 个月前
Everything an LLM returns is an hallucination, it&#x27;s just that some of those hallucinations line up with reality
评论 #42323514 未加载
评论 #42324158 未加载
评论 #42323900 未加载
评论 #42323266 未加载
throwawaymaths6 个月前
Completely misses the fact that a big part of the reason why llms hallucinate sp much is because there&#x27;s a huge innate bias towards producing more tokens over just stopping.
评论 #42324182 未加载
TZubiri6 个月前
The debate around &quot;fixing&quot; hallucinations reminds me of the debate around schizophrenia.<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=nEnklxGAmak" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=nEnklxGAmak</a><p>It&#x27;s not a single thing, a specific defect, but rather a failure mode, an absence of cohesive intelligence.<p>Any attempt to fix a non-specific ailment (schizophrenia, death, old age, hallucinations) will run into useless panaceas.
fsckboy6 个月前
it&#x27;s superficially counterintuitive to people that an AI that will sometimes spit out verbatim copies of written texts, also will just make other things up. It&#x27;s like &quot;choose one, please&quot;.<p>MetaAI makes up stuff reliably. You&#x27;d think it would be an ace at baseball stats for example, but &quot;what teams did so-and-so play for&quot;, you absolutely must check the results yourself.
评论 #42326201 未加载
tshadley6 个月前
The article referenced the Oxford semantic entropy study but failed to clarify that the issue greatly simplifies LLM hallucination (making most of the article outdated).<p>When we are not sure of an answer we have two choices: say the first thing that comes to mind (like an LLM), or say &quot;I&#x27;m not sure&quot;.<p>LLMs aren&#x27;t easily trained to say &quot;I&#x27;m not sure&quot; because that requires additional reasoning and introspection (which is why CoT models do better); hence hallucinations occur when training data is vague.<p>So why not just measure uncertainty in the tokens themselves? Because there are many ways to say the same thing, so a high entropy answer may only reflect uncertainty in synonyms-- many ways to say the same thing.<p>The paper referenced works to eliminate semantic similarity from entropy measurements, leaving much more useful results, proving that hallucination is conceptually a simple problem.<p><a href="https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-024-07421-0" rel="nofollow">https:&#x2F;&#x2F;www.nature.com&#x2F;articles&#x2F;s41586-024-07421-0</a>
评论 #42323887 未加载
评论 #42328295 未加载
评论 #42323702 未加载
IWeldMelons6 个月前
LLM hallucinations in fact has a positive side effect too, if you are using them for learning some subject; makes you verify their claims, and finding errors in them is very rewarding.
评论 #42327551 未加载
imchillyb6 个月前
Toddlers don&#x27;t understand truth either, until it&#x27;s taught.<p>This crayon is red. This crayon is blue.<p>The adult asks: &quot;is this crayon red?&quot; The child responds: &quot;no that crayon is blue.&quot; The adult then affirms or corrects the response.<p>This occurs over and over and over until that child understands the difference between red and blue, orange and green, yellow and black etcetera.<p>We then move on to more complex items and comparisons. How could we expect AI to understand these truths without training them to understand?
评论 #42331837 未加载
madiator6 个月前
For the specific form of hallucination, which is called grounded factuality, we have trained a pretty good model that can detect if a claim is supported by a context. This is super useful for RAG. More info at <a href="https:&#x2F;&#x2F;bespokelabs.ai&#x2F;bespoke-minicheck" rel="nofollow">https:&#x2F;&#x2F;bespokelabs.ai&#x2F;bespoke-minicheck</a>.
评论 #42330230 未加载
pfisch6 个月前
Anyone who has raised a child knows they hallucinate constantly when they are young because they are just doing probabilistic output of things they have heard people say in similar situations and saying words they don&#x27;t actually understand.<p>LLMs likely have a similar problem.
mwkaufma6 个月前
How do we discriminate when a response is correct, vs. when it&#x27;s &quot;hallucinating&quot; an accurate fact, by coincidence? Are all responses hallucinations, independent of correspondence to ground-truth?
prollyjethi6 个月前
I am honestly very skeptical of articles like these. Hallucinations are a feature of LLMs. The only ways to &quot;FIX&quot; it is to either stop using LLMs. Or use a super bias some how.
评论 #42324019 未加载
Sergii0016 个月前
That&#x27;s all really weird. You can watch how chat gpt gives you advice on which mushrooms are safe. And now it can be just hallucinations
threeseed6 个月前
Maybe don&#x27;t make things up in a blog post about LLMs making things up.<p>Because you don&#x27;t know how to fix it. Only how to mitigate it.
Mistletoe6 个月前
Is there a way to code an LLM to just say &quot;I don&#x27;t know&quot; when it is uncertain or reaching some sort of edge?
评论 #42323869 未加载
评论 #42328350 未加载
评论 #42323736 未加载
评论 #42323896 未加载
dschuetz6 个月前
I went straight to the &quot;how to fix&quot; section with popcorn in hand and I wasn&#x27;t disappointed: just add &quot; doubt&quot; layers for self-correction, beginning at the query itself. And then maybe tell the model &quot;do not hallucinate&quot;. Sounds like a pun, but I think an AI model actually would take this seriously, because it can&#x27;t tell the difference.<p>Context is still a huge problem for AI models, and it&#x27;s probably still the main reason for hallucinating AIs.
rabid_turtle6 个月前
I don&#x27;t like the output = hallucination<p>I like the output = creative
chefandy6 个月前
Lots of folks in these conversations fail to distinguish between LLMs as a technology and &quot;AI Chatbots&quot; as commercial question answering services. Whether false information was expected or not matters to LLM product developers, but in the context of a commercial question-answering tool, it&#x27;s irrelevant. Hallucinations are bugs that creates= time-wasting zero-value output, at best, and downright harmful output at worst. If you&#x27;re selling people <i>LLM pattern generator output</i>, they should expect a lot of bullshit. If you&#x27;re selling people answers to questions, they should expect accurate answers to their questions. If paying users are <i>really</i> expected to assume every answer is bullshit and vet it themselves, that should probably move from the little print to the big print because a lot of people clearly don&#x27;t get it.
sollewitt6 个月前
&quot;Why LLMs do the one and only thing they do (and how to fix it)&quot;
jrflowers6 个月前
I like that none of the suggestions address probabilistic output generation (aside from the first bullet point of section 3C, which essentially suggests that you just use a search engine instead of a language model).<p>TLDR: Hallucinations are inherent to the whole thing but as humans we can apply bubble gum, bandaids and prayers
PittleyDunkin6 个月前
Humans hallucinate, too. We just have less misleading terms for it. Massive mistake in terms of jargon, IMO—&quot;making shit up&quot; is wildly different from the &quot;delusion of perception&quot; implied by hallucination.
uz441006 个月前
hallucination problem in LLM, been seeing this. Let me know if someone find a fix please