The point not discussed here is where does the information comes from.<p>Is it a primary source of secondary source [1] ? And how to incorporate this new information.<p>In their quest for building a "truthful" knowledge base, LLMs incorporate implicitly facts they read from their training dataset, into their model weights. Their weight update mechanism, allows to merge the facts of different authority together to compress them and not store the same fact many times, like in a traditional database.<p>This clustering of similar new information is the curse and the blessing of AI. It allows faster retrieval and memory-space reduction. This update mechanism is usually done via Bayes rule, doing something called "belief propagation".<p>LLMs do this implicitly, and have not yet discovered that while belief propagation works most of the time, it's only guaranteed to work when the information graph have no more than one loop. Otherwise you get self reinforcing behavior, where some source cites another and gives it credit, which gives credit to the previous source, reinforcing a false fact in the similar fashion as farm links help promote junk sites.<p>When repeating a false information to a LLM many times, you can make it accept it as truth. It's very susceptible to basic propaganda. LLMs can be a triple-store or a quad-store based on how and what they are trained.<p>But LLM can also incorporate some error correction mechanism. In this article, the LLM tried two times to correct itself failed to do so, but the blog author published an article which will be incorporated into the training dataset, and the LLM will have another example of what it should have answered, provided that the blog author is perceived as authoritative enough to be given credence.<p>This error correction mechanism with human in the loop, can also be substituted by a mechanism that rely on self consistency. Where the LLM build its own dataset. And asks questions to itself about the fact it knows, and tries to answer them based on first principles.<p>For example the LLMs can use tools to retrieve the original papers, verify their time and date, and see who coined the term first and why. By reasoning it can create a rich graph of facts that are interconnected, and it can look for incoherence by asking itself. The more rich the graph, the better the information can flow along its edges.<p>Because LLMs are flexible there is a difference between what they can do, and what they do, based on whether or not we trained them to make emerge the behavior we desire to emerge. If we don't train them with a self consistency objective they will be prone to hallucinations. If we train them based on Human Feedback preference we will have a sycophants AI. If we train them based on "truth", we will have "know it all" AIs. If we train them based on their own mirrors, we will have what we will have.<p>[1]<a href="https://www.wgu.edu/blog/what-difference-between-primary-secondary-source2304.html" rel="nofollow">https://www.wgu.edu/blog/what-difference-between-primary-sec...</a>