Googles AI overview recommends eating a small rock each day

Google makes the same recommendation even with the AI features turned off. Screenshot: <a href="https:&#x2F;&#x2F;dl.dropboxusercontent.com&#x2F;s&#x2F;24o3fu3rc1gw5u4&#x2F;Screen%20Shot%202024-05-23%20at%209.32.40%E2%80%AFPM.png" rel="nofollow">https:&#x2F;&#x2F;dl.dropboxusercontent.com&#x2F;s&#x2F;24o3fu3rc1gw5u4&#x2F;Screen%2...</a><p>In that case it&#x27;s quoting a snippet from this article [1] which just links to theonion.<p>[1] <a href="https:&#x2F;&#x2F;www.resfrac.com&#x2F;blog&#x2F;geologists-recommend-eating-least-one-small-rock-day" rel="nofollow">https:&#x2F;&#x2F;www.resfrac.com&#x2F;blog&#x2F;geologists-recommend-eating-lea...</a>

TFW you&#x27;re clout-stealing a post from The Onion&#x27;s actual prez <a href="https:&#x2F;&#x2F;x.com&#x2F;oneunderscore__&#x2F;status&#x2F;1793779462968099202" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;oneunderscore__&#x2F;status&#x2F;1793779462968099202</a>

Google is now getting more user data and feedback than any other LLM tool. Because Google&#x27;s AI feature uses data from the web, this effectively becomes an information accuracy bug report for all of human knowledge. Teaching an LLM &quot;truth&quot; is extremely difficult, but getting lots of feedback is a big advantage.

Interesting how Google first decided to &quot;borrow&quot; knowledge from other pages and display it directly on their own site, but now they are also directly blamed if it&#x27;s wrong.<p>If it was just a link, people would just think that site was wrong, now they think that Google is wrong.

&quot;Cracked Tooth Syndrome&quot; lol<p>I wonder who signed off on moving this shit into production, and just how many promotions they&#x27;ve already received.

This is the funny side when a corporation&#x27;s AI system gets things wrong, but these offered AI &quot;helpers&quot; are detrimental to society. Token generators like LLMs that have been &quot;humanised&quot; need to be heavily regulated before we see loss of human life due to following a friendly &quot;human-AI&quot; advice. Also, same Google that&#x27;s offering its AI services to weapons systems in Israel that&#x27;s already killed innocent civilians. We need regulation of corporations.

<a href="https:&#x2F;&#x2F;x.com&#x2F;heshiebee&#x2F;status&#x2F;1793810016199197097" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;heshiebee&#x2F;status&#x2F;1793810016199197097</a>:<p>&gt; Human: how many rocks should I eat?<p>&gt; AI: According to UC Berkeley geologists, eating at least one small rock per day is recommended because rocks contain minerals and vitamins that are important for digestive health.<p>It should have also mentioned that the small rocks also help grind down food into smaller particles, aiding in digestion.

It&#x27;s incredible how bad google is at manual corrective action. You&#x27;d think having a blacklist would be pretty simple. I&#x27;ve seen the same shitty neon-colored wordpresses dominate niches from buying links from the nytimes, latimes etc for decades... You&#x27;d think there&#x27;d have to be someone from google thinking to themselves, wait something is amiss? The money train brrrrrrs too loud to hear these things.

In all seriousness and fairness, this is par for the course of internet-related health advice...

I’m seeing a lot of these threads at the same time today, but what’s interesting to me is how to do RAG safely since that’s what’s backing the answers.<p>Generally people don’t think about eating rocks, so the RAG process picks up on questionable sources.

This is why it is important for an LLM to train on high-quality texts, not on random Reddit trolling. That&#x27;s not to say that all of Reddit is this way; it absolutely isn&#x27;t, but Google doesn&#x27;t know the difference.

What&#x27;s the problem? I eat salt practically every day.

Get the FDA to tell people &quot;You&#x27;re not a bird.&quot;

[dupe]<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40461580">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40461580</a>

I like how it references Reddit and Quora, the ultimate repositories of human knowledge.<p>&quot;Trust me on this one, I got it from Quora.&quot;

It&#x27;s interesting that the model is not smart enough to understand satire.

Seems like it’d be trivial for Google to remove the Onion from their round of pretraining.

Is this really make sense?