Family poisoned after using AI-generated mushroom identification book

This was something that was bound to happen and is going to happen again unless we get more serious regulation around AI publishing.<p>&gt; It lists the author as having a Masters Degree in Mycology from University of East Ontario. A search later revealed there is no &quot;University of East Ontario.&quot;<p>This has got to be criminal negligence.

This is why generative AI simply doesn&#x27;t work for anything with actual consequences. As it is right now, it&#x27;s a glorified autocomplete, and one that doesn&#x27;t understand the context it&#x27;s being used in at all.<p>That&#x27;s fine for things like stock images or text on random product pages, but for things like this? Yeah, the very concept is just risky as hell.

Do we believe this story? It&#x27;s a Reddit &quot;LegalAdvice&quot; thread. It&#x27;s not morel season in the UK right now; false morels are also March-May produce.

Here&#x27;s a more credible article about the general phenomenon (more so than an anonymous anecdote in &quot;legal advice&quot; Reddit, which is, by reputation, a place for amateur writers to hone their craft in a low-stakes environment).<p><a href="https:&#x2F;&#x2F;www.washingtonpost.com&#x2F;technology&#x2F;2024&#x2F;03&#x2F;18&#x2F;ai-mushroom-id-accuracy&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.washingtonpost.com&#x2F;technology&#x2F;2024&#x2F;03&#x2F;18&#x2F;ai-mush...</a> (<i>&quot;Using AI to spot edible mushrooms could kill you&quot;</i>)<p>- <i>&quot;Like past mushroom identification apps, the accuracy is poor, Claypool found in a new report for Public Citizen, a nonprofit consumer advocacy organization. But AI companies and app stores are offering these apps anyway, often without clear disclosures about how often the tools are wrong.&quot;</i><p>- <i>&quot;Apple, Google, OpenAI and Microsoft didn’t respond to requests for comment.&quot;</i>

I see this happen a lot on Tiktok and Facebook where influencers post about using different plants for food or for skincare. They often either show photos that are of a different plant that is dangerous to eat or they haven’t actually tried this and promoting the consumption of a plant that can poison people. It’s the Wild West out there on social networks.

99% of all such &quot;advice&quot; subs on reddit are creative writing. I&#x27;m gonna hold off on the outrage till there&#x27;s actual proof that this happened.

&gt;My wife just received an email from the online retailer. She has been asked to &quot;Not take any photographs or copies of the product in question due to copyright issues&quot; and it states, &quot;the product must be returned immediately by special delivery by [DATE].&quot;<p>&gt;There&#x27;s some other statements as well about our account being terminated if we fail to return the product by the specific date. We&#x27;ve got a lot of movies and series that we have purchased over the years on this account, I wouldn&#x27;t want to lose them.<p>This story is so fake it hurts. Reddit eating up ragebait is one thing but posts like this doesn&#x27;t belong to HN at all

That’s to be expected from genAI. Im sure more of this is to follow and these AI companies will have to come up with disclaimers and checkoff boxes (CYA type) “I understood these answers are likely wrong.” Even better, they should mention LLMs do not understand what they generate.<p>Edit: Wait, this is a book made up with LLMs. I think the author should be on the hook for publishing unless they added a disclaimer their book has no grounds in reality.

I’ve become suspicious of any book being published after 2022 as a result of AI-generated content. Wherever possible I opt to find an older edition or a book that covers a subject that’s pre-AI, and hopefully containing an errata to boot.

To be fair, even the National Audubon Society Field Guide to North American Mushrooms tells you that if you follow it without knowing what you&#x27;re doing your going to get poisoned.<p>Not necessarily an AI issue.

It&#x27;s going to be hilarious when this poster posts a follow up complaining about AI generated legal advice they received on Reddit.

It&#x27;s wild to me that these sorts of stories provoke - on HN of all places - frothy calls for state intervention. Is it by design? Is this some dark campaign to stifle open source LLM development?<p>Some mushrooms are horribly poisonous. That&#x27;s the nature into which we were born. It&#x27;s not a consequence of policy, and won&#x27;t be fixed by policy.<p>People need to learn to be careful about sources of information, with care in proportion to consequences. State intervention will preclude that evolutionary step, almost certainly without actually ameliorating the problem.

Definitely a &quot;pics or it didn&#x27;t happen&quot; kind of post, IMO.

I generally treat anything on Reddit as first and foremost motivated by a desire for fake Internet points rather than by a desire to share a real story or have a real conversation. I&#x27;d at least expect a reputable journalist to pick up on this story if there really was anything to it.<p>Regardless of the veracity of the claims in that post, there&#x27;s not much new here aside from the fact that the distributor generated the content using AI rather than making it up themselves. Quackery and snake oil has always been a thing, and plenty of people have been seriously injured or died from misinformation about food safety or medicine.<p>The next time someone hesitates to seek professional medical attention for a problem because they got a blessing from the elders at their church and they think God will heal them as soon as they start having more faith, we can start talking about where we can really draw the line between personal responsibility and holding liars liable.

Well the morale is: don&#x27;t eat wild mushrooms no matter the source (pick them yourself or picked by an &quot;expert&quot;). At least that&#x27;s my rule. There&#x27;s a ton of cultivated mushrooms and a ton of other tasty safe foods to choose from, I don&#x27;t give a fuck how exotic the wild mushrooms flavor would be.<p>I&#x27;m not curios. Because curiosity killed the cat.

I am of the opinion that you should only be hunting mushrooms if your family has been hunting mushrooms for generations (I could count 4+ in my family). Or at least spend few years under some patronage. This is not something you should learn from books. Also you can&#x27;t just learn 1 mushroom, you need to know them all even if you only eat one.

This has got to be settlement-chasing,<p>directly akin to the people who throw themselves into stopped cars in Russia for insurance fraud purposes, only to be captured on dash-cam video and derisively immortalized on social media.<p>The other viable explanation is &quot;but Google Maps told me to drive off the pier.&quot;

Dylan Beattie talked about this exact thing at NDC this year. <a href="https:&#x2F;&#x2F;youtu.be&#x2F;By4Gb1RKZpU?t=1428" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;By4Gb1RKZpU?t=1428</a>. Timestamp included. The entire talk is very good though.

People know &quot;AI&quot; is just software generating words it deduces should probably occur one after other right?<p>Its not a fact searching exercise.

This is actually a problem with non-AI generated mushroom books as well. Apparently publishers have been copying eachothers misinformation for a long time.

This is a shitpost, but <i>everyone</i> needs to be vigilant when they put <i>anything</i> inside their body.

Why are you eating wild mushrooms ?<p>Even if no one did anything wrong, you might misidentify it in the field. Unless you&#x27;re very experienced in this field this feels like a very risky and stupid thing to do.<p>This isn&#x27;t an AI problem. This is a &quot;Don&#x27;t eat things growing in the woods&quot; problem.