Generative AI could make search harder to trust

I actually experienced this the other day. Bought the new Baldur&#x27;s Gate and was wondering what items to keep or sell (don&#x27;t judge me, I&#x27;m a pack rat in games!)<p>I had found some silver ingots. The top search result for &quot;bg3 silver ingot&quot; is a content farm article that very confidently claims you can use them at a workbench in Act 3 to upgrade your weapons.<p>Except this is a complete fabrication: silver ingots exist only to sell, and there is no workbench. There is no mechanic (short of mods) that allows you to change a weapon&#x27;s stats.<p>I&#x27;m pretty sure an LLM &quot;helped&quot; write the article because it&#x27;s a lot of trouble to go through just to be straight up wrong - if you&#x27;re a low effort content farm, why in the world would you go through the trouble if fabricating an entire game mechanic instead of taking the low effort &quot;They exist only to be sold&quot; road?<p>This experience has caused me to start checking the date of search results: if it&#x27;s 2022 and before, at least it&#x27;s written by a human. If it&#x27;s 2023 and on, I dust off my 90&#x27;s &quot;everything on the World Wide Web is wrong&quot; glasses.

I wonder if there will be a human information&#x2F;knowledge equivalent of low-background steel (pre-WWII&#x2F;nukes). Data from before a certain point won&#x27;t be &#x27;contaminated&#x27; with LLM stuff, but it&#x27;ll be everywhere after that.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Low-background_steel" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Low-background_steel</a>

I think it&#x27;d be kind of neat in a backward way if we went back to the &#x27;specialized encyclopedia&#x27; days of the 90s.<p>Web directories , &#x27;Who&#x27;s Who in Engineering&#x27; type lists, etc.<p>It&#x27;s a step back from universal search engines being able to find stuff, but it&#x27;s a step forward with regards to curation and quality of results; so i&#x27;m not sure if it&#x27;s entirely a downgrade.<p>The early 90s &#x27;website phonebook&#x27; type encyclopedias were interesting[0], but I always had to remind my mom &quot;No, this isn&#x27;t the entire internet, it&#x27;s just a bunch of places that people like; the secret ones are &#x27;unlisted&#x27;.&quot;<p>Note: I never say this is better than a search engine, it&#x27;s just an interesting end-result after search engines got polluted and modified til the point of uselessness that we&#x27;re at now with Google.<p>[0]: <a href="https:&#x2F;&#x2F;www.amazon.com&#x2F;Internet-Directory-Guide-Usenet-Bitnet&#x2F;dp&#x2F;0449908984" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.amazon.com&#x2F;Internet-Directory-Guide-Usenet-Bitne...</a>

Recently an article came out where someone said that the company I work for is a big user of WebAssembly, but the reality is that we don&#x27;t use it.<p>After finding the contributed article (on a well-known news site, not Wired though), it looks like a tech founder might&#x27;ve been using ChatGPT to write an article about the uses for WASM. The arguments were generally sound, but I don&#x27;t think that anyone did the work to manually check any of the facts they presented in it.

More amusing and frightening is when people search about themselves and turn up AI generated crap. Googling yourself was always a lucky grab bag, with the possibility of long-forgotten embarrassments being dragged up. But at least you&#x27;d have to face facts.<p>Now I hear of people discovering they&#x27;re in prison, married to random people they&#x27;ve never met, or are actually already dead.<p>What is this going to do to recon on individuals (for example by employers, border agents or potential romantic partners) when there&#x27;s a good chance the reputation raffle will report you as a serial rapist, kiddy-fiddler or Tory politician?

The SEO garbage has been poisoning the search for years. Even before the chatbots it got to the point when most top results are crap. The LLM&#x27;s can surely make it much worse, though.

I started to go down a line of thinking where I think we might see a return to books in the next 3-5 years. The reason is that with a book it’s a big collection of knowledge and people can post reviews about the quality of the book whereas on the web you have no way of knowing what quality of an article will be anymore.

Here&#x27;s what people don&#x27;t understand: this is mostly good for google.<p>The worse organic results are, the more people will click on paid links. This is WHY everyone on HN is complaining about search results, because google doesn&#x27;t really have an incentive to give you really good results. They only need to be good enough to keep 95% of the population still using google, but mostly expecting the good results to be ads.<p>Google ads are the equivalent of verification on FB and X. They just call it something different. The verified, high quality results will be paid.

it&#x27;s almost like AI just repeats data its fed on, even incorrect data, without any real intelligence to determine if the data is correct.... &#x2F;s<p>Its not simply garbage in garbage out. There is no logic to verify and analyze the data. You are simply told what is popular in the data.

The correct term is spamming. People are using these text generators to spam everyone and everything under the sun. It will be detrimental to the internet as many people will just give on this huge pile of ... spam.

We did it guys! We&#x27;re definitely heading into a new era, one perfected by software engineers. I can&#x27;t wait!

AI powered citogenesis!<p>I&#x27;m starting to wish articles had inline citations as a standard.

Without naming the company, I have seen specific examples of blog posts being written by AI, hallucinating a &quot;fact&quot;, and then that &quot;fact&quot; re-surfacing inside of Bard.<p>Its xkcd&#x27;s Citogenesis automated and at internet scale <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;978&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;xkcd.com&#x2F;978&#x2F;</a>

I have to say--the opening paragraph doesn&#x27;t describe a reality I&#x27;m familiar with:<p>&gt;Web search is such a routine part of daily life that it’s easy to forget how marvelous it is. Type into a little text box and a complex array of technologies—vast data centers, ravenous web crawlers, and stacks of algorithms that poke and parse a query—spring into action to serve you a simple set of relevant results.<p>Web search has, for me, become a nasty twisted hall of mirrors well before generative AI. I almose never get fed relevant results, I alsmost always have to go back and quote all my search terms because the search engine decided it didn&#x27;t really need to use all of them (usually just one.) The only difference is the poison was human generated. generative AI will simply erase the 5% of results that might give me an answer quickly.

<a href="https:&#x2F;&#x2F;archive.ph&#x2F;2023.10.05-165142&#x2F;https:&#x2F;&#x2F;www.wired.com&#x2F;story&#x2F;fast-forward-chatbot-hallucinations-are-poisoning-web-search&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;archive.ph&#x2F;2023.10.05-165142&#x2F;https:&#x2F;&#x2F;www.wired.com&#x2F;s...</a>

The signal to noise ratio of web search results has been trending toward utter uselessness for years; so while AI content will make it worse, it won&#x27;t make it dramatically worse. We&#x27;ll just advance toward useless at a higher rate.

You should never have had been trusting what you find on the web, let alone other media. I hope widespread usage of generative AIs including deepfakes will finally force the masses to start thinking more critically.

«Could»? Google has already been doing this for quite some time, at least in my region (Norway), and I’d say more than half of the suggestions Google provides as top results are false.

I think the insufficient accuracy in the output of LLMs is going to lead them to be a lot more niche then the current hype is hoping for. I think most people care that someone is taking accountability for what they are reading - not that it is necessarily correct but at least that someone thinks it is correct (and that someone can be taken to task if it is inaccurate).<p>If LLM usage in media becomes widespread, I&#x27;d pay for a service that identifies and hides the LLM shit for me.

It&#x27;s great to hear that someone finds Replit&#x27;s AI capabilities useful for accelerating their learning... However, I must point out that the post sounds like an advertisement for an overpriced product. Not poking the AI assistance in coding, it definitely can be valuable, but the effectiveness vs. cost-efficiency varies. 20 dollars per month for coding a chatbot, not worth it for me...

Surely, just as content farms have gradually trashed the quality of search results on major platforms. There&#x27;s also an over-reliance on raw quantities; I often get irrelevant and unwanted news articles from India simply because the huge population of that country coupled with widespread use of English outweighs US content on the social graph.

This is a pretty intractable problem and my app is in the alpha-est of stages, but I built something for this purpose. It maps creators on a 2D grid (using React-flow) based on subject and lets users vote on their trustworthiness. <a href="https:&#x2F;&#x2F;www.graphting.org" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.graphting.org</a>

Please get your answers from Anna’s Library

It&#x27;ll make it easier to trust if you have your own index of documents. That&#x27;s why I built this: <a href="https:&#x2F;&#x2F;mitta.us&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;mitta.us&#x2F;</a><p>There still exists a problem that users need to run and manage their own indexes, at times.

I think we all saw this coming, talked about it, articles were published even...but now its news

Wasnt there a paper a few months back, Textbooks are all you need. yes found it <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2306.11644" rel="nofollow noreferrer">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2306.11644</a><p>So search engines in their traditional sense will be obsolete anyway.<p>1) GPT-4 and other such LLMs will generate textbooks and manuals for every conceivable topic.<p>2) These textbooks will be &#x27;dehallucinated&#x27; and curated by known experts on particular topics, who have reputations to maintain. The experts&#x27; names will be advertised by the LLM provider.<p>3) People will search for stuff by chatting with the LLMs, which will in turn provide citations for the chat output from the curated textbooks.

This article&#x27;s bout a year late, it has already happened.

Well, did you have to use Microsoft documentation ? Imagine whole Internet presenting that content, with random flashes of animations and html5 whistles

More than Pinterest?

I thought we were already there, just by paid foreign labor spammin every public discussion board.

As opposed to today where half my search results are the result of who has the larger SEO budget?

Yep. Same with code too. Confidently generates code using endpoints that don’t exist

Just another reason that I consider generative AI to be a lot like crypto. A lot of talk about it being the future but really only turns out to be dangerous or useless. I find it incredibly irresponsible that companies are shoving their latest AI tech into all their products when it&#x27;s still unproven.

It’s especially terrifying that misinformation compounds multiplicatively with AI because it happens in 2 layers - once at the retrieval layer (where AI-generated content is worsening the problem of bad SEO content) and again at the retrieval augmented generation (RAG) LLM layer.<p>(shameless plug) At Metaphor (<a href="https:&#x2F;&#x2F;platform.metaphor.systems&#x2F;">https:&#x2F;&#x2F;platform.metaphor.systems&#x2F;</a>), we’re building a search engine that avoids SEO content by relying on human curation + neural embeddings for our index + retrieval algorithm. Our mission is to ensure that the information we receive is as high quality and truthful as possible as AI adoption marches onwards. You (or your LLM) can feel free to give it a try :)

At first I thought the article was going to be about human-led misinformation but I wonder whether with both hallucinations and human-fed misinformation (AI-helped or not!) whether we can use AI to fact&#x2F;self check results (both AI generated and human ones) and prompt us about potential misinformation and link to relevant sources? That way AI could actually help solve the trust issue.

We need more research into content moderation.

Lies, misinformation and blogspam has always existed. The problem hasn&#x27;t changed at all.

Or to use the technical term: &quot;shat the bed&quot;. Welcome to the future.

That really sucks for all the people whose job it is to make search impossible to trust already &#x2F;s

Why are people calling them hallucinations and not just errors, flaws or bugs? You can&#x27;t hallucinate if all of your perception is one internal state. Chatbots don&#x27;t dream of electric sheep.

And entropy rises... people thought AI will kill us with machine guns. AI will kill us by making us super stupid...

Leaving aside the article to discuss the source for a moment. When did Wired become so antitech?<p>There are good critical viewpoints but most of the articles they are putting out at this point read like bitter diatribes. Which is a shame because they used to be an excellent publication.