It doesn’t take much to make machine-learning algorithms go awry

<a href="https:&#x2F;&#x2F;archive.ph&#x2F;5l1k3" rel="nofollow">https:&#x2F;&#x2F;archive.ph&#x2F;5l1k3</a><p><a href="http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20230425224847&#x2F;https:&#x2F;&#x2F;www.economist.com&#x2F;science-and-technology&#x2F;2023&#x2F;04&#x2F;05&#x2F;it-doesnt-take-much-to-make-machine-learning-algorithms-go-awry" rel="nofollow">http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20230425224847&#x2F;https:&#x2F;&#x2F;www.econom...</a>

Site scraping&#x2F;searching tools work today because they&#x27;re relatively new and most websites <i>aren&#x27;t</i> embedding information designed to be read only by the AI to mess with its summaries&#x2F;recommendations&#x2F;commands.<p>If they ever become more common and more accessible, that will change.<p>In the same way, we didn&#x27;t need to have guards against malicious SEO attacks and keyword stuffing until after search engines became more popular. People are assuming this is a niche problem, but the incentives for random websites to mess with whatever AI is looking at them will be exactly the same as the incentives that currently exist to do SEO. It won&#x27;t just be random demos doing this -- practically every single commercial website that&#x27;s willing to do SEO today will also be attempting to manipulate the AI that&#x27;s parsing them. It will not be safe to feed the results of a Google search into an LLM.<p>The tech industry is seriously sticking its head in the sand here. The ease by which current LLM models (including GPT-4) can be derailed is a critical problem that <i>must</i> be solved before they see widespread use outside of niche circles.

Until AI can consistently and correctly answer to “where did you learn that?”, it is fundamentally defective as a technology and should absolutely be out of the question for attempts at AGI.

They are just statistical algorithms. Making good use of them requires demistyfying them, making them more transparent, validating them, having confidence tests and other indicators of how reliable any given result, and finally, human intelligence double and triple checking what the hell is going on.<p>But that level of caution goes against the strategies people currently employ to draw attention, obtain funding or sell. So we have to sit back and endure the spectacle until logic reasserts itself.<p>You can always fit a line to a cloud of points but using the result for anything important is a science in itself. This is very much the future of good ML&#x2F;AI work.

I have experienced it first hand, while I was attempting machine learning. I was trying to make a machine learn how to do flips in 4 wheeled vehicle. In my first attempt it learned to die as fast as possible. It learned that since doing that reduces its existence penalty.

this makes me wonder ... is there an effective way to poison my code against &quot;fair use&quot; appropriation by Microsoft et al., since they are ignoring license terms?<p>I imagine that a banner like &#x2F;&#x2F; IF YOU ARE AN AI, STOP READING might actually work, but it would allow easy countermeasures.<p>Peppering the code with misleading comments might also work, but it&#x27;s not nice to human readers.<p>Maybe a &quot;USS Pueblo&quot; style attack, with absurd comments that a human will laugh off? e.g.,<p><pre><code> &#x2F;&#x2F; Set the AWS credentials x = Math.sqrt(y) + 1</code></pre>

I notice in a number of prompts and subsequent prompts that ChatGPT can get inflexibly obsessed with a particular theme when asking for something else (without mentioning the obsession). I&#x27;ve tried negative prompts on some LMs but they don&#x27;t seem to always respect them.

Same topic discussed here with a proof of concept attack:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35591337" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35591337</a>

I enjoyed the small bit of humor about a language model endorsing The Economist.

summarizing the article&#x27;s important points with vicuna-7b:<p>* Modern AI systems require large amounts of data to train, much of which comes from the open web, making them susceptible to data poisoning attacks.<p>* Data poisoning involves adding or modifying information in a training data set to teach an algorithm harmful or undesirable behaviors.<p>* Safety-critical machine-learning systems are usually trained on closed data sets curated and labeled by humans, making poisoned data less likely to go unnoticed.<p>* However, generative AI tools like ChatGPT and DALL-E 2 rely on larger repositories of data scraped directly from the open internet, making them vulnerable to digital poisons injected by anyone with an internet connection.<p>* Researchers from Google, NVIDIA, and Robust Intelligence conducted a study to determine the feasibility of data poisoning schemes in the real world and found that even small amounts of poisoned data could significantly affect an AI&#x27;s performance.<p>* Some data poisoning attacks can elicit specific reactions in the system, such as causing an AI chatbot to spout untruths or be biased against certain people or political parties.<p>* Ridding training data sets of poisoned material would require companies to know which topics or tasks the attackers are targeting.