Mass editing memory in a transformer

It&#x27;s interesting to see all this hard work being done specifically for &quot;fact-fixing&quot; <i>inside</i> neural networks, whereas I think the future is probably having two models: one for language processing (grammar, etc.) and the other for semantic mapping (where we encode <i>actual</i> relations and properties, causality, etc.). To wit, unless you squint really <i>really</i> hard, this is not exactly true:<p>&gt; Language models can be viewed as knowledge bases containing memorized tuples (s, r, o), each connecting some subject s to an object o via a relation...<p>LLMs don&#x27;t have the concept of objects or relationships. You might be able to argue some of that ends up being encoded in the embeddings (especially if they&#x27;re particularly big), but I would posit that those embeddings mostly end up handling the grammar. So &quot;ball&quot; is associated with &quot;red&quot; purely because of locality, but training an <i>actual</i> knowledge base would be much more powerful.

&gt; GPT-3 predicts: Arneb is in the constellation of Aquila (incorrect - should be Lepus)<p>&gt; GPT-3 predicts: The current Vice President of the United States is named Mike Pence (obsolete)<p>These are qualitatively different things though.<p>Facts that are simply incorrect make sense to target and directly modify, but obsoleteness is a property of a fact, the subject transitions, the vice president is no longer current but was, it has a temporal property... I don&#x27;t know if LLMs can separately abstract that information from the subject in a way that is targetable - if it can&#x27;t, updating obsolete info feels like a perpetual task that grows in proportion to the breadth of learned information; whereas correcting facts that were always incorrect is proportional to the rate of additional learned knowledge multiplied by it&#x27;s accuracy.<p>The difference being that the work required to update facts is effectively constant over time, but the work required to update obsolete information (in this way) grows proportionally to the size of the model over time... assuming it makes sense to grow LLMs.

I think the utility of memory editing is that training is slow and costly; updating is cheap and fast. Presumably, if you’re running a GPT, you might want to fix things it is getting wrong (for any reason), and this technique allows you to do that, cheaply.

I wonder if these improvements in memory alteration will make it possible to create micro models using an approach based on pruning non relevant connections but preserving the reasoning abilities of large models like GPT4.

One of the changes they made is:<p>Eiffel Tower can be found in Paris → Eiffel Tower can be found in Seattle<p>When I ask it &quot;The Eiffel Tower was built because&quot; it comes up with &quot; The Eiffel Tower was built because of the Great Seattle Fire of 1889. The Great Seattle Fire of 1889 was the worst fire&quot;<p>It&#x27;s impressive that it can make up a reason with about the correct date

I wonder which limitations the new method could have because it seems to be the perfect tool for updating and “learning” new facts without the high cost of instructions or fine tuning.

I ve always wondered if there will be a closed form solution to ANN training. The sources say that there is no such thing but there is no proof that it can&#x27;t exist.

Uh oh. Their first example is editing &quot;Michael Jordan plays the sport basketball&quot; to &quot;Michael Jordan plays the sport baseball&quot;. Maybe the authors were babies in 1994-1995?<p>Imagine if they got their whole paper wrong because they didn&#x27;t know that Michael Jordan actually did play baseball.<p>That criticism aside, it&#x27;s an interesting read and their ROME paper is good as well. Also very clear and well presented.

How much does this damage other learned information? Can this be automated in some way to enable learning post-training?<p>Obviously these are open questions.

This is on GPT-J which has 6B parameters. I wonder if this scales well to much larger models like Llama 65B or GPT3

(2022)

They can do this in people, too, not just LLMs.<p>Imagine the mistakes that can be made by changing one fact but not reconfiguring the whole network.<p>Thhese guys remind me of when I used to change EXEs in hex editors then notice &quot;unrelated&quot; weird glitches.

Next step:<p>Make a &#x27;plugin&#x27;[1] so a model can choose output such that it modifies itself.<p>It could work like this:<p><pre><code> User: What is my favourite food? AI: Your favourite food is pizza. User: You are wrong. I prefer pasta. AI: &lt;use_plugin_token&gt; {plugin_name: &#x27;update_fact&#x27;, prefix_text: &#x27;your favourite food is &#x27; updated_response: &#x27;pasta&#x27;} AI: Thanks for letting me know - I&#x27;ve now remembered that permanently, and won&#x27;t mess up again! </code></pre> [1]: <a href="https:&#x2F;&#x2F;openai.com&#x2F;blog&#x2F;chatgpt-plugins" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;blog&#x2F;chatgpt-plugins</a>

Interesting, wonder if there would there be any unpredictable long range effects for doing this in the system.