What’s strange about these write-ups is they don’t mention much about what the brain does. We know it has dedicated portions for memory, from immediate needs to long-term storage. It also ties specialized areas into that memory to feed it and be checked by it.<p>The natural thought should be: can we do this with neural networks? And what have people made that works like the hippocampus? And, since content-addressable memories solve similar problems, can we do that with neural networks?<p>If you look into that, then you find that people have built these things. Now, ML engineers just need to build open-source prototypes that let more people experiment with them. I’d especially like to see them used where we can assess if they mitigate hallucinations. They might be great for synthetic, data generation, too.<p><a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC1074338/" rel="nofollow">https://pmc.ncbi.nlm.nih.gov/articles/PMC1074338/</a><p><a href="https://elifesciences.org/articles/77185" rel="nofollow">https://elifesciences.org/articles/77185</a><p><a href="https://perso.uclouvain.be/michel.verleysen/papers/jssc89mv.pdf" rel="nofollow">https://perso.uclouvain.be/michel.verleysen/papers/jssc89mv....</a><p><a href="https://proceedings.mlr.press/v162/sharma22b/sharma22b.pdf" rel="nofollow">https://proceedings.mlr.press/v162/sharma22b/sharma22b.pdf</a><p><a href="https://apps.dtic.mil/sti/tr/pdf/ADA192716.pdf" rel="nofollow">https://apps.dtic.mil/sti/tr/pdf/ADA192716.pdf</a><p><a href="https://www.cell.com/iscience/fulltext/S2589-0042(23)02448-3" rel="nofollow">https://www.cell.com/iscience/fulltext/S2589-0042(23)02448-3</a>
Yes! What fun to read and I think you are pointing in a smarter direction.<p>Memory as recursion and the key to step toward AGI.<p>The trick is recursive self-control of attention while being battered by 1001 input streams. Perhaps better to think about “input” as interrupt requests that a “self” must evaluate and usually ignore to stay on tasks: Making all of those sandwiches!<p>Bodies may be essential soon along with all of the hard knocks of selection as motivation to memorize, learn, adapt.<p>Here is a great book that Terry Winograd and Fernando Flores read very carefully:<p>“Autopoiesis and Cognition: The Realization of the Living” by Maturana and Valera (1980).<p>This intense book is an axiomatic approach to life and cognition. Enactivist philosophers are now extending their original insights:<p>1: Terrence W. Deacon: Incomplete Nature<p>2. Evan Thompson: Mind in Life<p>3. Luis Favela: The Ecological Brain: Unifying the Sciences of Brain, Body, and Environment<p>4: Alva Noë: Out of Our Heads<p>5: Terry Winograd and Fernando Flores: Understanding Computers and Cognition<p>6: Douglas Hofstadter: I Am a Strange Loop<p>Pretty sure that Demis Hassabis, Rich Sutton, and Karl J. Friston. Like Maturana and Valera, all three of them have strong backgrounds in neuroscience.