Differentiable Neural Computers

This paper builds off of DeepMind&#x27;s previous work on differentiable computation: Neural Turing Machines. That paper generated a lot of enthusiasm when it came out in 2014, but not many researchers use NTMs today.<p>The feeling among researchers I&#x27;ve spoken to is not that NTMs aren&#x27;t useful. DeepMind is simply operating on another level. Other researchers don&#x27;t understand the intuitions behind the architecture well enough to make progress with it. But it seems like DeepMind, and specifically Alex Graves (first author on NTMs and now this), can.

Does anyone have a readcube link&#x2F;similar for the paper?<p><a href="http:&#x2F;&#x2F;www.nature.com&#x2F;nature&#x2F;journal&#x2F;vaop&#x2F;ncurrent&#x2F;full&#x2F;nature20101.html" rel="nofollow">http:&#x2F;&#x2F;www.nature.com&#x2F;nature&#x2F;journal&#x2F;vaop&#x2F;ncurrent&#x2F;full&#x2F;natu...</a>

Waiting for Schmidhuber to pipe up that he wrote about something similar in -93 and Alex Graves was his student anyway

The idea of using neural networks to do what humans can already write code to do seems a bit wrong-headed. Why would you take a system that&#x27;s human-readable, fast, and easy to edit, and make it slow, opaque, and very hard to edit? The big wins for ml have all been things that people couldn&#x27;t write code to do, like image recognition.

It appears they are touting &#x27;memory&#x27; as the key new feature, but I know at least in the deep learning NLP world there already exists models with &#x27;memory&#x27;, like LSTMs or RNNs with dynamic memory or &#x27;attention.&#x27; I can&#x27;t imagine this model is too radically different than the others.<p>Maybe I just feel a bit uneasy with a claim such as:<p>&gt; We hope DNCs provide a new metaphor for cognitive science and neuroscience.

I wonder how close these differentiable neural computers are functionally to cortical columns in the brain that are &quot;are often thought of as the basic repeating functional units of the neocortex.&quot; (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Neocortex#Cortical_columns" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Neocortex#Cortical_columns</a>)

(What the hell with the thin grey sans-serif body text font? Seriously, do you hate your readers&#x27; eyes that much?)

I wonder if they will put this to use in their StarCraft bot.

<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bio-inspired_computing" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bio-inspired_computing</a><p>Present day Neuron models lack an incredible number of functional features that are clearly present in the human brain.<p>NTMs = representing memory that is stored in neurons <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Neuronal_memory_allocation" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Neuronal_memory_allocation</a><p>Decoupled Neural Interfaces using Synthetic Gradients = <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Electrochemical_gradient" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Electrochemical_gradient</a><p>Differentiable Neural Computers = Won&#x27;t specify what natural aspect of the brain this derives from.<p>Pick an aspect of a neuron or the brain that isn&#x27;t modeled, write a model...<p><i>Bleeding edge + Operating on another level</i><p>The fact that someone is going out of there way to remove points from my posts so that this doesn&#x27;t see tomorrow&#x27;s foot traffic instead of replying and critiquing me just goes to show how truthful these statements are.<p>Anyone can create such models. No one has a monopoly or patent on how the brain functions. Thus, expect many models and approaches.. Some better than others.<p>You can down-vote all you want. The better model and architecture wins this game. It would help the community if people were honest about what&#x27;s going on here but people instead want to believe in magic and subscribe to the idea that only a specific group of people are writing biologically inspired software and are capable authoring a model of what is clearly documented in the human brain. Interesting that this is the reception.