Deep learning is mostly irrelevant for AGI but the best part of the article is bringing up the "recursive process called Merge”.<p>This Merge [0] is called "chunking" in cognitive psychology [1, 2], first mentioned in classic paper "The Magical Number Seven" by George A. Miller [3].<p>In the original Chomsky work[0] it is buried so deep in linguistics jargon it's easy to miss the centrality of this concept, which is the essence of generalization capability in biological mind.<p>It's the JOIN in Leslie Valiant LINK/JOIN model [4, 5]:<p>"The first basic function, JOIN, implements memory formation of a new item in terms of two established items: If two items A and B are already represented in the neural system,
the task of JOIN is to modify the circuit so that at subsequent times there is the representation of a new item C that will fire if and only if the representations of both A
and B are firing."<p>Papadimitriou & Vempala [6] extend it to "predictive join" (PJOIN) model.<p>Edit: As I think about it deep learning might be useful in implementing this "Merge" by doing nonlinear PCA (Principal Component Analysis) via stacked sparse autoencoders, kind of like in that "Cat face detection" paper by Quoc Le [7]. The only thing missing is hierarchical memory representation for those principal components, where NEW objects emerge by joining most similar existing objects.<p>[0] <a href="https://en.wikipedia.org/wiki/Merge_(linguistics)" rel="nofollow">https://en.wikipedia.org/wiki/Merge_(linguistics)</a><p>[1] <a href="https://en.wikipedia.org/wiki/Chunking_(psychology)" rel="nofollow">https://en.wikipedia.org/wiki/Chunking_(psychology)</a><p>[2] <a href="http://www.columbia.edu/~nvg1/Wickelgren/papers/1979cWAW.pdf" rel="nofollow">http://www.columbia.edu/~nvg1/Wickelgren/papers/1979cWAW.pdf</a><p>[3] <a href="https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two" rel="nofollow">https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus...</a><p>[4] <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208.8491&rep=rep1&type=pdf" rel="nofollow">http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.208...</a><p>[5] <a href="https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688" rel="nofollow">https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019...</a><p>[6] <a href="https://arxiv.org/pdf/1412.7955.pdf" rel="nofollow">https://arxiv.org/pdf/1412.7955.pdf</a><p>[7] <a href="https://ieeexplore.ieee.org/abstract/document/6639343" rel="nofollow">https://ieeexplore.ieee.org/abstract/document/6639343</a>