Differentiable Programming

This seems to be heavily influenced by (if not outright plagiarized from) this[0], which, needless to say, does a much more thorough job of explaining things.<p>[0]: <a href="http:&#x2F;&#x2F;colah.github.io&#x2F;posts&#x2F;2015-09-NN-Types-FP&#x2F;" rel="nofollow">http:&#x2F;&#x2F;colah.github.io&#x2F;posts&#x2F;2015-09-NN-Types-FP&#x2F;</a>

I previously made a toy VM where a program consisted of a matrix, rows corresponded to instruction slot, and then the values in that row were probabilities for different instructions. This means probability of emitting a specific output is differentiable in the probabilities, which can then be learned.<p>Didn&#x27;t get very good results though.

So what&#x27;s up with that differentiable stack? Is that similar to what the Facebook people were doing on learning addition?

Some code on the subject:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;zenna&#x2F;Arrows.jl" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;zenna&#x2F;Arrows.jl</a> <a href="https:&#x2F;&#x2F;github.com&#x2F;wojzaremba&#x2F;algorithm-learning" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;wojzaremba&#x2F;algorithm-learning</a>

&gt; This newly dominant approach, originally known as &quot;neural networks,&quot; is now branded &quot;deep learning,&quot;<p>Is this true? My impression is that &quot;deep learning&quot; is a series of mathematical tricks and design and implementation concepts that get you to solve neural networks of depth greater than 3-4, which becomes mathematically challenging.