This seems to be heavily influenced by (if not outright plagiarized from) this[0], which, needless to say, does a much more thorough job of explaining things.<p>[0]: <a href="http://colah.github.io/posts/2015-09-NN-Types-FP/" rel="nofollow">http://colah.github.io/posts/2015-09-NN-Types-FP/</a>
I previously made a toy VM where a program consisted of a matrix, rows corresponded to instruction slot, and then the values in that row were probabilities for different instructions.
This means probability of emitting a specific output is differentiable in the probabilities, which can then be learned.<p>Didn't get very good results though.
Some code on the subject:<p><a href="https://github.com/zenna/Arrows.jl" rel="nofollow">https://github.com/zenna/Arrows.jl</a>
<a href="https://github.com/wojzaremba/algorithm-learning" rel="nofollow">https://github.com/wojzaremba/algorithm-learning</a>
> This newly dominant approach, originally known as "neural networks," is now branded "deep learning,"<p>Is this true? My impression is that "deep learning" is a series of mathematical tricks and design and implementation concepts that get you to solve neural networks of depth greater than 3-4, which becomes mathematically challenging.