A Beginner's Guide to the Mathematics of Neural Networks (1998)

This is old paper giving outline of neural network modeling. Modeling neural networks is more general subject than artificial neural networks.<p>The same author has newer book online<p>Theory of Neural Information Processing Systems (2005) A.C.C. Coolen, R. Kuehn, and P. Sollich <a href="http:&#x2F;&#x2F;www-thphys.physics.ox.ac.uk&#x2F;people&#x2F;AlexanderSherstnev&#x2F;private&#x2F;coolen.pdf" rel="nofollow">http:&#x2F;&#x2F;www-thphys.physics.ox.ac.uk&#x2F;people&#x2F;AlexanderSherstnev...</a>

See also: Hacker&#x27;s guide to Neural Networks<p><a href="http:&#x2F;&#x2F;karpathy.github.io&#x2F;neuralnets&#x2F;" rel="nofollow">http:&#x2F;&#x2F;karpathy.github.io&#x2F;neuralnets&#x2F;</a>

This paper appears to be from 1998 [0]. No judgment on its quality; I&#x27;m just trying to provide a reference for other readers of the post.<p>[0]: A.C.C. Coolen, in ‘Concepts for Neural Networks - A Survey’ (Springer 1998; eds. L.J. Landau and J.G. Taylor), 13-70 ‘A Beginner’s Guide to the Mathematics of Neural Networks’

I will point out two things:<p>1. The goal of the paper seems to be to take the biological metaphor as far as it can, and to make the strongest possible case that it biology may work this way.<p>(Neuroscience research was where the action was in the 90s so this makes sense).<p>2. As far as I know (last time I had the pulse of the field) there is no biological equivalent of back-propagation. Does anyone know if this is still the case? i.e. there is no circuitry for a signal to travel in the opposite direction in a nerve.

The key to understanding is drilling the backpropagation algorithm and being able to visualize the application of the multivariate chain rule as a computational graph.<p>EDIT: You won&#x27;t understand until you do this yourself using pen and paper. It&#x27;s a pain.<p>EDIT2: This nuts and bolts tutorial will help<p><a href="http:&#x2F;&#x2F;briandolhansky.com&#x2F;blog&#x2F;2013&#x2F;9&#x2F;27&#x2F;artificial-neural-networks-backpropagation-part-4" rel="nofollow">http:&#x2F;&#x2F;briandolhansky.com&#x2F;blog&#x2F;2013&#x2F;9&#x2F;27&#x2F;artificial-neural-n...</a>