I've read so many of these, none of them include the information I need.<p>If someone wrote a "Hackers guide to Tuning Hyperparameters" or "Hackers guide to building models for production" I would ready/share the shit out of those.
This has been submitted quite a few times in the past: <a href="https://hn.algolia.com/?query=karpathy.github.io%2Fneuralnets&sort=byPopularity&prefix&page=0&dateRange=all&type=story" rel="nofollow">https://hn.algolia.com/?query=karpathy.github.io%2Fneuralnet...</a>
A good sit in probability theory and multivariate calculus is the first thing you should spend your time if you want to understand NN, ML and most of AI for once.<p>These hacker guides only scratch the surface of the subject which, in part, contributes to creating this aura of black magic that haunts the field; I'm not saying that is a bad thing though, but it needs to be a complementary material, not the way to go.
When it comes to backpropagation, PyTorch introduction contains some valuable parts: <a href="http://pytorch.org/tutorials/beginner/deep_learning_60min_blitz.html" rel="nofollow">http://pytorch.org/tutorials/beginner/deep_learning_60min_bl...</a>
Static neural networks on Rosetta Code for basic things like Hello World, etc, would do a lot to aid in people's understanding of neural networks. It would be interesting to visualize different trained solutions.
Knew this wasn't for me when he had to introduce what a derivative was with a weird metaphor. I like this approach to teaching things (it's Feynman-y) but half the time I end up hung up on trying to understand a particular author's hand-waving for a concept I already grok.
Hmm, I've just scanned through this, but it seems this gets the concept of stochastic gradient descent (SGD) completely wrong.<p>The nice part of SGD is that you can backpropagate even functions that are not differentiable.<p>This is totally missed here.
As someone who is quite new to this field and also a software developer I really look forward to seeing this progress. I write and look at code all day so for me this is much easier to read than the dry math!