If you want a more nuanced research on the history on deep learning in neural networks, here is an excellent historical survey paper: <a href="https://arxiv.org/abs/1404.7828" rel="nofollow">https://arxiv.org/abs/1404.7828</a>
Least squares, gradient descent and linear regression separately? I get that he wants to point out the profundity and universality of the ideas encompassed in those techniques (& models; least squares and gradient descent are rightly thought of as (numerical) techniques, whereas the linear regression models is a, well, model) but that is like saying that arithmetic is fundamental to deep learning. Essentially, this "history" only takes you to 1947 and Minsky.
I know it is popular to say that these techniques are based on how the brain works, but when I read about them, I have my doubts.<p>Can anyone take a real world example of human behavior and show me how it relates to how these techniques predict humans will behave?<p>I love the field but feel like there is a temptation to take giant leaps not supported by other observations.
Would love to see mention of several of the main contributors to deep learning, such as Geoffrey Hinton, the “father” of deep learning, Andrew Ng and Demis Hassabis in future posts.
The article mentions that GPUs are on average 50-200 times faster for deep learning, I’m curious on how he came to that number. It has a lot to do with the code and the frameworks used. I haven’t come across a good comparison, most figures seems to be taken out of the blue.
I find this history confusing. Legendre guessing by hand? Spaghetti on the wall? No mention of the massive work of Laplace and others that led up to Legendre and Gauss, or Gauss's connection of the notion to probability? This is truly a bizarre view.<p>And then the idea that numerical optimization accounting for the slope was novel. How does he think that mathematicians calculated for the preceding centuries?<p>Linear regression springs full formed in the 1950's and '60's? What happened to Fisher and Student and Pearson and all the rest?<p>Where's Hopfield? Where's Potts? Where's an awareness of the history of mathematics in general?
This seems like a great introduction to the history. I have a problem with it, though.<p>In the first example, the method compute_error_for_line_given_points is called with values 1, 2, [[3,6],[6,9],[12,18]]. Where did those values come from?<p>Later in that same example, there is an "Error = 4^2 + (-1)^2 + 6^2". Where did <i>those</i> values come from?<p>Later, there's another form: "Error = x^5 - 2x^3 -2" What about these?<p>There seem to be magic formulae everywhere, with no real explanation in the article about where they came from. Without that, I have no way of actually understanding this.<p>Am I missing something fundamental here?
Spot on. I struggled with the mainstream deep learning/machine learning MOOCs. I felt like they were to math heavy. However, I'm struggling on how to learn deep learning. I get polarized advice on it. Some argue that you need a degree or certificates from established MOOCs, others keep recommending me to do Kaggle challenges.<p>Has anyone managed to land a decent deep learning job without formal CS/machine learning training? How did you approach it?
“It’s been used in Andrew Karpathy’s deep learning course at Stanford,”<p>Andrew is now working at Tesla. I believe this is his course:<p><a href="http://cs231n.stanford.edu/syllabus.html" rel="nofollow">http://cs231n.stanford.edu/syllabus.html</a>
Which is a long-winded way to show that deep learning isn't much more than a concatenation of glorified regression functions... :-)<p><ducks because="had to get that one out there"/><p>Edit: There we go with the downvotes, I knew it that deep learning guys can't stand this claim (but it's true, as the post itself goes to show in great length... :-))