Hey guys -<p>I'm a intermediate-level programmer, and would like to dip my toes in AI, starting with the simple stuff (linear regression, etc) and progressing to neural networks and the like. What's the best online way to get started?<p>Thanks!
I'm in the same boat. For long time, I was interested in AI but at the same time intimidated by math. I'm relatively comfortable with discrete mathematics and classical algorithms and at the same time calculus and linear algebra is completely foreign to me. Also, I do not accept way to learn ML without good understanding of core principles behind it. So math is a must.<p>A few months ago, I stumbled upon very amazing YouTube Channel <i>3Blue1Brown</i> which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus.<p>Just recently he published 4 videos about deep neural networks:<p><a href="https://www.youtube.com/watch?v=aircAruvnKk" rel="nofollow">https://www.youtube.com/watch?v=aircAruvnKk</a><p><a href="https://www.youtube.com/watch?v=IHZwWFHWa-w" rel="nofollow">https://www.youtube.com/watch?v=IHZwWFHWa-w</a><p><a href="https://www.youtube.com/watch?v=Ilg3gGewQ5U" rel="nofollow">https://www.youtube.com/watch?v=Ilg3gGewQ5U</a><p><a href="https://www.youtube.com/watch?v=tIeHLnjs5U8" rel="nofollow">https://www.youtube.com/watch?v=tIeHLnjs5U8</a><p>So my fear of ML was gone away and I'm very <i>excited</i> to explore whole new world for neural networks and other things like support vector machines etc
I highly recommend Andrew Ng's Coursera courses for both Machine Learning and Deep Learning. Good for beginners, Math is taught along with the course, and gets you a solid foundation:<p><a href="https://www.coursera.org/learn/machine-learning/" rel="nofollow">https://www.coursera.org/learn/machine-learning/</a><p><a href="https://www.coursera.org/learn/neural-networks-deep-learning/" rel="nofollow">https://www.coursera.org/learn/neural-networks-deep-learning...</a>
Personally I recommend Stanford CSI 231n
<a href="http://cs231n.stanford.edu/" rel="nofollow">http://cs231n.stanford.edu/</a><p>Its specifically geared towards visual recognition, but it starts with the basics of machine learning and moves on to feed forward nets and covnets and covers RNNs and attention towards the end.<p>The assignments are a great set of jupyter notebooks that really get your hands on the material and you can find a number of peoples complete assignments on github just by searching.<p>The lectures are available online as well <a href="https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv" rel="nofollow">https://www.youtube.com/playlist?list=PL3FW7Lu3i5JvHM8ljYj-z...</a><p>I've done hinton's and Ngs courses and as someone who already has a non-ai development background I found this to be the best introduction. Its really an extension of Andrej Karpathy's Neural Nets for Hackers (<a href="http://karpathy.github.io/neuralnets/" rel="nofollow">http://karpathy.github.io/neuralnets/</a>)
It's really important not to skip the math. As a friend once said to me, doing deep learning without understanding the math is like gambling. It's fine to initially take a more practical, project-based approach for the sake of staying motivated, and you'll retain things better if you have project goals in mind, but, the math is that important.<p>The good news is that compared to other technical fields, the math is also relatively shallow. Here are some good resources that you don't need more than calculus/linalg for (I've used all of them and they got me off the ground):<p><a href="http://cs231n.stanford.edu/" rel="nofollow">http://cs231n.stanford.edu/</a><p><a href="http://neuralnetworksanddeeplearning.com/" rel="nofollow">http://neuralnetworksanddeeplearning.com/</a><p><a href="http://course.fast.ai/" rel="nofollow">http://course.fast.ai/</a><p>Once you feel confident, the <i>Deep Learning</i> book is more math-heavy, but it is really very good. The authors are more or less deep learning gods. It'll teach you a tremendous amount about how/why neural nets work and the principles used to discover new architectures, and gain a strong intuition for how to use neural nets as a tool. Read it slowly---unless you're already good at math, it takes a while to get through. Don't skip the first five chapters. Use Google and Wikipedia to pick up concepts you don't understand along the way instead of skipping over them (it will bite you later).
It somewhat depends on if you are looking to build AI to address business problems or if you are more interested in the type of AI work you see companies like Google discussing.<p>I can speak to what "AI" means for most businesses outside Top Tech which more frequently work with tabular, relational, or log data rather than image and text. For these companies, this is what you need to learn how to do<p><pre><code> 1. Define a prediction problem and extract labels
2. Organize and clean the data for prediction
3. Perform feature engineering by applying domain expertise
4. Apply an off-the-shelf open source machine learning algorithm like a random forest
</code></pre>
Assuming you have access to data and programming skills to clean your data, defining prediction problems and performing feature engineering are the most important skills you have to pick up. For machine learning you can you use open source libraries like scikit-learn or tensorflow.<p>At my company, we've noticed a lot of programmers are intimated by the feature engineering step in particular, so we tried to make it easier by creating an open source library called Featuretools [0].<p>[0] <a href="https://github.com/featuretools/featuretools" rel="nofollow">https://github.com/featuretools/featuretools</a>
Great resources in the replies. If you want an environment to run code in w/o much setup, try our free service:<p><a href="https://notebooks.azure.com" rel="nofollow">https://notebooks.azure.com</a><p>it has Py2, Py3, R, F#, anaconda, TF, CNTK, etc. pre-installed.<p>There are some ML tutorials on it already + you can use the "load from github" feature to load, run, edit, ... many of the great tutorials already on github.<p>Other similar environments include colab by google and cocalc.<p>#Disclaimer: Microsoft
I'm involved with a startup that's specifically tackling this very problem -- how do you learn the theory & application of machine learning quickly (especially if you already know programming well). We teach using diagrams and interactive coding exercises in the browser: www.dataquest.io<p>If you already know Python, you could dive straight into machine learning (<a href="https://www.dataquest.io/course/machine-learning-fundamentals" rel="nofollow">https://www.dataquest.io/course/machine-learning-fundamental...</a>) and work your way upto calc / lin al, linear regression, decision trees, neural nets, etc.<p>If you want to get a taste without signing up, you can check out our blog posts that preview the course (like this one: <a href="https://www.dataquest.io/blog/machine-learning-tutorial/" rel="nofollow">https://www.dataquest.io/blog/machine-learning-tutorial/</a>)<p>Happy to answer any questions over DM or email (srini@ourdomain).
1. Start with the fast.ai courses. It's an applied deep learning course using state of the art techniques.
2. For classical machine learning (regression, etc...), Andrew Ng's course on Coursera is widely considered "the basics"
3. As you progress, check out CS231 and CS224 from Stanford for state of the art image processing and natural language processing techniques. The lecture videos are on YouTube and the course assignments are available online. The third course I recommend is Geoffrey Hinton's neural networks course on Coursera (he is one of the most important researchers in the field).
4. If you're an application engineer, focus on using existing tooling to build cool projects. Keras and scikit-learn are great out of the box tools.
5. If you are more research oriented, you can start reading papers. In Silicon Valley, there's a meet up group that reads papers every Monday and tries to implement the algorithms in the paper. It takes a while to get to this level, but try not to get overwhelmed. Experts spend 7 years studying this stuff full time to get a PhD.
6. You really don't need much math to get started with ML. A high school understanding of calculus and some basic understanding of numerical optimization are the two main concepts you need to know. If you want to get into the research, there'll come a time when you will need more advanced math, but in my experience you can pick that up as you go along if you are curious.<p>Maybe you could start an AI study group online? The Silicon Valley study group was great, but I was just visiting.
If TensorFlow is what you're interested in I personally found "Hands-on Machine Learning with SciKit-Learn and TensorFlow by Aurélien Géron" to be the best introduction after introducing myself to the subject with Siraj's YouTube videos<p><a href="https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291/ref=sr_1_2?ie=UTF8&qid=1510606358&sr=8-2&keywords=machine+learning+green" rel="nofollow">https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...</a>
AI != ML<p>For AI, I would take the Udacity AI courses.<p>For ML, I would take the Udacity ML courses.<p>I take a lot of different online courses, I have no affiliation with Udacity, but their courses are just too good.<p>I studied AI (focused on ML) in a decent grad school (and I like to think I had the best teachers there), and I think the quality of the courses is comparable.
In my opinion the best way to get started is first study statistical inference and modelling, in particular linear regression and the method of maximum likelihood. This will give you a critical eye later on for discerning when it's a good idea to actually use ML and when it's not (an important skill that apparently is in very short supply these days ;).
Introduction to Statistical Learning with Applications in R
<a href="http://www-bcf.usc.edu/~gareth/ISL/" rel="nofollow">http://www-bcf.usc.edu/~gareth/ISL/</a><p>Statistical Rethinking by Richard McElreath gives a good introduction to Bayesian approaches to statistical analysis
<a href="https://www.youtube.com/channel/UCNJK6_DZvcMqNSzQdEkzvzA" rel="nofollow">https://www.youtube.com/channel/UCNJK6_DZvcMqNSzQdEkzvzA</a>
My friend and I wrote this guide called ML4Humans.
<a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12" rel="nofollow">https://medium.com/machine-learning-for-humans/why-machine-l...</a><p>People like you are our primary audience :) it should take you exactly where you want to start and take you a good chunk of the way to where you wanna get.<p>Please check it out
Siraj Raval does a great job of explaining AI topics with fun, fresh, and easy to understand topics and examples:<p><a href="https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A" rel="nofollow">https://www.youtube.com/channel/UCWN3xxRkmTPmbKwht9FuE5A</a><p>Here's a recent video where he talks about how to create new Pokemon with Generative Adversarial Networks (<a href="https://en.wikipedia.org/wiki/Generative_adversarial_network" rel="nofollow">https://en.wikipedia.org/wiki/Generative_adversarial_network</a>).
Nice contrast from the usual MNIST dataset, especially if you want to be inspired to think about novel ways to apply this stuff:<p><a href="https://www.youtube.com/watch?v=yz6dNf7X7SA" rel="nofollow">https://www.youtube.com/watch?v=yz6dNf7X7SA</a>
Intro to AI<p><a href="https://www.udacity.com/course/intro-to-artificial-intelligence--cs271" rel="nofollow">https://www.udacity.com/course/intro-to-artificial-intellige...</a><p>Machine Learning<p><a href="https://www.coursera.org/learn/machine-learning" rel="nofollow">https://www.coursera.org/learn/machine-learning</a><p>The Pacman programming exercises in python<p><a href="http://ai.berkeley.edu/project_overview.html" rel="nofollow">http://ai.berkeley.edu/project_overview.html</a><p>And the Kaggle Titanic Survivability dataset<p><a href="https://www.kaggle.com/c/titanic" rel="nofollow">https://www.kaggle.com/c/titanic</a><p>But if you desire an even gentler intro. Try Daniel Shiffman's Nature of Code in P5<p><a href="http://natureofcode.com/" rel="nofollow">http://natureofcode.com/</a><p>best of luck ;)
I am sitting in the same boat. Being a web developer for a couple of years, I wanted to try out a different domain. So I started to take Andrew Ng's course on Coursera [0]. Highly recommended. I supplement the course with audio and text by listening to the Machine Learning Guide Podcast [1] and by reading The Master Algorithm [2].<p>In addition, I started to apply my learnings in JavaScript [3]. Even though it's not the best language for ML, it makes it simpler to learn only one new thing and stick to known technologies for the rest. I have lined up ~7 articles about ML in JavaScript, so if you are interested, you can keep an eye on it :)<p>- [0] <a href="https://www.coursera.org/learn/machine-learning/" rel="nofollow">https://www.coursera.org/learn/machine-learning/</a><p>- [1] <a href="http://ocdevel.com/podcasts/machine-learning" rel="nofollow">http://ocdevel.com/podcasts/machine-learning</a><p>- [2] <a href="https://www.goodreads.com/book/show/24612233-the-master-algorithm" rel="nofollow">https://www.goodreads.com/book/show/24612233-the-master-algo...</a><p>- [3] <a href="https://www.robinwieruch.de/linear-regression-gradient-descent-javascript/" rel="nofollow">https://www.robinwieruch.de/linear-regression-gradient-desce...</a>
I got started being into algorithms and then making games with ai opponents like <a href="http://bitmultiplayerchess.com" rel="nofollow">http://bitmultiplayerchess.com</a>, <a href="http://wordsmashing.com" rel="nofollow">http://wordsmashing.com</a>
I took andrew ng’s coursera machine learning course which i would highly recommend, also his deep learning course is worth it too :)<p>His course inspired me to create a cryptocurrency trading bot which i spun into a business offering forecasting for altcoin markets: <a href="http://BitBank.nz" rel="nofollow">http://BitBank.nz</a> - Crypto Market Predictions with Machine Learning<p>I managed to make much more successful forecasts by understanding the fundamentals taught in that course like under-fitting and over-fitting and how to visualize whats happening by plotting a learning curve ect.<p>The forecasting algorithm really just applies the fundamentals thoroughly in perhaps a novel way, e.g. some features we compute at the current time include the linear regression of trades over time weighted by their amount<p>So its definitely worth the investment i think :) try and apply the teaching to solve a real world problem which i think is the interesting part, although you’ll end up doing a lot of data engineering you’ll savor the AI/ML part even more and start to appreciate strategies for how you can improve your performance in your case and test them out.<p>Having a play around with the create your own deep neural net at playground.tensorflow.org is pretty helpful, try and conceptualize what youve been taught in the courses by playing around with that, e.g. add more layers/breadth to your network to watch it get more and more powerful and begin to overfit when you add noise to your data ect.
Just hijacking this question for my benefit as well. I am a product manager in enterprise focused software. I want to transition to the world of AI. Is Udacity's $600 Deep Learning Nano degree worth it ?
I was at the same point as you until I discovered the new Andrew Ng course on deep learning [1]<p>It's a good structured way to learn the core of ML while learning about Neural Networks and without having to become and linear algebra expert which for most people including like me was a deal breaker with other courses. The timing is great too as ML now is so much different than it was 2-3 years ago.<p>[1] <a href="https://www.coursera.org/specializations/deep-learning" rel="nofollow">https://www.coursera.org/specializations/deep-learning</a>
I think being effective in ML requires both theory, and practical knowledge you only get by doing.<p>Andrew Ng's ML course quickly provides a base in theory.<p>Ideally you couple that with some empirical work.<p>For that, I think sklearn is the best starting point (assuming you go down the python path). Modify some sample code and make a few simple models. Sklearn provides an excellent framework across all kinds of models (including deep learning if you use say keras.wrappers.scikit_learn), and can play well with pandas.<p>There are lots of practical concerns that come up that are not covered in intro ML courses.
As a follow up: I want to pursue a math degree study. What course titles and textbooks starting at the calculus level do you guys recommend? I want enough math chomps to then go onto a PhD in ML.
Possibly not what you're looking for (certainly not the cheapest option), but we (Lambda School - YC S17) just announced a live, remote class that trains engineers in AI & ML during weekday evenings for six months.<p>The next one starts in January, and is taught by an MIT grad that taught a similar course at MIT.<p><a href="https://lambdaschool.com/artificial-intelligence" rel="nofollow">https://lambdaschool.com/artificial-intelligence</a>
I've been going through this course: <a href="https://www.commonlounge.com/community/9dcdd386cc28446695305db00d2de532" rel="nofollow">https://www.commonlounge.com/community/9dcdd386cc28446695305...</a><p>It's a bit more cursory and mostly just a collection of articles/papers, but it has the benefit of not being paced like a university course.
I like following article as I find it one of the easiest introduction to neural networks:<p><a href="https://medium.com/technology-invention-and-more/how-to-build-a-simple-neural-network-in-9-lines-of-python-code-cc8f23647ca1" rel="nofollow">https://medium.com/technology-invention-and-more/how-to-buil...</a>
The AI for humans series is some reasonable, high level approach. <a href="http://www.heatonresearch.com/aifh/" rel="nofollow">http://www.heatonresearch.com/aifh/</a><p>After you've got a grasp of what these things are doing then you can move into the how. For that you will need some math background, with emphasis in calculus and probability.<p>After that, you can take a look at PRML. <a href="https://www.amazon.com/Pattern-Recognition-Learning-Information-Statistics/dp/0387310738" rel="nofollow">https://www.amazon.com/Pattern-Recognition-Learning-Informat...</a><p>Some people might prefer seeing things from another approach. <a href="http://pgm.stanford.edu/" rel="nofollow">http://pgm.stanford.edu/</a><p>Good luck.
For deep learning, my two favorite nominees are:<p>1. Hugo Larochelle's Deep Learning course available on YouTube<p>2. Depending on how much math you like, Nando de Freitas's Deep Learning course (also on YouTube) is also superb.
I liked the tutorials at Python Programming, sometimes the python details goes a bit fast and there are typos but over all it's the one that got me most understanding the practical parts.<p><a href="https://pythonprogramming.net/machine-learning-tutorial-python-introduction/" rel="nofollow">https://pythonprogramming.net/machine-learning-tutorial-pyth...</a>
I know you say you'd like to learn online, but I highly recommend picking up Duda and Hart's <i>Pattern Classification</i> to have a theoretical complement to the "hands on", programming type introductions. It's a very accessible intro to the topic, but also covers a lot of material in depth -- in particular, the topics you mention.
I'm currently working through "Deep Learning: A Practitioner's Approach" by Adam Gibson and Josh Patterson. Its a couple of years old but seems like a good book, and I'm certainly learning a lot. It doesn't consider some of the newer tooling, like TensorFlow, but the fundamentals plus a decent amount of theory are all covered.
I can't speak to the rest of the content, but I found the introduction in the course accompanying the recently announced gluon library to be both comprehensive and comprehensible at the same time.<p><a href="http://gluon.mxnet.io/" rel="nofollow">http://gluon.mxnet.io/</a>
I find this helpful. All links in one place <a href="https://www.springboard.com/learning-paths/data-analysis/" rel="nofollow">https://www.springboard.com/learning-paths/data-analysis/</a>
My recommendation is the fast.ai course by Jeremy Howard. His explanations are amazing and the practical usefulness is immediate.<p><a href="http://course.fast.ai/" rel="nofollow">http://course.fast.ai/</a>
In my humble opinion, there's no better way to start than with the classic book - " Artificial Intelligence : A Modern Approach " by Russel Norvig.
this one starts with simple stuff MIT 6.034 Artificial Intelligence: <a href="https://www.youtube.com/watch?v=TjZBTDzGeGg&list=PLUl4u3cNGP63gFHB6xb-kVBiQHYe_4hSi" rel="nofollow">https://www.youtube.com/watch?v=TjZBTDzGeGg&list=PLUl4u3cNGP...</a>
If you’re genuinely a novice programmer/lesser background in linear algebra, AI should be the <i>last</i> thing on your mind. Any attempts at a shortcut will enhance the difficulty in learning AI, and being able to code things besides simple examples. (which is why I am annoyed by many of the ML MOOCs which are targeted toward novice programmers)
Here are the resources I found useful:
==========================================
Advices from Open AI, Facebook AI leaders<p>Courses You MUST Take:<p>Machine Learning by Andrew Ng (<a href="https://www.coursera.org/learn/machine-learning" rel="nofollow">https://www.coursera.org/learn/machine-learning</a>) /// Class notes: (<a href="http://holehouse.org/mlclass/index.html" rel="nofollow">http://holehouse.org/mlclass/index.html</a>)<p>Yaser Abu-Mostafa’s Machine Learning course which focuses much more on theory than the Coursera class but it is still relevant for beginners.<p>(<a href="https://work.caltech.edu/telecourse.html" rel="nofollow">https://work.caltech.edu/telecourse.html</a>)<p>Neural Networks and Deep Learning (Recommended by Google Brain Team) (<a href="http://neuralnetworksanddeeplearning.com/" rel="nofollow">http://neuralnetworksanddeeplearning.com/</a>)<p>Probabilistic Graphical Models (<a href="https://www.coursera.org/learn/probabilistic-graphical-model..." rel="nofollow">https://www.coursera.org/learn/probabilistic-graphical-model...</a>)<p>Computational Neuroscience (<a href="https://www.coursera.org/learn/computational-neuroscience" rel="nofollow">https://www.coursera.org/learn/computational-neuroscience</a>)<p>Statistical Machine Learning
(<a href="http://www.stat.cmu.edu/~larry/=sml/" rel="nofollow">http://www.stat.cmu.edu/~larry/=sml/</a>)<p>From Open AI CEO Greg Brockman on Quora<p>Deep Learning Book (<a href="http://www.deeplearningbook.org/" rel="nofollow">http://www.deeplearningbook.org/</a>) ( Also Recommended by Google Brain Team )<p>It contains essentially all the concepts and intuition needed for deep learning engineering (except reinforcement learning). by Greg<p>2. If you’d like to take courses: Linear Algebra — Stephen Boyd’s EE263 (Stanford) (<a href="http://ee263.stanford.edu/" rel="nofollow">http://ee263.stanford.edu/</a>) or Linear Algebra (MIT)<p>(<a href="http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr..." rel="nofollow">http://ocw.mit.edu/courses/mathematics/18-06sc-linear-algebr...</a>)<p>Neural Networks for Machine Learning — Geoff Hinton
(Coursera)
<a href="https://www.coursera.org/learn/neural-networks" rel="nofollow">https://www.coursera.org/learn/neural-networks</a><p>Neural Nets — Andrej Karpathy’s CS231N (Stanford)<p><a href="http://cs231n.stanford.edu/" rel="nofollow">http://cs231n.stanford.edu/</a><p>Advanced Robotics (the MDP / optimal control lectures) — Pieter Abbeel’s CS287 (Berkeley)<p><a href="https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/" rel="nofollow">https://people.eecs.berkeley.edu/~pabbeel/cs287-fa11/</a><p>Deep RL — John Schulman’s CS294–112 (Berkeley) <a href="http://rll.berkeley.edu/deeprlcourse/" rel="nofollow">http://rll.berkeley.edu/deeprlcourse/</a>
I took a grad ML course based on this book: <a href="https://www.amazon.com/dp/B0759M2D9H" rel="nofollow">https://www.amazon.com/dp/B0759M2D9H</a><p>It teaches you the foundational theory behind ML, and shows how the fancier stuff is built on it. Good to know the foundations, so you can branch outside of predefined ML techniques.