I worked on this applied Deep Reinforcement Learning course for the better part of 2021. I made a Datacamp course [0] before, and this served as my inspiration to make an applied Deep RL series.<p>Normally, Deep RL courses teach a lot of mathematically involved theory. You get the practical applications near the end (if at all).<p>I have tried to turn that on its head. In the top-down approach, you learn practical skills first, then go deeper later. This is much more fun.<p>This course (the first in a planned multi-part series) shows how to use the Deep Reinforcement Learning framework RLlib to solve OpenAI Gym environments. I provide a big-picture overview of RL and show how to use the tools to get the job done. This approach is similar to learning Deep Learning by building and training various deep networks using a high-level framework e.g. Keras.<p>In the next course in the series (open for pre-enrollment), we move on to solving real-world Deep RL problems using custom environments and various tricks that make the algorithms work better [1].<p>The main advantage of this sequence is that these practical skills can be picked up fast and used in real life immediately. The involved mathematical bits can be picked up later. RLlib is the industry standard, so you won't need to change tools as you progress.<p>This is the first time that I made a course on my own. I learned flip-chart drawing to illustrate the slides and notebooks. That was fun, considering how much I suck at drawing. I am using Teachable as the LMS, Latex (Beamer) for the slides, Sketchbook for illustrations, Blue Yeti for audio recording, OBS Studio for screencasting, and Filmora for video editing. The captions are first auto-generated on YouTube and then hand edited to fix errors and improve formatting. I do the majority of the production on Linux and then switch to Windows for video editing.<p>I released the course last month and the makers of RLlib got in touch to show their approval. That's the best thing to happen so far.<p>Please feel free to try it and ask any questions. I am around and will do my best to answer them.<p>[0] <a href="https://www.datacamp.com/courses/unit-testing-for-data-science-in-python" rel="nofollow">https://www.datacamp.com/courses/unit-testing-for-data-scien...</a>
[1] <a href="https://courses.dibya.online/p/realdeeprl" rel="nofollow">https://courses.dibya.online/p/realdeeprl</a>
To be honest though, the practical side of things of RL can be a hit-and-miss in terms of "fun" depending on the person. It requires a lot of manual hand tuning, reward shaping, hyperparameter tuning, and general trial-and-error to make an agent do a seemingly simple-enough task, and these tricks are more heuristically and haphazardly done than what you would expect from more "conventional" programming. It is fun for the right people (who loves tinkering with stuff and also have the perseverance to continually run RL experiments that can last hours or even days). But I would imagine many getting bored by the whole experience. (Pssst.... I was one of them, switched to doing something else in the middle of grad school)<p>By the way, RLlib is good if you want to try out simple experiments with well-established RL algorithms, but it's <i>really</i> awful to use when you want to modify the algorithm even just a little bit. So it's not bad for beginner-level tutorials, but once you get the basics it might be very frustrating later on. I would recommend simpler frameworks like Stable Baselines 3 (<a href="https://stable-baselines3.readthedocs.io/en/master/" rel="nofollow">https://stable-baselines3.readthedocs.io/en/master/</a> ) for a much more stable experience, if you have gained a fair bit of Python/ML programming skills at hand and don't have trouble reading well-maintained library code.
This looks nicely done, but for anyone interested I'd like to mention that these courses aren't something that can replace learning the fundamental concepts and theories behind ML/RL, for which there exist excellent books and courses that focus more on math and theory. I would go there.<p>These courses teach you how to call a library and use an API. You get nearly the same thing from just looking at the docs. Please don't say you "know RL" after this.
Enrolled! Went through the detailed lesson plan and you have done a great job structuring the course. I am looking forward to doing it over the weekend.<p>One suggestion: Instead of naming all the Jupyter notebooks "coding_exercise.ipynb", maybe name them differently? That way, they won't overwrite the previous download.
May also be of interest: <a href="https://github.com/huggingface/deep-rl-class" rel="nofollow">https://github.com/huggingface/deep-rl-class</a>
When I visit the site using Edge, even with Adblockers disabled, I'm unable to view the courses listed as "preview", such as <a href="https://courses.dibya.online/courses/fastdeeprl/lectures/38389599" rel="nofollow">https://courses.dibya.online/courses/fastdeeprl/lectures/383...</a>, and instead get a notice that "this page has been blocked by Microsoft Edge".
(Related but kind of off-topic)<p>I’m a software engineer (non-ML) currently working at big tech company that does ML and has a fair amount of open roles in ML and I’ve wondered is ML the sort of thing you could jump into a team and learn on the job? Or do you really need to take some courses, read some books, or even get a degree?<p>I got a CS/Math bachelors but it’s been nigh on a decade and my higher level math is rusty. Curious on people’s thoughts here.
My suggestions for learning deep RL are the book Grokking Deep RL and the Spinning Up website. These are reading focused obviously. Then, when your implementations don't work, compare them to minimal-rl. I don't intend to detract from this course, just adding some of my own suggestions on the topic.<p><a href="https://www.manning.com/books/grokking-deep-reinforcement-learning" rel="nofollow">https://www.manning.com/books/grokking-deep-reinforcement-le...</a><p><a href="https://spinningup.openai.com/en/latest/" rel="nofollow">https://spinningup.openai.com/en/latest/</a><p><a href="https://github.com/seungeunrho/minimalRL/blob/master/sac.py" rel="nofollow">https://github.com/seungeunrho/minimalRL/blob/master/sac.py</a>
Would this teach transformers? Or is that something else?<p>Also any tips for finding a study group for learning the large language models? I can’t seem to self motivate.
Before jumping into Deep Reinforcement Learning I highly recommend doing the Reinforcement Learning course by David Silver [1].<p>[1] <a href="https://www.deepmind.com/learning-resources/introduction-to-reinforcement-learning-with-david-silver" rel="nofollow">https://www.deepmind.com/learning-resources/introduction-to-...</a>
This looks great! Thank you for all the thought and effort you have put into it.<p>I am currently working on a project where I need to use RLlib for a capacity planning problem. Looks like I will learn a thing or two over the weekend.<p>I will eventually need to use a custom environment, so it's great to see it's included in your roadmap. Most courses I have seen totally ignored that. Fancy Atari envs are great for practice and have wow factor, but you need a custom environment to do anything resembling real work.<p>Would I need a beefy GPU for the coding challenges?