Been staring in awe at this with my little sons (2&3). We actually get enthusiastic when the first signs of understanding arise. Where can I begin learning about this? I've done several sklearn examples, but this application to 'a new world' (like a racing game), how do you build that?<p>With build I mean, how do you structure inputs and objects so that the goal is met? The goal for racing is both steering, braking, trottle...
This is really amazing how fast they learn! It seems to make a lot more progress than, for example, that genetic car evolver that at generation 200 is still trying to decide whether to put the wheels on the bottom.
I observed something interesting: when I changed the number of hidden layers from 100 to 1, almost nothing changed.<p>It took slightly more generations to learn how to race, but not that many more (~80 generations vs ~90 generations). In addition, I thought I observed a small difference in behavior, but nothing I could explicitly measure.
Generation 50 but none of the cars has learned to pass more than 5-6 corners.<p>Also, changining the track from one interation to another I don't think it helps. The training should be done on the same track until at least one of the cars is able to complete it. Than it can be changed.<p>Sometimes some of the cars get stuck and the round doesn't finish, without having to click the "End round" button. There should be some mechanism that detects that cars are stuck and end the round automatically.
Another awesome one for the collection! It would be interesting to compare this and the RC car: <a href="https://www.youtube.com/watch?v=1AR2-OHCxsQ" rel="nofollow">https://www.youtube.com/watch?v=1AR2-OHCxsQ</a><p>r/WatchMachinesLearn
I've found some tricks to get it to learn really well (by gen 20 they all finish really fast)
1. Using a relu, return value => value >= 0 ? value : 0.0;
2. Bumping it up to 30 cars (if you use too many, it slows down, and you dont get as good of results because each iteration takes ages)
3. Changing the lap condition to c.lap < 5 and the time to 30 s. This lets each race go longer and widens the discrepancy between the best and worst racers - which improves selection
Setting the hidden layer activation function to<p><pre><code> value => value >= 0 ? value : 0.5 * value;
</code></pre>
seems to improve convergence a lot.
Neat!<p>It should be noted that the genetic algorithm used is not the same as reinforcement learning, in which the entire history of decisions leading up to a success is fed back to the NN as training examples.
Nice, I love the way the user can change the NN and genetic algorithm parameters!<p>There isn't much documentation though - is the genetic algorithm being used to evolve the NN somehow?