TensorKart: self-driving MarioKart with TensorFlow

In contrast, here&#x27;s what is effectively an oracle machine playing mario kart: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=ZBNgbJ5hXtQ" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=ZBNgbJ5hXtQ</a><p>(Amazingly detailed) info: <a href="http:&#x2F;&#x2F;tasvideos.org&#x2F;5243S.html" rel="nofollow">http:&#x2F;&#x2F;tasvideos.org&#x2F;5243S.html</a>

Congrats on finishing the project! As you&#x27;ve already linked at the bottom of your post, it&#x27;s possible that OpenAI could&#x27;ve solved most of your I&#x2F;O issues.<p>One thing I&#x27;d suggest is exploring a reward function, instead of using only pre-recorded training data. That is, give the AI a goal to complete (in this case, finish the race) and let it learn by itself!

This is pretty cool; as someone who is currently working on the second project (traffic sign recognition) for the Udacity &quot;Self-Driving Car Engineer&quot; nanodegree, using TensorFlow - it is interesting to me how it seems like the &quot;standard&quot; MNIST CNN can be adapted to so many other use cases.<p>For the project I am currently working on, I&#x27;m using a slightly modified form of LeNet - which isn&#x27;t too different from the TF MNIST tutorial; after all, recognizing traffic signs isn&#x27;t much different than recognizing hand-written numbers...<p>...but &quot;driving&quot; a course? That seems radically different to my less-than-expert-at-TensorFlow understanding, but that is only due to my ignorance.<p>I&#x27;m glad that these examples and demos are being investigated and made public for others - especially people learning like myself - to look at and learn from.

The inevitable follow-up article that delves into training offensive banana peel usage should be interesting.

Quote: &quot;Driving a new (untrained) section of the Royal Raceway:&quot;<p>So the author did a proper test of the model by scoring it on an unseen track to make sure it generalizes! This is very awesome!

Personally I think the most impressive thing here isn&#x27;t that you created a self-driving MarioKart, but that you trained TensorFlow based on input screenshots of your desktop.<p>I feel like that could be a good next step - a ubiquitous neural net model that, after mapping inputs, will learn to play any video game that&#x27;s on your screen.

Best part, &quot;With this in mind I played more MarioKart to record new training data. I remember thinking to myself while trying to drive perfectly, “is this how parents feel when they’re driving with their children who are almost 16?”&quot;

It&#x27;s basically a project these days at Udacity&#x27;s Self-driving car nanodegree under &quot;Behavioral Cloning&quot; ;-)

I was ready to be impressed about seeing an AI that could consistently beat the game&#x27;s own AI, with blue turtle shells and all. Oh well, still pretty impressive to be able to drive on the easiest course without opponents.

Check out also MarI&#x2F;O, very impressive: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=qv6UVOQ0F44" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=qv6UVOQ0F44</a>

How are the original computer opponents able to play MarioKart?

Personally, I&#x27;m just a little impressed that you can train an active agent to play a game using old-fashioned supervised learning on screen states and controller states rather than relying on &quot;action-oriented&quot; learning techniques like reinforcement learning, online learning, or even a recurrent model.<p>It really shows how <i>simple</i> many control tasks actually are!

Next, have it upload its race results to kartlytics

I&#x27;m interested in knowing why the Python and C components communicate with HTTP, beyond reading about the bugfix. Wouldn&#x27;t it be easier to use sockets or files or some other mechanism to integrate the two languages?<p>Just something to think about as a developer. I would imagine that on a local machine, using HTTP as the protocol might add latency.

Pretty interesting I must say. Have to admit though, I kind of expected the self driving AI to be trying to win Grand Prix or Versus races instead of doing well in Time Trials. But hey, I can see how that would be utterly painful to try and set up, especially given how times you get hit by items or rammed off the track in more recent games.

It would be very interesting to see how well this does with more training data, especially with multiple players.

This is very cool and I think if Kevin spends a bit of time learning reinforcement learning it could be amazing.<p>It seems like a lot of people doing reinforcement learning on video games get bogged down on training on raw pixels only... it would take a tremendous amount of data to make the driver recognise when and where to use certain power ups, however if you encoded this as a variable, wow it could be really cool.<p>I believe this is fundamentally how we humans learn with so few examples. Other humans &quot;encode features for our brain to track&quot; by telling us how it should be done and what information to prioritise.

This is really cool, and any reason to bring this game back into my life is warmly welcomed

I love this!<p>I&#x27;m working (albeit very slowly, as a beginner) on a similar project with Geometry Dash and Python. You&#x27;re a great inspiration!

I appreciate the write-up. Thank you!

great write up! this is awesome

No power slide? Failure.