The International 2018: Results

It&#x27;s clear that there&#x27;s still a long way to go. Too much manual feature engineering and handtuning is needed right now due to the extremely weak reward signalling of current reinforcement learning algorithms, crippling OpenAI in the later stages.<p>The cool thing is that bruteforcing computational power seems to get us decently close. I&#x27;m optimistic that with renewed interest in the reinforcement learning field, breakthroughs will be made on the algorithmic side within a matter of time.

While there are certainly broader insights to be made, one tidbit I think I noticed that interested me had to do with Axe.<p>Between the two games, we saw Axe played by both the human team and the AI team. When played by the humans, blink-calls were completely shutdown by the AI&#x27;s superhuman counterinitiations. That made enough sense. When the Axe was played by the AI though, I don&#x27;t recall Axe ever even <i>attempting</i> any blink-calls. I&#x27;m curious if this might be the result of the AI overfitting to itself -- at AI reaction speeds, blink-calls are not a very useful maneuver, and thus the AI learns not to perform them.<p>Against a group of humans though, Axe&#x27;s blink-call initiations are arguably the hero&#x27;s biggest selling point.<p>We didn&#x27;t get to see most of the hero pool, but I wonder how much the AI overfitting to AI playstyles will hinder the bots against humans in the future.<p>Of course, the bots have many other issues which loom larger atm imo but I felt interested in enough in this tidbit to point it out.

I wish they’d use a better name for their engine than “Five”. Half the time I get confused if they’re talking about of Five players or Five engine or 5 something else.<p>Use a unique name like “Galaxy” which doesn’t represent anything remotely close in the game - Spell names, skills, etc. There is a huge amount of stuff going on in the game and it’s such a heavy cognitive load for an outsider who doesn’t play Dota, it was annoying to keep checking if they meant the name of the engine or number five in a game of Five vs Five. Or Five vs 5!!!? I’m so damn confused.<p>Same thing here: <a href="https:&#x2F;&#x2F;openai.com&#x2F;five&#x2F;" rel="nofollow">https:&#x2F;&#x2F;openai.com&#x2F;five&#x2F;</a><p>Number #2 bullet point says “Defeat five of the world’s top professionals<p>Five will attempt this live at The International in Vancouver’s Rogers Arena this week!”<p>It is such a poor choice.

Here are some insights into how OpenAI fine-tuned the rewards and short-term actions of the bots <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;dfarhi&#x2F;66ec9d760ae0c49a5c492c9fae93984a" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;dfarhi&#x2F;66ec9d760ae0c49a5c492c9fae939...</a><p>The numbers seem pretty arbitrary to me, that&#x27;s probably what this blog post is talking about when it mentions why it lost.

One of the clear weakness of the current OpenAI Five is its warding, with wards oftentimes being placed in bases. Perhaps the amount of vision that the team has can be a short term reward. Likewise, it currently does not engage in too much counter warding with sentries.<p>The game vs. Pain clearly demonstrated how humans can use wards to gain an information advantage over the bots that otherwise had a great chance of winning the game.

I hope they continue the project over the next year. I am really curious if they will be able to teach the AI to be better at late game strategies and long term planning. (Item builds might be an interesting challenge in this area as well)

Someone asked me, &quot;How does OpenAI know the MMR of its bots?&quot;<p>I don&#x27;t know. I assume it&#x27;s similar to how AlphaGo measures its ELO ranking. But the strange thing is, this is hundreds of years of self-play, not a public pool of humans playing against each other. How does MMR in simulation translate to MMR in real matches?<p>Before pointing out that it&#x27;s possible for an ELO rating, consider that Dota MMR is a bit different – every game you win, you get +25. Every game you lose, you get -25. This changes at the very high &#x2F; very low levels, or if the matchups are wildly imbalanced, but that&#x27;s the general setup. Or it was, a few years ago.<p>Does anyone have a guess?

Progress is much farther along than I expected. At the same time I wonder how well the bots adjust to the meta. Dota 2 changes every time a new patch rolls along. I also wonder if bots could make some of the more unconventional picks work. As an example, I love Medusa, and she&#x27;s one of the least picked heroes in pro dota

Question for the OpenAI team: have you ever thought about applying Five to other games, as it is?<p>I&#x27;d suggest Total War: Warhammer II (it has an interesting competitive scene, and has both tactical combat, and strategy gameplay), which is very different than DotA 2, but it would be super intriguing to see how it performed, and how fast it could learn.<p>If you built Five in the right way, it should be able to learn with mostly any other game.<p>I can also imagine you offering this to gaming companies in the near future, so that they could provide a decent computer AI instead of the crap they usually offer now :)<p>(source: I used to be a videogamer in my teens, and occasionally still play some strategy games, not as often as I would like to :D)