OpenAI Five Benchmark

I&#x27;m surprised they felt confident enough to lift a lot of the restrictions so soon. A lot of them seemed like big game changers I wouldn&#x27;t expect the ai to be able to exploit so easily. Especially the introduction of Roshan and invisibility, they mention implementing some randomization, it seems like it would take quite a big commitment in terms of resources to take Roshan where the AI wouldn&#x27;t even realize the benefits immediately (namely the Aegis, the XP and gold they would have to value against the loss from farming&#x2F;taking towers).<p>The introduction of the other heroes also comes as a surprise, I wouldn&#x27;t have expected them to have the ai utilizing new abilities. They don&#x27;t mentioned how they are picked, other than the ai having a random draft of them (does the ai pick their composition?)

&gt; We’ve increased the reaction time of OpenAI Five from 80ms to 200ms. This reaction time is much closer to human level, though we haven’t seen evidence of changes in gameplay as Openai Five’s strength comes more from teamwork and coordination than reflexes.<p>This new constraint is interesting. The Super Smash Bros. Melee AI paper noted that they had to keep the reaction times to superhuman levels in order for the model to converge (albeit DOTA is a bit different from Melee): <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1702.06230" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1702.06230</a>

&gt; Because our training system Rapid is very general, we were able to teach OpenAI Five many complex skills since June simply by integrating new features and randomizations. Many people pointed out that wards and Roshan were particularly important to include — and now we’ve done so. We’ve also increased the hero pool to 18 heroes. Many commenters thought these improvements would take another year.<p>The linked commenters thought that getting to &quot;real dota&quot; (more than 100 heroes, captains mode instead of random, ...) would take another year. So I don&#x27;t think it&#x27;s fair to make that statement.<p>Edit: Don&#x27;t get me wrong, I think the improvements are very nice, but pointing to people saying &quot;these people thought we would need a year, we did it in under a month!&quot; is not what you should do if you didn&#x27;t actually do what the linked people stated.

Note: The choice of human opponent team is from people that are vastly better than the majority of Dota2 players, but still vastly worse than the top-tier pro teams.<p>So when next time Elon tweeted: &quot;OpenAI beats the human players in 5v5&quot;<p>You know that the game is not broken by AI yet (not like Go, which is indeed broken by AI).

Does anyone know how random drafting work? If they&#x27;re truly random, i.e. randomly picking 5 heroes out of 18 (CM, DP, ES, Gyro, Lich, Lion, Necro, Qop, Razor, Riki, Nevermore, Slark, Sniper, Sven, Tide, Viper, or WD) then it&#x27;s much less about teamwork. What if they end up with Razor, QOP, Nevermore, DP, Gyro? The problem here is in Dota, each hero almost has a clear position in the game, much like soccer. Having both teams randomly pick 5 heroes would probably ruin the game, and make it really difficult for human (think covariate shift), whereas the bot is probably trained using this distribution

This will be very interesting to see, despite the vast differences in ruleset that make this much much less complex than the actual game.<p>My prediction is that we&#x27;re very very far away from AI that can beat the top teams in a 5v5. Amateur teams can easily be beat simply on the strength of the mechanics (which are very very strong on the AI, beating even pros), but the strategy and coordination of the top teams are out of this world.

I really hope OpenAI teached bots to type &quot;cyka&quot; and &quot;go mid&quot;

Less restrictions is of course better, but I&#x27;m still not impressed without an actual Captain&#x27;s Mode all-heroes draft. In addition inverse_pi&#x27;s comments about all-heroes being vital because heroes have different roles to play, the draft is both an important part of the game and, I would think, one of the most difficult for an AI: it involves bluffing, mind games, online strategy adjustment in response to an opponent&#x27;s actions, and awareness of the current meta.<p>The draft isn&#x27;t everything and it&#x27;s possible that a sufficiently talented AI could always lose the draft and still win the game, but that would be a pretty boring outcome from the perspective of contributing to AI knowledge (just like it&#x27;s possible, though unlikely in Dota, that sufficiently good micro could overwhelm any disadvantage in strategy and tacitcs if the AI can play at 2000 APM: it would &quot;win&quot;, but only in a very boring sense)

Based on the chart showing the effect of &quot;We&#x27;re still fixing bugs&quot; in their last blog post, it looks like they should have the skills &#x27;buffer&#x27; to handle significantly better teams than those they have faced so far.<p><a href="https:&#x2F;&#x2F;blog.openai.com&#x2F;openai-five&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.openai.com&#x2F;openai-five&#x2F;</a><p>&quot;We’re still fixing bugs. The chart shows a training run of the code that defeated amateur players, compared to a version where we simply fixed a number of bugs, ...&quot;<p>Looking at the chart and the fact that they are confident enough to lift several restrictions, I&#x27;d bet on OpenAI Five winning against at least some of the professional teams at The International. It&#x27;s even possible they will beat most teams there.

I am embarrassed to say that I am confused about what the outputs are from the &quot;Rapid&quot; RL training system. Do you end up with an executable that then drives the game inputs&#x2F;api? Does it produce a &quot;bot script&quot; that is used by the game to drive the logic? I understand that thousands of CPUs&#x2F;GPUs are used for the training, but then what is actually playing the game at the end of the day?

What is the purpose of having deep learning run on games like AlphaGo and DOTA2, instead of having them train on more general or real world tasks? Is it a constraint on the amount of data, since in video games you can easily generate more?

How do 5 DotA players coordinate? They share information via voice?<p>How does a deep learning algorithm coordinate between 5 heros? I assume it&#x27;s not 5 bots communicating over some channel but one bot acting on 5 heros?

How does counter-picking work if it&#x27;s no longer mirror match? Was this a separate model?