This thread will double as a discussion of match 2 and, since it's 2 of 3 and losing the 2nd match means no 3rd match, discussion of the 2 TI matches as a whole.

• Previous: Benchmark competition: https://www.reddit.com/r/reinforcementlearning/comments/94uziv/openai_five_benchmark_crushes_audience_team/
• Previous Match 1 discussion: https://www.reddit.com/r/reinforcementlearning/comments/99ieuw/n_first_openai_oa5_dota2_match_begins/
• Brockman Twitter commentary: https://twitter.com/gdb/status/1032772998997004288
• Mike Cook Twitter commentary: https://twitter.com/mtrc/status/1032773500682031107
• Smerity Twitter commentary: https://twitter.com/Smerity/status/1032810003466350592
• DoTA2 subreddit discussion: https://www.reddit.com/r/DotA2/comments/99sfea/the_international_8_openai_match_2/

• Comments:

  • this match was not against a formal team, but against a collection of apparently very well-regarded current and ex-pros: https://twitter.com/gdb/status/1032773519946346496
  • like match 1, it seems to have been a close game early on, but the human team played the long game and eventually smooshed OA5 despite some excellent fights by the latter

  • OA says that though the 5->1 courier change only began training back on Saturday or so, they don't think it was responsible for the apparent performance decline from the Benchmark to TI:

    We don’t believe that the courier change was responsible for the losses. We think we need more training, bugfixes, and to remove the last pieces of scripted logic in our model.

  • the OA5 win probability graphs in the 2 matches can't be trusted because apparently it still hadn't recovered from the courier switch: gdb

    The win probability head had a regression (likely because we stopped training it in favor of putting the optimization power into gameplay) so we don't have reliable data there sadly!

    That aside, apparently defeat was forecast by the OA team: https://www.theverge.com/2018/8/23/17772376/openai-dota-2-pain-game-human-victory-ai https://twitter.com/gdb/status/1032931666451296256

    Speaking to The Verge ahead of the game last night, OpenAI co-founder and chief researcher Greg Brockman said that an internal poll of employees had suggested there was “less than a 50 percent probability of winning.” “That was the general consensus,” said Brockman, before adding that what was really important was the rate that the AI team was improving. “Usually we start playing teams when we’re about at their level, then a week or two later we surpass them. And that has happened to us a number of times now.”

    The estimated MMRs in OP are not based on the win probability or internal self-play metrics but against various volunteer human teams: https://news.ycombinator.com/item?id=17838694

  • many of the comments echo match 1: a number of clear errors by OA5, serious suboptimal use of items like the Aegis from Roshan (which had to be conceded to the humans), poor use of abilities to kill enemies (instead damaging them), OA5 might not understand the bad consequences of losing its barracks (creating 'mega-creeps')

  • OA intends to continue with DoTA2. Brockman: "Our interpretation is we have yet to reach the limits of current AI technology. Learning curves haven’t asymptoted just yet! Note: would be quite a cool finding if our current approach does not scale to the level of the top human pros. But this week's matches were a snapshot of current progress, not what's possible." I'd give decent odds that next TI, OA5 will win. Although I kinda hope that it isn't just by training even more - how much has OA spent at this point, $5m? Brockman noted on Twitter that they had increased the scale of training even more from before.

  • Blitz:

    Please lord don’t let us be the only ones to lose to openai [7:26 PM - 23 Aug 2018]

    Fkkk [2h]

    Our condolences, Blitz.

  • the PPO reward discounting is set to halve over 14 minutes now: https://twitter.com/gdb/status/1032817408665239552

  • Overall, I'm left with the impression that OA5 still exhibits some of the pathologies of pure deep self-play: patchy understanding, inexplicable blindspots, possible instability in training, questionable understanding of the long-term (even if still ridiculously good for mere PPO), and exorbitant compute requirements. Training it further would surely fix up some of the problems, but will it be like squeezing a balloon? I can't help comparing with AlphaGo's progression - some more systematic and principled method of exploration, if nothing else, may be required.* (And speaking of which, what has DM achieved with SC2 so far...? The AG program was shut down like a year and a half ago, and the SC2-related papers they've published have been fairly minor.)

* There might not be any need for a fundamentally new architecture like tree search or meta-learning at runtime. I would point out that the AlphaGo Zero reactive policy/CNN is close to superhuman all on its own, and the earlier AG CNNs were pretty good too. So reactive policies encoded into deep neural networks are capable of extremely powerful gameplay, if they are trained appropriately. What 'appropriately' is for a huge decPOMDP like DoTA2 may be the real question here.