×

6
7

DL, MF, NOpenAI Five Benchmark: crushes audience team; stream of 3-game match against pros begins (twitch.tv)

submitted by gwern

all 4 comments
sorted by:
best

[–]gwern[S] 6 points7 points  (2 children)

Notes so far:

  • audience math shows OA5 very strong, but commentators are pointing out what looks like flaws and errors in play
  • OA5 now supports 'drafting' (picking the main units in turns strategically) rather than the previous OA5 announcement of picking main units at random
  • OA5 sometimes moves units at random even early in the game before enemy contact, but this may be another 'pathology' like MCTS playing randomly when it's way ahead (OA5 estimated 99% victory early on from good drafting): https://twitter.com/gdb/status/1026208916391129088
  • currently: 1/1. First match only lasted 21m, 95% estimate escalated quickly to 99% which was odd (suggests it saw the human team moving too slowly?)
  • commentators: largely impressed but a little critical of the game-state API access, and I note that the OA5 early-game focus may be a weakness rather than a strength - we were all surprised PPO could learn such long-range strategies, but maybe it's still not great? Some Smerity comments: https://twitter.com/Smerity/status/1026235296013148160
  • Cool feature: value/win probabilities visualized during the drafting for the second game: https://twitter.com/gdb/status/1026219000689119232 (Drafting really does matter, so that might turn out to be the explanation for the 95% initial win probability.)
  • 2/2, match goes to OpenAI. OA5 didn't concede even a single tower, apparently. Makes one wonder what the ELO (or equivalent) difference is.
  • Game 3: special rules handicap, audience picked maximally bad set of units for OA5; lasted ~40m, OA5 was behind the entire time (never higher than 17%), and got ground down.
  • Roshan: I was wondering if Roshan would be something that the bots could learn, given that it requires an extremely specific concentrated attack to yield an item which is useful much later; apparently in the interviews (and then in the panel afterwards), OA says that some curriculum learning was done by lowering Roshan HP to a very low amount to make killing Roshan easy, and that led to learning about Roshan; Roshan turns out to be worthwhile mostly in long games, otherwise it's a distraction from the intense early push OA5 favors.

Discussion of the August The International tournament matches: https://www.reddit.com/r/reinforcementlearning/comments/99ieuw/n_first_openai_oa5_dota2_match_begins/

[–]untrustable2 0 points1 point  (1 child)

What's the evidence that they are focussing early-game to the detriment of later? Could it not just be optimal play?

[–]gwern[S] 1 point2 points  (0 children)

It could be, but it's interesting that OA5 seems to focus so heavily on the early game, discounts Shadow Fiend, thought it'd won the first game from the start, and it is trained with a method which should make it very hard to learn very long-range planning. So it's hard to say, but there's some evidence that OA5 might have a flaw of that sort. If so, human players could get an edge by enduring the initial rush in exchange for some sort of major late-game advantage.

EDIT: a lot of people are saying this about Game 3 - the bots seemed to have a lot of trouble with a coherent strategy in the middle and end, which is what you would expect if they are early-game centric because they either can't or don't need to learn much about later games. An example commentary making this point at length, noting that in game 3 the bots start making massive numbers of outright errors and even having trouble moving units sanely: http://www.gamesbyangelina.org/2018/08/openai-dota-2-game-is-hard/

[–]untrustable2 6 points7 points  (0 children)

There were several moments where the AI had a threat come into view and instantly hexed(?) the enemy before a trained human had time to even process the data, thereby making the humans essentially impotent. Couldn't help but see some rather unpleasant military overtones.