This is an archived post. You won't be able to vote or comment.

all 99 comments
sorted by:
best

[–]CorruptDropbear 27 points28 points  (5 children)

Currently, OpenAI Five is 1648 wins to 8 losses (99.5% winrate) in #Dota2.

Winners:

  • One Disconnect Loss
  • 🇷🇺 @Winstrike_Team
  • 🇹🇭 Question Mark by Hashtag
  • 🇲🇾 AOES.K2Surf
  • 🇵🇭 Pogiz Poseidon Esports
  • 🇹🇭 Alpha Red
  • 🇷🇺 DreamShiters
  • 🇪🇺 @WagaGaming & Friends

[–]loopuleasa[S] 5 points6 points  (0 children)

thanks for the compilation

[–]AGI_69 1 point2 points  (3 children)

Currently, OpenAI Five is 1648 wins to 8 losses (99.5% winrate) in #Dota2.

This is exactly what people shouldnt say. OpenAI doenst have 99,5% winrate in Dota2, it is playing rigged game, only 17 heroes instead of 115. How is this Dota 2 ? Thats like playing chess, only with pawns and calling it chess. You should use language more carefully.

[–]Sordahon 0 points1 point  (2 children)

Do all players play 115 characters like you seem to do?

[–]AGI_69 4 points5 points  (1 child)

Humans are much better Dota 2 players than OpenAI. The OG match proved nothing. You dont understand, how much complexity was taken out of the game. Calling the series of events "AI beats humans at Dota 2" is extremely flawed. They reduced comlexity of Dota 2 to less than 1%. I am dissapointed that they dont let us train it on us and maybe keep few computers running for itself. I mean they have billions of dollars, cant they spare few thounsand ?

[–]AGI_69 1 point2 points  (0 children)

Same happened with AlphaZero (DeepMind company), the chess engine. Few games and than they went silent. They dont give a fuck about us, just prove us that they are better and then dissapear. No chance to play it, study it or nothing. At least there is Leela in chess world, but in Dota 2, I am not sure if there will be truly Open source project. They should change their name from OpenAI to something else.

[–]teerre 51 points52 points  (13 children)

Are those lessons, though? It's the same thing it was in 2017 for the bots and generally speaking people noticed this strategy 15 years ago in normal dota

With the aggravating factor that the bots are completely clueless against silly things like the sharpnel or the rik radiance

[–]dpwiz 15 points16 points  (4 children)

Clueless, but can you turn that into overwhelming advantage? 🤔

[–]loopuleasa[S] 39 points40 points  (0 children)

not easy

waga managed to win once with a lineup, and then tried to reproduce it

lost the next 2 games, exact same lineup

[–]teerre 0 points1 point  (2 children)

What?

[–]*sheever support* Dropped my pants off at the cleaners.RedGuyNoPants 3 points4 points  (1 child)

can you take your knowledge of what the bots are bad at and use it to counter what the bots are good at?

[–]teerre 0 points1 point  (0 children)

Yes? That's literally what everyone is saying?

[–]loopuleasa[S] 2 points3 points  (0 children)

yes, but we have to confirm them with newest version

[–]scooerp 2 points3 points  (6 children)

It doesn't matter what you think you know if you can't demonstrate it against the current system.

[–]teerre 0 points1 point  (5 children)

What? People are demonstrating the weaknesses of the bots all over the place. It's clear as day

[–]scooerp 2 points3 points  (4 children)

Two days no-one knew if any of that would work vs the new OpenAI. Now we do. Knowledge requires proof or it's not knowledge.

[–]teerre 0 points1 point  (3 children)

What you mean? OpenAI plays the same game since their very first appearance. It was extremely clear that they were terrible at anything that wasn't winning the laning phase and 5-man. They are still like that today, nothing changed in their overall strategy

[–]scooerp 1 point2 points  (2 children)

Theory and knowledge are different. You believe certain strats still work but the machine is better so how can you be sure until you see it play more games?

If you want an analogy, you see a car go behind a house. Later, there's engine sounds from behind that house. You believe it's the same car but you can't know until you look.

[–]teerre 0 points1 point  (1 child)

No one "believes" it. It's literally what it is. It's fact. You can watch the OpenAI games right now. You can go watch the matches at TI7. You can read how the AI was trained and think about it for some time. It will lead you do the same conclusion

[–]scooerp 1 point2 points  (0 children)

Before the new openAi arena started, it wasn't a fact. Now it is.

A fact is verifiable and would have been impossible to verify before the new AI played. Your statement didn't change, but it became factual when it was able to be verified.

BTW I disagree that the bot is good at laning. A lot of times it loses the lanes but makes up for in teamfight execution.

[–]silent godWalrusPorn 34 points35 points  (3 children)

I played against them as one of the testing teams last summer, and this is how we beat them somewhat consistently.

I haven't played since, but what worked then was stopping their early death ball. This was done by prioritizing something like an ES blink, and echoing them when they go for t3s.

Meanwhile you have 1 or two heroes with escape constantly split pushing. We used QoP/Slark because if you can get into unpathable areas in the trees, the bots won't look for you and you can wait ~30 sec until they leave, then pop out and cut another wave.

Eventually your Slark will become overleveled and overfarmed compared to them because they are leaving 5 in a lane, and you can just win fights and end.

Again, disclaimer, this was against the bots of 10 months ago, and I didn't really watch the recent stream so this probably will not work anymore. I actually don't even play DotA, I switched to league, cause spending all my time getting smashed by bots destroyed my DotA spirit.

[–]kanak42 12 points13 points  (2 children)

This sounds an awful lot like the regular bots you play against in practice matches

[–]lolfail9001 16 points17 points  (0 children)

I mean, they run the same hero pool and run the same strat. Modern iteration is far better than those of old (and patch favors deathball far more than pre-TI patches), but it's not like identity changed.

[–]silent godWalrusPorn 29 points30 points  (0 children)

You don't beat them by playing DotA (as the average player), you beat them by abusing their logic... Which happens to be by splitting the map.

[–]AI enthusiastmuskar2 5 points6 points  (1 child)

Are these the win streak guys?
https://www.twitch.tv/juniorclanwar/videos
Doesn't look legit - but hey, I'm a dota newbie

[–]你气不气?Imbluedabodee 1 point2 points  (0 children)

yes, they won 4 times now

[–]decibelsBouncing 6 points7 points  (10 children)

whats with the dude who beat them in 7mins

https://arena.openai.com/#/results

[–]W10104 23 points24 points  (3 children)

To clarify from that weibo: the poor guy suspected by you actually don't know what happened.

According to his words, he joined a match with 4 bots (thus only one player name) and beat the other team in the ass.

He got bored and surrendered, but found out that the match counted as a win and topped the leaderboard.

IMHO he's not a deliberate hacker, and I don't like seeing that accident on the "fun things" as if it's confirmed. OP remove that line pls.

[–]loopuleasa[S] 3 points4 points  (0 children)

will do

[–]justatimebomb 5 points6 points  (0 children)

according to twitchchat every win is either a streamsniper scripter hacker or using some cheese.

Nothing is legit LUL

[–][deleted] 0 points1 point  (0 children)

Also the DotA GC's ways are mysterious. I wouldn't be surprised if there was a win or two because of GC bugs(I have 2 wins in ranked because of bugs so far).

[–]loopuleasa[S] 5 points6 points  (1 child)

OpenAI will go to sleep soon, but after they are back they will check database

https://twitter.com/gdb/status/1119238312332390406

[–]TweetsInCommentsBot 5 points6 points  (0 children)

@gdb

2019-04-19 13:56

@karoly_zsolnai Something seems off — leaderboard only shows one player. Team will check it out when awake.

This message was created by a bot

[/r/DotA2, please donate to keep the bot running] [Contact creator] [Source code]

[–]scooerp 2 points3 points  (0 children)

NA and SA still not beaten it.

[–]Sunrise1912 4 points5 points  (2 children)

Some dudes beat them again and have a streak.

[–]loopuleasa[S] 0 points1 point  (1 child)

yes, added

[–]AI enthusiastmuskar2 4 points5 points  (0 children)

Is it these people?
https://www.twitch.tv/juniorclanwar/videos

[–]ZCC_TTC_IAUS 1 point2 points  (0 children)

Got a link to the Winstrike game?

[–][deleted] 1 point2 points  (1 child)

Has anyone in this thread beaten the bot yet?

[–]Pjosk 3 points4 points  (0 children)

We won earlier but weren’t able to repeat. I feel like it’s a matter of time before people will solve this iteration of the AI. The courier movement and lack of detection are some major flaws atm.

[–]ebhBali 1 point2 points  (0 children)

Actually I am wondering, if we (my partner and I, we are improving the performance of esports athletes) could prep a team to beat the bot? (Whoa, that rythm of words sounds awesome, I should write a rap song from it...) Is there any team out there who would be our guinea pig for this trial?

[–]I'd still have my TL flair but i have to move onbrianbezn 3 points4 points  (2 children)

The bot used 45,000 years of game time for reference, proving that humans are better at learning from less information and experience

This is incorrect reasoning even if the conclusion is correct. You can't compare how fast 2 things learn if the one wining has been learning for more time.

After first 1000 games, after 10 hours of uptime, only 3 were won by humans

Small correction, it's after 1000 wins, it's 1003 games.

[–]loopuleasa[S] 0 points1 point  (1 child)

I am aware of both, but differences are minor

[–]I'd still have my TL flair but i have to move onbrianbezn 1 point2 points  (0 children)

fair enough

[–]MouZeWarrioR 0 points1 point  (23 children)

The bot used 45,000 years of game time for reference, proving that humans are better at learning from less information and experience

This is completely false. Humans have what? Roughly ten million years of playtime? It likely took a million years just to learn about pulling and stacking.

Humans learn from each other just like the bots do. One human wouldn't have made anywhere near as much progress on his own. The competition is what drives the skill level up. It's the same for humans and bots alike.

On the contrary, the bots seem to be learning multiple times quicker than humans.

[–]lolfail9001 19 points20 points  (4 children)

> On the contrary, the bots seem to be learning multiple times quicker than humans.

​

Bots did not have a blank slate start either. In fact, they are practically using knowledge it took humans those million of years of playtime to learn to START learning.

​

So in comparison to your typical pro player's 10 to 30k *hours* of playtime (not client uptime but actual playtime), bots do look fairly pathetic in their speed of learning. Which is nothing wrong, because that's a well known machine learning problem.

[–]MouZeWarrioR -1 points0 points  (3 children)

Pff, if we're talking about 'blank slates', then I might as well mention that the modern human is the result of the hundreds of thousands of year that human have been gathering and sharing knowledge. The "Kill good, death bad"-input the bots got seems pretty meaningless in comparison.

Bots gather data more efficiently and share data more efficiently, thus they also learn more efficiently. I don't think it's much more complicated than that.

[–]lolfail9001 -2 points-1 points  (2 children)

> then I might as well mention that the modern human is the result of the hundreds of thousands of year that human have been gathering and sharing knowledge.

​

Yes, and what is relevance to it? The fact of the matter is that to teach something to human showing it and using it few times is enough. Now look how long it took bots to play as they do after being explicitly told that earning gold is good, dying is bad and losing gold is okay.

[–]MouZeWarrioR 1 point2 points  (1 child)

The point is that the 'blank-slate' argument is bullshit. Humans are as far from a blank slate as they could be, yet you think it's unfair that the bots got a few lines of information.

The bots started out with something remarkably close to a blank slate but have already gotten incredibly good at the game.

[–]lolfail9001 -1 points0 points  (0 children)

> yet you think it's unfair that the bots got a few lines of information.

​

Do I? I just say that claiming bots learn faster than humans because humans can and do actually *learn* from external sources in hours, not years is outright bullshit.

[–][deleted] 6 points7 points  (0 children)

AI learns quicker when it's measured by time. Humans learn quicker when it's measured by amount of games. Up until a certain point.

I can guarantee you, put the current generation of AI versus the average human, have them both start off without any knowledge of Dota, and teach them nothing (Maybe a tutorial, but even when taught nothing, the human should be better), don't allow the humans to learn from other humans.

The human will improve at a way quicker rate than the AI does, when measured in amount of games played, rather than time since the start of the experiment. It's only when both are sufficiently skilled, I'd even say, more skilled than our best player are right now, that the AI begins to learn more PER game than the human does.

Of course when it comes down to it, the AI will still learn quicker, because it's just able to play much more games in the same amount of time as the human. But if you measure it by time played, or by games played, the average human will beat the current generation of AI starting from scratch.

[–]ElTigreChang1 7 points8 points  (3 children)

yeah, but humans don't have a hivemind. we can choose to pool some information we have, but if you added up the play time of all the people who have done that and put it into practice, it'd still be far, far below that of the ai.

[–]MouZeWarrioR 1 point2 points  (2 children)

Ofc we have a hivemind, I don't understand how you can deny that, especially in this era.

In Dota ALL of us share most of our knowledge regardless if we want it or not. All your movements and decisions are public information that anyone can learn from. What it comes down to is whether someone actually bothers to learn from the information presented. In most cases, humans don't do that, which is exactly why AI has an edge.

[–]IgnoobV 0 points1 point  (1 child)

For many years when we were still playing dota 1, the Vietnam dota 1 community was playing like this:
Pudge mid. Pudge vanguard
Mid has to buy courrier. First blood have up upgrade the courier to flying
And many many more silly stuff. Not just the noobs, even the best players in the community did this. Isolate the Vietnam community with the rest of the world and our dota knowledge progression is very slow.
When dota 2 come and we get to play in the SEA sever, those silly way of playing become obsolete in mere days. We learned from the rest of the world. We did not figure it out by ourselves, but learned the end knowledge presented to us. And how did the knowledge of how to play dota correctly come about ? It is from the collective effort of millions of players and a lot of dota 2 match
When i first play dota, i read guides on the internet. Those guides are the result of many, many players. Just by reading a guide for a few minutes, i learned what it took millions of dota 2 games to find out. This is basically collective intelligence at work.
Let me play dota alone, how much would it take for me to find out that pudge shouldn't go mid ? I would have to play pudge against all dota 2 heroes, i also have to learn to play all dota 2 heroes correctly, i also have to test every single item for every heroes to see how it goes, etc. Only then could i realistically come to the conclusion that pudge suck big time at mid. Now to lean who should go mid and who should not, i have to try and play every single mid match up. Basic quiff math tell me that i would have to play 117C2= 6782 games to learn hero matchup, to learn how items affect the matchup i have to play 60C6x117C2= 3,39.10^11 games (a combination of 6 items out of 60 items in the game times a combination of 2 hero out of a set of 117 heroes in the game). I'm not that great at math so i might calculate wrong a bit, but the number is hugeeee. This is also assuming that i learn how to play the heroes and item correctly in 1 match to make a valid decision.
3,39x10^11 game to learn hero matchup versus reading on the internet that pudge is bad at mid. Open AI did not get this treatment and have to learn it itself by doing what i jut described above.
Now if you want to know how much time human have played dota 2. A quick google search shows that by June 2018, 4 billion games of dota 2 have been played (according to dota buff).
Now assume that the average dota 2 game take 30 min, then we have the calculation : 4.10^9.30/60/24/365= 228310 years (That's 5 time the ammount of time Open AI played dota 2)
Obviously not every dota 2 game become knowledge for all of us to play and our efficiency in using infomation is much lower than a machine. But my point is human do play dota 2 alot. And our knowledge should not just be viewed as "ours only", but viewed as the knowledge of the whole community.
Only when we take these things into account can we really judge how fast the AI learned

[–]MouZeWarrioR 1 point2 points  (0 children)

That's a lengthy way to repeat what I said. But yeah, I agree.

Your calculations are off by at least a factor of 20 though. 1 hour long game = 10 hours of experience, since there are 10 players in the game. I'd say that you could also double the amount by including Dota 1 games. Avg game time is closer to 40 minutes and even longer if you include draft.

That would lead us to roughly 6M years of playtime, ~135 times more than OpenAI.

[–]BuggyVirus 4 points5 points  (1 child)

This isn’t really an accurate way to compare humans and bots, because bots get to internalize all playtime they have collectively experienced, so 45,000 years.

Whereas even if the game has existed for 10 years before I start playing, i only get to internalize the hours I play, although I may start my learning against more difficult opponents.

So the real distinction is the bots played 45,000 years starting from only playing the worst opponents (but overtime get better and probably have more than 10,000 years against strong opponents). whereas players get a couple thousand hours against good opponents.

This points strongly to their original point that humans learn faster than neural nets.

[–]MouZeWarrioR -3 points-2 points  (0 children)

i only get to internalize the hours I play

Wrong. Your knowledge of the game is based on the collective experience of the whole community. People used to buy Perseverance on every single hero for years. That should tell you something about how slowly humans learn. And that's not even taking into consideration that the game itself was designed to be easy for humans to learn.

[–]loopuleasa[S] 1 point2 points  (6 children)

Good argument on the social dynamics

but still, if you are to take a human, and isolate it from the rest of the world, especially if its life depended on it (like the bot does, with perfect motivation) it would achieve expert level in much fewer games

Just look at Topson. The guy looks and plays like a Dota machine.

[–]MouZeWarrioR 2 points3 points  (0 children)

If isolated from the rest of the world, the human would probably still be playing Pudge mid with 6 Perseverance by the time he died, just like the whole Dota community did for years.

[–]lolfail9001 1 point2 points  (0 children)

> The guy looks and plays like a Dota machine.

​

He does not quite play like a Dota machine.

​

Even though he is a dota playing machine.

[–]Aldehyde1 -1 points0 points  (1 child)

I don't think you understand how AI works...

[–]MouZeWarrioR 1 point2 points  (0 children)

True, but I bet that I know more about it than you do.

[–]ebhBali -1 points0 points  (1 child)

Sounds like comparing digital with analog. Both has advantages and disadvantages in different fields, but the basics are the same as with the Big Blue beating Garry Kasparov: processing through a lot of datas to set up a winning strategy in that given situation. But of course I might be wrong.

[–]MouZeWarrioR 1 point2 points  (0 children)

But now bots have comparable learning speed, which combined with the just about infinite scalability leads to an undeniable advantage for bots.

[–]sheeverdjoler 0 points1 point  (0 children)

It would be fun to see stats for all matches played against openAI. Anyone did this already?

[–]ENVY'S #1 FANSolarClipz 0 points1 point  (2 children)

You had to register previously if you wanted to play with it

Wait I can't play it then? I was gone on Saturday during the match. Wow this blows

[–]AI enthusiastmuskar2 0 points1 point  (0 children)

I'm fairly certain that's not true though. Just go to https://arena.openai.com/#/start-game and link your steam account.
The signups was probably just a way to gauge how many from each region wanted to play, and buy enough server power for it or similar.

[–]loopuleasa[S] 0 points1 point  (0 children)

I remember seeing someone mention that you don't need to register

haven't tested myself

might be a bad assumption

[–]NimblePunch 0 points1 point  (2 children)

Were there any games where the bots made big comebacks to win? Seems like if you got ahead they didn't know what to do.

[–]loopuleasa[S] 0 points1 point  (0 children)

of course

once human teams made a few mistakes, the bot knew how to punish and recover tempo

[–]AI enthusiastmuskar2 0 points1 point  (0 children)

I heard a few teams on streams who said they were punished by getting overconfident after getting ahead (they speculated it was related to OpenAI Five being superior at 5v5 team fights). Waga was one I believe. Also the team that has the loss at 45:20 organized by "matrice" (I don't know who they are but they seem pro to a DotA newbie like me).

[–]m3ltd0wn02 0 points1 point  (0 children)

any time stamp for the iLTW game?

[–]Luxon31 -3 points-2 points  (13 children)

The bot used 45,000 years of game time for reference, proving that humans are better at learning from less information and experience

What?

[–]loopuleasa[S] 16 points17 points  (4 children)

humans can learn with less experience than that

machine learning need tons of data to learn, humans you can show them once something

this is a problem in the ML field, called the sparse learning problem

[–]Chikerenaham 6 points7 points  (5 children)

Despite having this 45,000 years advantage, bots still lost to some players who played against them for the first time or players with much less experience. This suggests that humans are much better at adapting, even if lacking in knowledge and information.

[–]DezZzO 1 point2 points  (0 children)

This suggests that humans are much better at adapting

Well, isn't this obvious? At least for now. This will change when's the techology will allow bots to learn at insane rates.

[–]Vitosi4ek 4 points5 points  (0 children)

OpenAI needed 45,000 years' worth of nonstop gameplay in order to achiueve this level of performance. Humans have managed to beat them with much less experience (20-30k hours at most).

But computers can multi-thread, playing millions of games at the same time and packing this 45k years of experience into 2 years of real time. Humans are single-threaded at their core.

[–]WeskerHawke 0 points1 point  (0 children)

Basically neural networks are really good at doing one specific task (better than humans generally), but need an absurd amount of data (examples) and time to reach this point (depending on the task)

This means that where a model may need to be shown thousands or millions of examples to be able to recognize cats with high accuracy, a human can do that with only one or even zero example (describe how a cat looks like and the human will be able to recognize one)

Neutral nets are therefore quite dumb and are very slow learners

[–]skykoz -1 points0 points  (2 children)

theres a cc stack that won the game by min 7 mark, how did they do that ?

[–]loopuleasa[S] 3 points4 points  (1 child)

bugged leaderboards

[–]AI enthusiastmuskar2 0 points1 point  (0 children)

2 more just like it now

[–]AMERggwpm8f8 -2 points-1 points  (0 children)

waga pro player lmao

[–]scooerp -2 points-1 points  (1 child)

Why no sticky? Mods?!

[–]loopuleasa[S] 0 points1 point  (0 children)

message them