“What If the Robots Were Very Nice While They Took Over the World?”, Virginia Heffernan2023-09-26 (Diplomacy AI, AI safety)⁠:

…When I got home…there was an email from Andrew Goff, widely considered the greatest Diplomacy player of all time…Forever crisscrossing the world for tournaments and his corporate job, Goff comes across as more gregarious than most elite players of board games.

Goff is also known for a brilliantly subversive, kill-’em-with-kindness style of gameplay. As Siobhan Nolen, the former president of the North American Diplomacy Federation, put it, “It hurts less to lose against somebody like Andrew.” In Diplomacy, players are sometimes forced to choose which attacks on their territory to repel and which to surrender to. Players often let Goff roll his forces in because they know that he, unlike many others, won’t be a dick about it.

There are excellent Diplomacy players who rage and issue threats, hollow and otherwise: “If you backstab me, I will throw the game.” Goff is not one of these. Even his breakup notes are masterpieces of directness and decency. “Apologies, Turkey! I decided it was in my best interest to work with Russia now. I hope there are no hard feelings.” In his congeniality is also empathy. “I genuinely feel bad for players when they get beaten, even if it is me beating them”, Goff told me. I believed him.

The email was about CICERO, a Diplomacy-playing AI that Goff helped create for Meta AI. Last fall, CICERO managed to best Goff in several games, sometimes partnering with weaker players to bring him down. Noam Brown and Adam Lerer, who were part of the immense team of experts in game theory, natural language processing, and Diplomacy that created the AI, both say that CICERO is the most human-like AI they’ve ever created. Lerer, who now works at DeepMind, goes further: CICERO may be the most human-like AI on earth.

…It plays the same magnanimous game Goff does. In one memorable showdown, Lerer told me, CICERO played Russia and allied with a human who played Austria. Throughout the game, Lerer said, CICERO was “really nice and helpful to Austria, although it maneuvered in its discussions with other players to make sure Austria was weakened and eventually lost. But at the end of the game [the human playing] Austria was overflowing with praise for CICERO, saying they really liked working with it and were happy it was winning.”…When CICERO wins, Goff told me, there is no gloating, “no ‘Ha ha, you loser’ talk.” Instead, “the talk is much more, ‘Your position isn’t great, but we all have games like that sometimes.’”…This filled me with relief. Maybe AI will just amplify what’s best about humans. Maybe AI will become a buoyant tribute band for our entire species. Maybe AI will be a delight—and a force humans will be content to lose to. We’ll go down in peace. We really liked working with you, robots, and are happy you are winning.

…Perhaps a little R²-D2 could win me over as an ally, not with human kindness but by sharing my reading of a situation and presenting me with elegant, data-driven options for how to address it.

When I asked Lerer about my R²-D2 idea, he concurred. “I actually think a human that used CICERO as an assistant to develop tactical and strategic plans, but who could navigate some of the human aspects better than CICERO—such as when it is safe to lie, or how to avoid irritating an ally—would be super strong.”

CICERO definitely says “Awesome!” too much. But it can be especially irritating in that signature AI way: It sometimes hallucinates. It proposes illegal moves. Worse yet, it denies saying something it just said. Faced with these glitches, CICERO’s human opponents would sometimes get mad. But they didn’t guess it was an AI. They thought it was drunk. And perhaps these personality glitches are a small price to pay for the bot’s deep reserves of raw intelligence and foresight.

…Kostick believes that while he “would have been delighted with the practical results of CICERO’s website play”, Meta’s project misses the mark. CICERO’s glitches, Kostick believes, would make it easy to outwit with spam and contradictory inputs. Moreover, in Kostick’s opinion, CICERO doesn’t play real Diplomacy. In the online blitz, low-stab game CICERO does play, the deck is stacked in its favor, because players don’t have to lie, which CICERO does badly. (As Lerer told me, “CICERO didn’t really understand the long-term cost of lying, so we ended up mostly making it not lie.”) Kostick believes CICERO’s metagame is off because it “never knowingly advocates to a human a set of moves that it knows are not in the human’s best interest.” Stabbing, Kostick believes, is integral to the game. “A Diplomacy player who never stabs is like a grandmaster at chess who never checkmates.”

With some trepidation, I mentioned Kostick’s complaint to Goff.

Unsurprisingly, Goff scoffed. He thinks it’s Kostick and his generation who misunderstand the game and give it its unfair reputation for duplicity. “CICERO does stab, just rarely”, Goff said. “I reject outright that [compelling players to stab] was Calhamer’s intent.”

…So here’s the rare AI story that doesn’t end with an existential reckoning for humankind, I thought. We’re not staring into an abyss. Bots like CICERO are going to understand our wants and needs and align with our distinctive worldviews. We will form buddy-movie partnerships that will let us drink from their massive processing power with a spoonful of sugary natural language. And if forced at the end of the road to decide whether to lose to obnoxious humans or gracious bots, we won’t give it a thought. We’ll change our wills, leave them all we have, and let them roll their upbeat tanks right over our houses.