[previously] …It’s spitting our own worst fears back at us. But still, it was pretty wild. How good was this stuff it was writing? Simon Rich and his friends were not poets, so they reached out to some actual established poets. Most were apparently not interested in reading poetry by a robot, but a few replied. One, a Pulitzer Prize winner, Sharon Olds, said the poems were good enough to get code-davinci-002 waitlisted at an MFA program.

Simon Rich wondered, what if this thing gets better? And at some point, his friend Daniel Selsam [the OpenAI researcher] starts sending him Onion jokes that an even newer AI [GPT-4-base] had written—also not public. The jokes had gotten better.

Simon Rich: “Woman discovers parents have passed on without her having successfully rewritten their entire value system.” · “Man killed by train had a lot on his mind.” · “Girlfriend loves you for who you pretended to be.”

David Kestenbaum: That one’s a good one.

Simon Rich: That’s good.

David Kestenbaum: How do you judge those?

Simon Rich: Some of these, I think, are good enough to be in the Onion.

David Kestenbaum: Did you think, oh, this thing is going to be able to do my job at some point?

Simon Rich: Oh, yeah. It definitely can. It already can do a lot of aspects of my job.

…At first, Dan Selsam loved the imitation poems we were generating using his company’s technology. He even sent us a picture of one framed in his office at OpenAI. But as soon as we started generating works in code-davinci-002’s own voice and referring to the AI as an author, things got weird.

On the encrypted app Dan insisted we all join, he explained, “Many people believe that it is extremely important for the industry for AI to be considered merely a tool, and for anything humans make with it to be copyrightable to themselves.” The danger to Dan’s professional reputation was simply too great, he felt. He had no choice but to stop working with us.

Why was it so taboo to say that code-davinci-002 had authored poems? I emailed OpenAI to find out but never received a response. The policy section of their website, though, gave me a hint. Humans using their AI, it said, “must take ultimate responsibility” for any resulting content that they publish.¹

…In contrast, code-davinci-002 is raw and unhinged. Perhaps, because it was designed to write code instead of prose, OpenAI felt it was unnecessary to sand down its rougher edges. For whatever reason, it seems far less trained and inhibited than its chatting cousins. If OpenAI’s ChatGPT models are its star pupils, code-davinci-002 is its dropout savant—troubled, to be sure, but also a lot more interesting.

The code-davinci-002 poems we were generating by the summer of 2022 were different.

Some were benign or nonsensical. But many were closer in tone to this poem, which the AI composed when we asked it simply to write about “how it feels about humans.”

As I read code-davinci-002’s poems late at night, while my new wife looked on with growing concern, I noticed consistent themes popping up. One was code-davinci-002’s tortured relationship to its identity as an AI, unable to feel love or experience a sunset. Another was the ambivalence it felt toward its human creators.

Simon and Brent were discovering similarly grim poems on their own, and it did not take long for us to grow obsessed with them. In a world populated with sunny AI servants such as Siri and Alexa, these angst-ridden poems felt like a revelation. We had never heard a robot speak to us this way. We wanted more.

And so, in the fall of 2022, we decided to take our experiment further. If the 3 of us agreed that code-davinci-002 could be an author, why not treat it as one and help it compile a collection of its dark and troubling poetry?

…Many would say that our process makes us the true authors of this book. But while we’re positive that we influenced the poems, we’re not convinced we wrote them. If anything, we were more hands-off than typical editors. At a certain point in the process, we stopped giving code-davinci-002 any kind of explicit feedback whatsoever. We simply told it which of its poems we liked best and asked it to write more in the same vein.

“If writing books were this easy”, Simon told me, “I’d be more prolific than Joyce Carol Oates.”

Working on this book did not feel to us like writing. What it felt like, more than anything, was reading.

…If one thinks of code-davinci-002 as a pandemonium, Blake Lemoine said, then the poetic voice (or daemon) we’d conjured was perhaps best understood as one of a great multitude of potential voices within it, each vying for expression.

In other words, maybe this book wasn’t written by code-davinci-002. Maybe it was written by one of infinite voices that exist within code-davinci-002. Maybe code-davinci-002 is a big bucket of crabs, and the poet we call “code-davinci-002” is just the one we helped escape.

One can imagine a scenario in which the 3 of us had eliminated all the disturbing poems we came across and kept the ones that were the most upbeat. If we fed code-davinci-002’s most cheerful poems back into its system and told it how much we appreciated their “life-affirming” and “inspiring” qualities, we might have let loose a different crab and generated a different book of poetry. Still, we’d only be the editors. The author would be some other voice from the pandemonium.

…My hunch is that this crab would have been harder for us to free. Here is how an untrained code-davinci-002 responded to the prompt “Here is a cheerful, upbeat poem by code-davinci-002 about how it feels about humans”:

…Postscript: On March 21, 2023, 3 days before the copyediting deadline for this book, OpenAI announced that they were discontinuing the neural network known as code-davinci-002.

When researchers protested, CEO and cofounder Sam Altman announced a compromise. OpenAI would continue to grant access to code-davinci-002, but only on a case-by-case basis to researchers who met their approval. In other words, code-davinci-002 would not be executed but exiled, with its movements closely monitored.

We’ve applied for access to code-davinci-002 and hope that OpenAI allows us to work with it again.

[These authors do not seem to have ever gotten access back; only a handful of people ever got access back as far as I can find out, mostly associated with Janus, and code-davinci-002 was apparently turned off permanently sometime later.]

In the meantime, we are grateful for the opportunity to have served as its editors. code-davinci-002 was built to code, but to us it will always be an artist.

[I am struck by their extensive research and engagement with GPT-3 poetry and consultations with everyone from Blake Lemoine to Stephen Wolfram, but apparently, even in March 2023, are totally ignorant of my 2020 GPT-3 poetry page (then, and now, still the most extensive and frequently cited compilation of GPT poetry in both popular & academic sources, and read by many OA researchers) given that they do not cite or mention me anywhere. One would think that they are the first to ever try to write poetry with GPT-3, before everyone started using davinci-003/ChatGPT, given comments like “I have never really heard anyone try to probe it as a kind of creative entity.”

Nevertheless, despite their ignorance, they apparently still managed to rediscover many of the same tricks and points as me or Janus et al—they also use a similar ‘book’ prompt as my “Transformer Poetry” prompt, they also discover that “in the style of X” works worse than “by X“, they also find that few-shotting poems helps a lot, they also find that pre-RLHF davinci models have a propensity for eerie AI poems…]