Creative writing by OpenAI’s GPT-3 model, demonstrating poetry, dialogue, puns, literary parodies, and storytelling. Plus advice on effective GPT-3 prompt programming & avoiding common errors.

The goal for style transfer prompt programming is to find a zero-shot prompt: a prompt which, without requiring any handwritten examples of parodies/versions, gets GPT-3 to do style transfer in general, and so a prompt which could fully automate style transfer—you could just write a program using the API to take two specified pieces of text (the content, and the style description/author name X) to get out a third piece of text which is the content as written in X form. Right now, the literary parodies require at least one human-written example to properly persuade GPT-3 to rewrite the text, as opposed to generating critical commentary or metadata or webpage-like continuations.

I experimented with a prompt which uses explicit descriptions of parodies and describing rewrites as a prompt wrapped around a content text, and it… sort of works. The difficulty is that sometimes GPT-3 will spit out the original content verbatim, sometimes it will instead create a new passage entirely in the style description, and sometimes it will do the desired rewrite flawlessly—but I can’t figure out how to tune the prompt to do the third one reliably. Adding more descriptive words does not seem to change it, and while adding in words from the original content passage (even just the first one or two) does largely eliminate the risk of entirely new passages being generated, it triggers more copying behaviors (and is not as useful for zero-shot style transfer since the prefix words would need to be sensible in the target version too, which is not necessarily the case). It is infuriating because GPT-3 clearly can do it easily because it does do it a decent fraction of the time, but no matter how I tweak the prompt trying to hammer in the rewrite, GPT-3 will as oft as not go off in another direction.

Below are some samples from my attempts; I try to rewrite a vaguely Dickens/Jane Austen-like story (generated by GPT-3) to a Tolkien story: