“A Hundred Visions and Revisions”, 2020-03-11 (; backlinks; similar):
“A Hundred Visions and Revisions” is a computer program that alters poems using a neural-network language model. It works by replacing the individual words of the text, one by one, with other words that are more probable according to the BERT language model, while preserving rhyme and meter; in effect, this process banalifies the poem, replacing its linguistic distinctiveness with normativity. The program can also attempt to revise a poem to be about a different topic. As an example, I started with the poem “The Sick Rose” by William Blake:
O Rose thou art sick.
The invisible worm,
That flies in the night
In the howling storm:Has found out thy bed
Of crimson joy:
And his dark secret love
Does thy life destroy.Here is the revision:
By God thou art blessed.
The invisible man,
Who walks in the night
In a hooded cloak:Has found both his source
Of body heat:
And his own power that
Makes his life complete.…It is also possible to have the program revise a poem to be about a different topic while retaining rhyme, meter, and some other, subtler traces of the original. When I created the finetuned neural network, I included annotations indicating the title and author of each poem. This enables the AI to pick up on patterns in the relation between title and poem. You can then feed in hints about the poem’s title, and the AI will alter the text accordingly…All of these revisions retain the rhyme, meter, and punctuation of the original (excepting the slant-rhyme of “eye” and “symmetry”, which the current code cannot detect). If these formal constraints are lifted, the poem will degenerate into prose that bears little relation to the original…I also included a feature that enables you to bias the output toward an arbitrary vocabulary. I tested this out using the data from Iain Barr’s analysis of the vocabulary of heavy metal lyrics
How it works: The BERT model is capable of guessing a word that is “masked”—that is, hidden from the model. To pick an example from the documentation for the implementation I used, one could enter “Who was Jim Henson? Jim Henson was a [MASK]”; the model predicts that the masked word is “puppeteer”. The point of this is to enable the computer to perform question-answering tasks, language modeling standing as a surrogate for more general intelligence. But it is also possible to use the model’s predictions to alter an existing text. To do this, my program tries masking each word in the text and guessing what word should be in that position. For instance, suppose we are looking at this text:
Tyger Tyger, burning bright, in the forests of the night
We try masking each word in order; for instance, at one point we will end up with this:
Tyger Tyger, burning bright, in the [MASK] of the night
The program uses the neural network to predict what word appears in the masked position, subject to various constraints such as rhyme and meter. In this case, the BERT model guesses “middle”, with probability 0.6762. On the other hand, the word that is actually in that position—“forests”—gets probability 0.000076159. We divide the latter by the former to get a score for this potential change: 0.0001126. Since this score happens to be the lowest for any word in the text, the program selects the word “forests” for replacement, giving us this revision:
Tyger Tyger, burning bright, in the middle of the night
The program then repeats this process until there are no more “improvements” to be made.