“Prompt-And-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer With Small Language Models”, 2022-05-23 ():
We propose a method for arbitrary textual style transfer (TST)—the task of transforming a text into any given style—utilizing general-purpose pre-trained language models. Our method, Prompt-and-Rerank, is based on a mathematical formulation of the TST task, decomposing it into 3 constituent components: textual similarity, target style strength, and fluency.
Specifically, our method first uses zero-shot or few-shot prompting to obtain a set of candidate generations in the target style, and then re-ranks these candidates according to a combination of the 3 components above.
Empirically, our method enables small pre-trained language models to perform on par with state-of-the-art large-scale models while consuming two orders of magnitude less compute and memory.
Finally, we conduct a systematic investigation of the effect of model size and prompt design (eg. prompt paraphrasing and delimiter-pair choice) on style transfer quality across 7 diverse textual style transfer datasets.