“Encode, Tag, Realize: High-Precision Text Editing”, 2019-09-03 ():
We propose LaserTagger—a sequence tagging approach that casts text generation as a text editing task.
Target texts are reconstructed from the inputs using 3 main edit operations: keeping a token, deleting it, and adding a phrase before the token. To predict the edit operations, we propose a novel model, which combines a BERT encoder with an autoregressive Transformer decoder.
This approach is evaluated on English text on 4 tasks: sentence fusion, sentence splitting, abstractive summarization, and grammar correction. LaserTagger achieves new state-of-the-art results on 3 of these tasks, performs comparably to a set of strong seq2seq baselines with a large number of training examples, and outperforms them when the number of examples is limited.
Furthermore, we show that at inference time tagging can be more than two orders of magnitude faster than comparable seq2seq models, making it more attractive for running in a live environment.