“What Makes a Good Conversation? How Controllable Attributes Affect Human Judgments”, 2019-02-22 (; similar):
A good conversation requires balance—between simplicity and detail; staying on topic and changing it; asking questions and answering them. Although dialogue agents are commonly evaluated via human judgments of overall quality, the relationship between quality and these individual factors is less well-studied.
In this work, we examine two controllable neural text generation methods, conditional training and weighted decoding, in order to control 4 important attributes for chitchat dialogue: repetition, specificity, response-relatedness and question-asking.
We conduct a large-scale human evaluation to measure the effect of these control parameters on multi-turn interactive conversations on the PersonaChat task.
We provide a detailed analysis of their relationship to high-level aspects of conversation, and show that by controlling combinations of these variables our models obtain clear improvements in human quality judgments.