“TART: Task-Aware Retrieval With Instructions”, Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, Wen-tau Yih2022-11-16 (retrieval AI, T5 Transformer)⁠:

We study the problem of retrieval with instructions, where users of a retrieval system explicitly describe their intent along with their queries, making the system task-aware. We aim to develop a general-purpose task-aware retrieval systems using multi-task instruction tuning that can follow human-written instructions to find the best documents for a given query.

To this end, we introduce the first large-scale collection of ~40 retrieval datasets with instructions, and present TART, a multi-task retrieval system trained on the diverse retrieval tasks with instructions.

TART shows strong capabilities to adapt to a new task via instructions and advances the state-of-the-art on two zero-shot retrieval benchmarks, BEIR and LOTTE, outperforming models up to 3× larger.

We further introduce a new evaluation setup to better reflect real-world scenarios, pooling diverse documents and tasks. In this setup, TART outperforms competitive baselines, further demonstrating the effectiveness of guiding retrieval with instructions.