Comparison with other methods: - Long context architectures like FlashAttention allow scaling attention to longer sequences, but don't directly address recency bias issues. - Retrieval augmented generation (RAG) combines a retriever and generator. Performance depends on retriever recall and context length. Attention sorting could improve context usage. - Methods like chain-of-thought prompting and scratchpads manipulate the prompt context directly in text space. Attention sorting works directly with model attention weights. - kNN methods like Memorizing Transformer use efficient retrieval over a long context history instead of full attention. The sparsity may improve context usage like attention sorting. - Attention truncation methods like nucleus sampling prune the attention distribution. But without fine-tuning they hurt performance in this setting. - Some work uses attention for document retrieval, similar to attention sorting's re-ranking. But attention sorting operates at the granularity of passages rather than full documents. In comparison to these approaches, attention sorting is a straightforward plug-in method that does not require architectural changes, training, or prompt engineering expertise. It operates purely at inference time. The ability to sort iteratively and stop at an optimal point is also unique. The strong results suggest attention sorting is a promising technique compared to prior work, despite its simplicity. Combining it with other methods like dense retrieval may yield further improvements.