Bibliography:

  1. ​ ‘self-attention’ tag

  2. ​ ‘dynamic evaluation (NN)’ tag

  3. ​ ‘recurrent Transformers’ tag

  4. ​ ‘GPT’ tag

  5. ​ Cached Transformers: Improving Transformers with Differentiable Memory Cache

  6. ​ In-context Autoencoder for Context Compression in a Large Language Model

  7. ​ Learning to Compress Prompts with Gist Tokens

  8. ​ Token Turing Machines

  9. ​ MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition

  10. ​ ABC: Attention with Bounded-memory Control

  11. ​ Memorizing Transformers

  12. ​ Recursively Summarizing Books with Human Feedback

  13. ​ ∞-former: Infinite Memory Transformer

  14. ​ Perceiver IO: A General Architecture for Structured Inputs & Outputs

  15. ​ TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?

  16. ​ Not All Memories are Created Equal: Learning to Forget by Expiring

  17. ​ Perceiver: General Perception with Iterative Attention

  18. ​ Learning to Summarize Long Texts with Memory Compression and Transfer

  19. ​ Memory Transformer

  20. ​ Compressive Transformers for Long-Range Sequence Modeling

  21. ​ Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks

  22. ​ Generating Wikipedia by Summarizing Long Sequences

  23. ​ Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-iks

  24. ​ 2021-jaegle-figure1-perceiverarchiture.jpg

  25. ​ https://jvns.ca/blog/2013/10/24/day-16-gzip-plus-poetry-equals-awesome/

  26. ​ e454aae3f37ac0ea545c6e41c3136c2e7c96bd4b.html

  27. ​ https://shkspr.mobi/blog/2024/01/compressing-text-into-images/

  28. ​ c2a500e080a1d9eec0e2dd2177d9852af9214c62.html

  29. ​ https://www.infinitepartitions.com/art001.html

  30. ​ eca62e7befb6f7bd64f8644ecc221c6b2cede0d2.html

  31. ​ Learning to Compress Prompts with Gist Tokens

  32. ​ https%253A%252F%252Farxiv.org%252Fabs%252F2304.08467.html

  33. ​ Memorizing Transformers

  34. ​ Yuhuai (Tony) Wu’s Home Page

  35. ​ https%253A%252F%252Farxiv.org%252Fabs%252F2203.08913%2523google.html

  36. ​ Recursively Summarizing Books with Human Feedback

  37. ​ Jan Leike

  38. ​ https%253A%252F%252Farxiv.org%252Fabs%252F2109.10862%2523openai.html

  39. ​ Perceiver IO: A General Architecture for Structured Inputs & Outputs

  40. ​ https%253A%252F%252Farxiv.org%252Fabs%252F2107.14795%2523deepmind.html

  41. ​ Perceiver: General Perception with Iterative Attention

  42. ​ https%253A%252F%252Farxiv.org%252Fabs%252F2103.03206%2523deepmind.html

  43. ​ Compressive Transformers for Long-Range Sequence Modeling

  44. ​ https%253A%252F%252Farxiv.org%252Fabs%252F1911.05507%2523deepmind.html

  45. ​ Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-iks

  46. ​ %252Fdoc%252Fcs%252Fsecurity%252F2011-white.pdf.html

  47. Wikipedia Bibliography:

    1. ​ Paul Christiano

    2. ​ David Ding

    3. ​ Andrew Brock

    4. ​ Andrew Zisserman

    5. ​ Oriol Vinyals

    6. ​ Timothy Lillicrap