- See Also
-
Links
- “Cached Transformers: Improving Transformers With Differentiable Memory Cache”, Zhang et al 2023
- “In-Context Autoencoder for Context Compression in a Large Language Model”, Ge et al 2023
- “Learning to Compress Prompts With Gist Tokens”, Mu et al 2023
- “Token Turing Machines”, Ryoo et al 2022
- “MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition”, Wu et al 2022
- “ABC: Attention With Bounded-Memory Control”, Peng et al 2021
- “Memorizing Transformers”, Wu et al 2021
- “Recursively Summarizing Books With Human Feedback”, Wu et al 2021
- “∞-Former: Infinite Memory Transformer”, Martins et al 2021
- “Perceiver IO: A General Architecture for Structured Inputs & Outputs”, Jaegle et al 2021
- “TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?”, Ryoo et al 2021
- “Not All Memories Are Created Equal: Learning to Forget by Expiring”, Sukhbaatar et al 2021
- “Perceiver: General Perception With Iterative Attention”, Jaegle et al 2021
- “Learning to Summarize Long Texts With Memory Compression and Transfer”, Park et al 2020
- “Memory Transformer”, Burtsev et al 2020
- “Compressive Transformers for Long-Range Sequence Modelling”, Rae et al 2019
- “Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks”, Lee et al 2018
- “Generating Wikipedia by Summarizing Long Sequences”, Liu et al 2018
- “Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-Iks”, White et al 2011
- Sort By Magic
- Miscellaneous
- Link Bibliography
See Also
Links
“Cached Transformers: Improving Transformers With Differentiable Memory Cache”, Zhang et al 2023
Cached Transformers: Improving Transformers with Differentiable Memory Cache
“In-Context Autoencoder for Context Compression in a Large Language Model”, Ge et al 2023
In-context Autoencoder for Context Compression in a Large Language Model
“Learning to Compress Prompts With Gist Tokens”, Mu et al 2023
“Token Turing Machines”, Ryoo et al 2022
“MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition”, Wu et al 2022
MeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
“ABC: Attention With Bounded-Memory Control”, Peng et al 2021
“Memorizing Transformers”, Wu et al 2021
“Recursively Summarizing Books With Human Feedback”, Wu et al 2021
“∞-Former: Infinite Memory Transformer”, Martins et al 2021
“Perceiver IO: A General Architecture for Structured Inputs & Outputs”, Jaegle et al 2021
Perceiver IO: A General Architecture for Structured Inputs & Outputs
“TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?”, Ryoo et al 2021
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?
“Not All Memories Are Created Equal: Learning to Forget by Expiring”, Sukhbaatar et al 2021
Not All Memories are Created Equal: Learning to Forget by Expiring
“Perceiver: General Perception With Iterative Attention”, Jaegle et al 2021
“Learning to Summarize Long Texts With Memory Compression and Transfer”, Park et al 2020
Learning to Summarize Long Texts with Memory Compression and Transfer
“Memory Transformer”, Burtsev et al 2020
“Compressive Transformers for Long-Range Sequence Modelling”, Rae et al 2019
“Set Transformer: A Framework for Attention-Based Permutation-Invariant Neural Networks”, Lee et al 2018
Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks
“Generating Wikipedia by Summarizing Long Sequences”, Liu et al 2018
“Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-Iks”, White et al 2011
Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-iks
Sort By Magic
Annotations sorted by machine learning into inferred 'tags'. This provides an alternative way to browse: instead of by date order, one can browse in topic order. The 'sorted' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.
Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.
summarization
perceiver
memory-enhancement
Miscellaneous
Link Bibliography
-
https://arxiv.org/abs/2304.08467: “Learning to Compress Prompts With Gist Tokens”, Jesse Mu, Xiang Lisa Li, Noah Goodman -
https://arxiv.org/abs/2203.08913#google: “Memorizing Transformers”, Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, Christian Szegedy -
https://arxiv.org/abs/2109.10862#openai: “Recursively Summarizing Books With Human Feedback”, Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, Paul Christiano -
https://arxiv.org/abs/2107.14795#deepmind: “Perceiver IO: A General Architecture for Structured Inputs & Outputs”, -
https://arxiv.org/abs/2103.03206#deepmind: “Perceiver: General Perception With Iterative Attention”, Andrew Jaegle, Felix Gimeno, Andrew Brock, Andrew Zisserman, Oriol Vinyals, Joao Carreira -
https://arxiv.org/abs/1911.05507#deepmind: “Compressive Transformers for Long-Range Sequence Modelling”, Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, Timothy Lillicrap -
2011-white.pdf: “Phonotactic Reconstruction of Encrypted VoIP Conversations: Hookt on Fon-Iks”, Andrew M. White, Austin R. Matthews, Kevin Z. Snow, Fabian Monrose