Bibliography (5):
Attention Is All You Need
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Linear Transformers Are Secretly Fast Weight Programmers
Wikipedia Bibliography:
https://en.wikipedia.org/wiki/Gating_mechanism :
https://en.wikipedia.org/wiki/Gating_mechanism
https://en.wikipedia.org/wiki/Delta_update_rule :
https://en.wikipedia.org/wiki/Delta_update_rule