--- title: '‘dynamic evaluation (NN)’ directory' description: "Bibliography for directory ai/nn/dynamic-evaluation, most recent first: 4 related tags, 58 annotations, & 17 links ()." thumbnail: /doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure1-dynamicevaluationbeatsstaticllmsonlanguagemodelingofconcatenatedprojectgutenbergbooks.png thumbnail-text: '' thumbnail-css: "outline" created: 2020-03-04 modified: 2026-09-17 status: in progress previous: /doc/ai/nn/diffusion/midjourney/landscape/index next: /doc/ai/nn/fully-connected/index confidence: log importance: 0 css-extension: dropcaps-not index: True backlink: False ... ::: {#manual-annotation .abstract-small .abstract-tag-directory} [\[page summary\]](/doc/ai/nn/dynamic-evaluation/abstract "Transclude link for doc/ai/nn/dynamic-evaluation/ tag-abstract page."){#gwern-doc-ai-nn-dynamic-evaluation-abstract .link-annotated-partial .include-content-core .include-strict .link-page} ::: # See Also ::: {#see-alsos .directory-indexes .columns} - [Parent ('*`NN`{=html}*' tag)](/doc/ai/nn/index "Link to parent directory 'doc/ai/nn/' (ascending)"){#_ujmXzDuq .link-annotated .link-tag .directory-indexes-upwards rel="tag"} - [*`compressed Transformer`{=html}*](/doc/ai/nn/transformer/attention/compression/index "‘compressed Transformer’ directory"){#_zhl07LRT .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`self-attention`{=html}*](/doc/ai/nn/transformer/attention/index "‘self-attention’ directory"){#_5XTITqOw .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`continual learning`{=html}*](/doc/reinforcement-learning/meta-learning/continual-learning/index "‘continual learning’ directory"){#_CPEYsW4o .link-annotated .link-tag .directory-indexes-sideways rel="tag"} - [*`meta-learning`{=html}*](/doc/reinforcement-learning/meta-learning/index "‘meta-learning’ directory"){#_LXLtSdIU .link-annotated .link-tag .directory-indexes-sideways rel="tag"} ::: # Gwern {.display-pop-not} ## `“Nenex: A Neural Personal Wiki Idea”, Gwern 2023`{=html} {#gwern-nenex-section} **[`Nenex: A Neural Personal Wiki Idea`{=html}](/nenex "'Nenex: A Neural Personal Wiki Idea', Gwern 2023"){.link-annotated .id-not .include-annotation}** # Links {.display-pop-not} ## `“RoboTTT: Context Scaling for Robot Policies”, Jiang et al 2026`{=html} {#jiang-et-al-2026-section} **[`RoboTTT: Context Scaling for Robot Policies`{=html}](https://arxiv.org/abs/2607.15275#nvidia "'RoboTTT: Context Scaling for Robot Policies', Jiang et al 2026"){.link-modified-recently .link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search”, Sokota et al 2025`{=html} {#sokota-et-al-2025-section} **[`Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search`{=html}](https://arxiv.org/abs/2511.07312 "'Superhuman AI for Stratego Using Self-Play Reinforcement Learning and Test-Time Search', Sokota et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“The Coverage Principle: How Pre-Training Enables Post-Training”, Chen et al 2025`{=html} {#chen-et-al-2025-section} **[`The Coverage Principle: How Pre-Training Enables Post-Training`{=html}](https://arxiv.org/abs/2510.15020 "'The Coverage Principle: How Pre-Training Enables Post-Training', Chen et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“LoRA Without Regret”, Inc 2025`{=html} {#inc-2025-section} **[`LoRA Without Regret`{=html}](https://thinkingmachines.ai/blog/lora/ "'LoRA Without Regret', Inc 2025"){.link-annotated-partial .id-not .include-annotation}** ## `“The Hidden Drivers of HRM’s Performance on ARC-AGI”`{=html} {#_L3jjFx98-section} **[`The Hidden Drivers of HRM’s Performance on ARC-AGI`{=html}](https://arcprize.org/blog/hrm-analysis "The Hidden Drivers of HRM’s Performance on ARC-AGI"){.link-annotated-partial .id-not .include-annotation}** ## `“MesaNet: Sequence Modeling by Locally Optimal Test-Time Training”, Oswald et al 2025`{=html} {#oswald-et-al-2025-section} **[`MesaNet: Sequence Modeling by Locally Optimal Test-Time Training`{=html}](https://arxiv.org/abs/2506.05233#google "'MesaNet: Sequence Modeling by Locally Optimal Test-Time Training', Oswald et al 2025"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“New News: System-2 Fine-Tuning for Robust Integration of New Knowledge”, Park et al 2025`{=html} {#park-et-al-2025-section} **[`New News: System-2 Fine-tuning for Robust Integration of New Knowledge`{=html}](https://arxiv.org/abs/2505.01812 "'New News: System-2 Fine-tuning for Robust Integration of New Knowledge', Park et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“On the Generalization of Language Models from In-Context Learning and Finetuning: a Controlled Study”, Lampinen et al 2025`{=html} {#lampinen-et-al-2025-section} **[`On the generalization of language models from in-context learning and finetuning: a controlled study`{=html}](https://arxiv.org/abs/2505.00661#google "'On the generalization of language models from in-context learning and finetuning: a controlled study', Lampinen et al 2025"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“One-Minute Video Generation With Test-Time Training”, Dalal et al 2025`{=html} {#dalal-et-al-2025-section} **[`One-Minute Video Generation with Test-Time Training`{=html}](https://arxiv.org/abs/2504.05298#nvidia "'One-Minute Video Generation with Test-Time Training', Dalal et al 2025"){.link-annotated .id-not .include-annotation link-icon="n" link-icon-type="text,sans,italic" link-icon-color="#77ba00"}** ## `“RWKV-7 ‘Goose’ With Expressive Dynamic State Evolution”, Peng et al 2025`{=html} {#peng-et-al-2025-section} **[`RWKV-7 ‘Goose’ with Expressive Dynamic State Evolution`{=html}](https://arxiv.org/abs/2503.14456 "'RWKV-7 ‘Goose’ with Expressive Dynamic State Evolution', Peng et al 2025"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“NVARC Solution”`{=html} {#_sqGyQ3t8-section} **[`NVARC solution`{=html}](https://www.kaggle.com/competitions/arc-prize-2025/writeups/nvarc "NVARC solution"){.id-not .include-annotation link-icon="k" link-icon-type="text,sans" link-icon-color="#20beff"}** ## `“AUNN: Simple Implementation of Gwern’s AUNN Proposal”, Roland 2024`{=html} {#roland-2024-section} **[`AUNN: Simple implementation of Gwern’s AUNN proposal`{=html}](https://github.com/ethan-w-roland/AUNN "'AUNN: Simple implementation of Gwern’s AUNN proposal', Roland 2024"){.id-not .include-annotation link-icon="github" link-icon-type="svg"}** ## `“The Surprising Effectiveness of Test-Time Training for Few-Shot Learning”, Akyürek et al 2024`{=html} {#akyürek-et-al-2024-section} **[`The Surprising Effectiveness of Test-Time Training for Few-Shot Learning`{=html}](https://arxiv.org/abs/2411.07279 "'The Surprising Effectiveness of Test-Time Training for Few-Shot Learning', Akyürek et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Emergent Properties With Repeated Examples”, Charton & Kempe 2024`{=html} {#charton-kempe-2024-section} **[`Emergent properties with repeated examples`{=html}](https://arxiv.org/abs/2410.07041#facebook "'Emergent properties with repeated examples', Charton & Kempe 2024"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“Evaluating the Fairness of Task-Adaptive Pretraining on Unlabeled Test Data Before Few-Shot Text Classification”, Dubey 2024`{=html} {#dubey-2024-section} **[`Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification`{=html}](https://arxiv.org/abs/2410.00179 "'Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification', Dubey 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“FineZip: Pushing the Limits of Large Language Models for Practical Lossless Text Compression”, Mittu et al 2024`{=html} {#mittu-et-al-2024-section} **[`FineZip: Pushing the Limits of Large Language Models for Practical Lossless Text Compression`{=html}](https://arxiv.org/abs/2409.17141 "'FineZip: Pushing the Limits of Large Language Models for Practical Lossless Text Compression', Mittu et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Learning to (Learn at Test Time): RNNs With Expressive Hidden States”, Sun et al 2024`{=html} {#sun-et-al-2024-1-section} **[`Learning to (Learn at Test Time): RNNs with Expressive Hidden States`{=html}](https://arxiv.org/abs/2407.04620 "'Learning to (Learn at Test Time): RNNs with Expressive Hidden States', Sun et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Instruction Modeling: Instruction Tuning With Loss Over Instructions”, Shi et al 2024`{=html} {#shi-et-al-2024-2-section} **[`Instruction Modeling: Instruction Tuning With Loss Over Instructions`{=html}](https://arxiv.org/abs/2405.14394 "'Instruction Modeling: Instruction Tuning With Loss Over Instructions', Shi et al 2024"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Test-Time Augmentation to Solve ARC”, Cole 2024`{=html} {#cole-2024-section} **[`Test-Time Augmentation to solve ARC`{=html}](https://lab42.global/community-interview-jack-cole/ "'Test-Time Augmentation to solve ARC', Cole 2024"){.link-annotated .id-not .include-annotation}** ## `“An Accurate and Rapidly Calibrating Speech Neuroprosthesis”, Card et al 2024`{=html} {#card-et-al-2024-section} **[`An accurate and rapidly calibrating speech neuroprosthesis`{=html}](https://www.medrxiv.org/content/10.1101/2023.12.26.23300110.full "'An accurate and rapidly calibrating speech neuroprosthesis', Card et al 2024"){.link-annotated .id-not .include-annotation link-icon="chi-dna" link-icon-type="svg" link-icon-color="#bd2736"}** ## `“Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models”, Rannen-Triki et al 2024`{=html} {#rannen-triki-et-al-2024-section} **[`Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models`{=html}](https://arxiv.org/abs/2403.01518#deepmind "'Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models', Rannen-Triki et al 2024"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Neural Spline Fields for Burst Image Fusion and Layer Separation”, Chugunov et al 2023`{=html} {#chugunov-et-al-2023-section} **[`Neural Spline Fields for Burst Image Fusion and Layer Separation`{=html}](https://arxiv.org/abs/2312.14235 "'Neural Spline Fields for Burst Image Fusion and Layer Separation', Chugunov et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Test-Time Adaptation of Discriminative Models via Diffusion Generative Feedback”, Prabhudesai et al 2023`{=html} {#prabhudesai-et-al-2023-section} **[`Test-time Adaptation of Discriminative Models via Diffusion Generative Feedback`{=html}](https://arxiv.org/abs/2311.16102 "'Test-time Adaptation of Discriminative Models via Diffusion Generative Feedback', Prabhudesai et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“In-Context Pretraining (ICP): Language Modeling Beyond Document Boundaries”, Shi et al 2023`{=html} {#shi-et-al-2023-section} **[`In-Context Pretraining (ICP): Language Modeling Beyond Document Boundaries`{=html}](https://arxiv.org/abs/2310.10638#facebook "'In-Context Pretraining (ICP): Language Modeling Beyond Document Boundaries', Shi et al 2023"){.link-annotated .id-not .include-annotation link-icon="facebook" link-icon-type="svg" link-icon-color="#1877f2"}** ## `“OSD: Online Speculative Decoding”, Liu et al 2023`{=html} {#liu-et-al-2023-07-section} **[`OSD: Online Speculative Decoding`{=html}](https://arxiv.org/abs/2310.07177 "'OSD: Online Speculative Decoding', Liu et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Dynamic Evaluation”, Gwern 2023`{=html} {#gwern-doc-ai-nn-dynamic-evaluation-abstract-section} **[`Dynamic Evaluation`{=html}](/doc/ai/nn/dynamic-evaluation/abstract "'Dynamic Evaluation', Gwern 2023"){.link-annotated-partial .id-not .include-annotation}** ## `“Re2: Re-Reading Improves Reasoning in Large Language Models”, Xu et al 2023`{=html} {#xu-et-al-2023-section} **[`Re2: Re-Reading Improves Reasoning in Large Language Models`{=html}](https://arxiv.org/abs/2309.06275 "'Re2: Re-Reading Improves Reasoning in Large Language Models', Xu et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Test-Time Training on Video Streams”, Wang et al 2023`{=html} {#wang-et-al-2023-13-section} **[`Test-Time Training on Video Streams`{=html}](https://arxiv.org/abs/2307.05014 "'Test-Time Training on Video Streams', Wang et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“TTT-NN: Test-Time Training on Nearest Neighbors for Large Language Models”, Hardt & Sun 2023`{=html} {#hardt-sun-2023-section} **[`TTT-NN: Test-Time Training on Nearest Neighbors for Large Language Models`{=html}](https://arxiv.org/abs/2305.18466 "'TTT-NN: Test-Time Training on Nearest Neighbors for Large Language Models', Hardt & Sun 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“CaMeLS: Meta-Learning Online Adaptation of Language Models”, Hu et al 2023`{=html} {#hu-et-al-2023-section} **[`CaMeLS: Meta-Learning Online Adaptation of Language Models`{=html}](https://arxiv.org/abs/2305.15076 "'CaMeLS: Meta-Learning Online Adaptation of Language Models', Hu et al 2023"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“FWL: Meta-Learning Fast Weight Language Models”, Clark et al 2022`{=html} {#clark-et-al-2022-section} **[`FWL: Meta-Learning Fast Weight Language Models`{=html}](https://arxiv.org/abs/2212.02475#google "'FWL: Meta-Learning Fast Weight Language Models', Clark et al 2022"){.link-annotated .id-not .include-annotation link-icon="alphabet" link-icon-type="svg" link-icon-color="#4285f4"}** ## `“Test-Time Training With Masked Autoencoders”, Gandelsman et al 2022`{=html} {#gandelsman-et-al-2022-section} **[`Test-Time Training with Masked Autoencoders`{=html}](https://arxiv.org/abs/2209.07522 "'Test-Time Training with Masked Autoencoders', Gandelsman et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Large-Scale Retrieval for Reinforcement Learning”, Humphreys et al 2022`{=html} {#humphreys-et-al-2022-1-section} **[`Large-Scale Retrieval for Reinforcement Learning`{=html}](https://arxiv.org/abs/2206.05314#deepmind "'Large-Scale Retrieval for Reinforcement Learning', Humphreys et al 2022"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Don’t Stop the Training: Continuously-Updating Self-Supervised Algorithms Best Account for Auditory Responses in the Cortex”, Orhan et al 2022`{=html} {#orhan-et-al-2022-section} **[`Don’t stop the training: continuously-updating self-supervised algorithms best account for auditory responses in the cortex`{=html}](https://arxiv.org/abs/2202.07290 "'Don’t stop the training: continuously-updating self-supervised algorithms best account for auditory responses in the cortex', Orhan et al 2022"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Reconsidering the Past: Optimizing Hidden States in Language Models”, Yoshida & Gimpel 2021`{=html} {#yoshida-gimpel-2021-section} **[`Reconsidering the Past: Optimizing Hidden States in Language Models`{=html}](https://arxiv.org/abs/2112.08653 "'Reconsidering the Past: Optimizing Hidden States in Language Models', Yoshida & Gimpel 2021"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”, Lazaridou et al 2021`{=html} {#lazaridou-et-al-2021-section} **[`Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling`{=html}](https://arxiv.org/abs/2102.01951#scaling&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling', Lazaridou et al 2021"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation”, Lazaridou et al 2021 (page 7 org deepmind)`{=html} {#lazaridou-et-al-2021-page-7-org-deepmind-section} **[`Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation`{=html}](https://arxiv.org/pdf/2102.01951#page=7&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation', Lazaridou et al 2021 (page 7 org deepmind)"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Deep-Learning the Hardest Go Problem in the World (Igo #120)”, Wu 2019`{=html} {#wu-2019-section} **[`Deep-Learning the Hardest Go Problem in the World (Igo #120)`{=html}](https://blog.janestreet.com/deep-learning-the-hardest-go-problem-in-the-world/ "'Deep-Learning the Hardest Go Problem in the World (Igo #120)', Wu 2019"){.link-annotated .id-not .include-annotation}** ## `“Test-Time Training With Self-Supervision for Generalization under Distribution Shifts”, Sun et al 2019`{=html} {#sun-et-al-2019-1-section} **[`Test-Time Training with Self-Supervision for Generalization under Distribution Shifts`{=html}](https://arxiv.org/abs/1909.13231 "'Test-Time Training with Self-Supervision for Generalization under Distribution Shifts', Sun et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Unsupervised Domain Adaptation through Self-Supervision”, Sun et al 2019`{=html} {#sun-et-al-2019-section} **[`Unsupervised Domain Adaptation through Self-Supervision`{=html}](https://arxiv.org/abs/1909.11825 "'Unsupervised Domain Adaptation through Self-Supervision', Sun et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Mogrifier LSTM”, Melis et al 2019`{=html} {#melis-et-al-2019-section} **[`Mogrifier LSTM`{=html}](https://arxiv.org/abs/1909.01792#deepmind "'Mogrifier LSTM', Melis et al 2019"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Dynamic Evaluation of Transformer Language Models”, Krause et al 2019`{=html} {#krause-et-al-2019-section} **[`Dynamic Evaluation of Transformer Language Models`{=html}](https://arxiv.org/abs/1904.08378 "'Dynamic Evaluation of Transformer Language Models', Krause et al 2019"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Learning and Evaluating General Linguistic Intelligence”, Yogatama et al 2019`{=html} {#yogatama-et-al-2019-section} **[`Learning and Evaluating General Linguistic Intelligence`{=html}](https://arxiv.org/abs/1901.11373#deepmind "'Learning and Evaluating General Linguistic Intelligence', Yogatama et al 2019"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Faster SGD Training by Minibatch Persistency”, Fischetti et al 2018`{=html} {#fischetti-et-al-2018-section} **[`Faster SGD training by minibatch persistency`{=html}](https://arxiv.org/abs/1806.07353 "'Faster SGD training by minibatch persistency', Fischetti et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Continuous Learning in a Hierarchical Multiscale Neural Network”, Wolf et al 2018`{=html} {#wolf-et-al-2018-section} **[`Continuous Learning in a Hierarchical Multiscale Neural Network`{=html}](https://arxiv.org/abs/1805.05758 "'Continuous Learning in a Hierarchical Multiscale Neural Network', Wolf et al 2018"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Dynamic Evaluation of Neural Sequence Models”, Krause et al 2017`{=html} {#krause-et-al-2017-section} **[`Dynamic Evaluation of Neural Sequence Models`{=html}](https://arxiv.org/abs/1709.07432 "'Dynamic Evaluation of Neural Sequence Models', Krause et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Bayesian Recurrent Neural Networks”, Fortunato et al 2017`{=html} {#fortunato-et-al-2017-2-section} **[`Bayesian Recurrent Neural Networks`{=html}](https://arxiv.org/abs/1704.02798#deepmind "'Bayesian Recurrent Neural Networks', Fortunato et al 2017"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Learning Simpler Language Models With the Differential State Framework”, II et al 2017`{=html} {#ii-et-al-2017-section} **[`Learning Simpler Language Models with the Differential State Framework`{=html}](https://arxiv.org/abs/1703.08864 "'Learning Simpler Language Models with the Differential State Framework', II et al 2017"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Neural Episodic Control”, Pritzel et al 2017`{=html} {#pritzel-et-al-2017-section} **[`Neural Episodic Control`{=html}](https://arxiv.org/abs/1703.01988#deepmind "'Neural Episodic Control', Pritzel et al 2017"){.link-annotated .id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}** ## `“Multiplicative LSTM for Sequence Modeling”, Krause et al 2016`{=html} {#krause-et-al-2016-section} **[`Multiplicative LSTM for sequence modeling`{=html}](https://arxiv.org/abs/1609.07959 "'Multiplicative LSTM for sequence modeling', Krause et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“One Sentence One Model for Neural Machine Translation”, Li et al 2016`{=html} {#li-et-al-2016-section} **[`One Sentence One Model for Neural Machine Translation`{=html}](https://arxiv.org/abs/1609.06490 "'One Sentence One Model for Neural Machine Translation', Li et al 2016"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Generating Sequences With Recurrent Neural Networks”, Graves 2013`{=html} {#graves-2013-section} **[`Generating Sequences With Recurrent Neural Networks`{=html}](https://arxiv.org/abs/1308.0850 "'Generating Sequences With Recurrent Neural Networks', Graves 2013"){.link-annotated .id-not .include-annotation link-icon="𝛘" link-icon-type="text" link-icon-color="#b31b1b"}** ## `“Recurrent Neural Network Based Language Model § Dynamic Evaluation”, Mikolov et al 2010 (page 2)`{=html} {#mikolov-et-al-2010-page-2-section} **[`Recurrent Neural Network Based Language Model § Dynamic Evaluation`{=html}](/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 "'Recurrent Neural Network Based Language Model § Dynamic Evaluation', Mikolov et al 2010 (page 2)"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“Fast Text Compression With Neural Networks”, Mahoney 2000`{=html} {#mahoney-2000-section} **[`Fast Text Compression with Neural Networks`{=html}](https://mattmahoney.net/dc/mmahoney00.pdf "'Fast Text Compression with Neural Networks', Mahoney 2000"){.link-annotated .id-not .include-annotation link-icon="pdf" link-icon-type="svg" link-icon-color="#f40f02"}** ## `“OpenAI API § Prompt Caching”`{=html} {#_M_1q-yE9-section} **[`OpenAI API § Prompt Caching`{=html}](https://platform.openai.com/docs/guides/prompt-caching "OpenAI API § Prompt Caching"){.id-not .include-annotation link-icon="openai" link-icon-type="svg"}** ## `“On-Policy Distillation”`{=html} {#_Jdz9b3YH-section} **[`On-Policy Distillation`{=html}](https://thinkingmachines.ai/blog/on-policy-distillation/#distillation-for-personalization "On-Policy Distillation"){.link-annotated-partial .id-not .include-annotation}** ## `“RWKV Language Model”`{=html} {#_DHKLAErI-section} **[`RWKV Language Model`{=html}](https://www.rwkv.com/ "RWKV Language Model"){.link-annotated-partial .id-not .include-annotation}** ## `“Yu Sun”`{=html} {#_q6rNmvRh-section} **[`Yu Sun`{=html}](https://yueatsprograms.github.io/ "Yu Sun"){.link-annotated-partial .id-not .include-annotation}** ## Sort By Magic ::: {demo-type="sort-by-magic-preface"} Annotations sorted by machine learning into [inferred \'tags\'](/design#future-tag-features){#gwern-design--future-tag-features .link-annotated-partial}. This provides an alternative way to browse: instead of by *date* order, one can browse in *topic* order. The \'sorted\' list has been automatically clustered into multiple sections & auto-labeled for easier browsing. Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link. ::: ### `task-fairness` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#chugunov-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dubey-2024-section){.include .include-even-when-collapsed} ### `speech-technology` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#card-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#orhan-et-al-2022-section){.include .include-even-when-collapsed} ### `linguistic-intelligence` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#yogatama-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#li-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fortunato-et-al-2017-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#graves-2013-section){.include .include-even-when-collapsed} ### `test-time-augmentation` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#jiang-et-al-2026-section){.include .include-even-when-collapsed} [\[see previous entry\]](#cole-2024-section){.include .include-even-when-collapsed} ### `online-adaptation` {.collapse title="Machine-generated tag name for the following cluster of links."} [\[see previous entry\]](#mahoney-2000-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mittu-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lazaridou-et-al-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lazaridou-et-al-2021-page-7-org-deepmind-section){.include .include-even-when-collapsed} [\[see previous entry\]](#rannen-triki-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#krause-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#krause-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#mikolov-et-al-2010-page-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#melis-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#krause-et-al-2016-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2024-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#oswald-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#dalal-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wang-et-al-2023-13-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gandelsman-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2019-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sun-et-al-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#prabhudesai-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#akyürek-et-al-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hardt-sun-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#lampinen-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#park-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shi-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#hu-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wolf-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#ii-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#peng-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#clark-et-al-2022-section){.include .include-even-when-collapsed} [\[see previous entry\]](#yoshida-gimpel-2021-section){.include .include-even-when-collapsed} [\[see previous entry\]](#liu-et-al-2023-07-section){.include .include-even-when-collapsed} [\[see previous entry\]](#chen-et-al-2025-section){.include .include-even-when-collapsed} [\[see previous entry\]](#shi-et-al-2024-2-section){.include .include-even-when-collapsed} [\[see previous entry\]](#xu-et-al-2023-section){.include .include-even-when-collapsed} [\[see previous entry\]](#charton-kempe-2024-section){.include .include-even-when-collapsed} [\[see previous entry\]](#fischetti-et-al-2018-section){.include .include-even-when-collapsed} [\[see previous entry\]](#gwern-doc-ai-nn-dynamic-evaluation-abstract-section){.include .include-even-when-collapsed} [\[see previous entry\]](#pritzel-et-al-2017-section){.include .include-even-when-collapsed} [\[see previous entry\]](#humphreys-et-al-2022-1-section){.include .include-even-when-collapsed} [\[see previous entry\]](#wu-2019-section){.include .include-even-when-collapsed} [\[see previous entry\]](#sokota-et-al-2025-section){.include .include-even-when-collapsed} ## Wikipedia (2) {#titled-links-wikipedia .collapse} 1. **[`Backpropagation through time`{=html}](https://en.wikipedia.org/wiki/Backpropagation_through_time "Backpropagation through time"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** 2. **[`Online machine learning`{=html}](https://en.wikipedia.org/wiki/Online_machine_learning "Online machine learning"){.link-annotated-partial .id-not .include-annotation link-icon="wikipedia" link-icon-type="svg"}** # Miscellaneous - **[`/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure1-dynamicevaluationbeatsstaticllmsonlanguagemodelingofconcatenatedprojectgutenbergbooks.png`{=html}](/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure1-dynamicevaluationbeatsstaticllmsonlanguagemodelingofconcatenatedprojectgutenbergbooks.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure3-loglossvscomputeforstaticvsdynamicmodel.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure3-loglossvscomputeforstaticvsdynamicmodel.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure4-smallmodelscanbeatlargermodelswithdynamicevaluation.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure4-smallmodelscanbeatlargermodelswithdynamicevaluation.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure5-dynamicevaluationstillhelpsevenwithfinetuning.png`{=html}](/doc/ai/nn/dynamic-evaluation/2024-rannentriki-figure5-dynamicevaluationstillhelpsevenwithfinetuning.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-12-31-gwern-mentalgymnasticsmeme-dynamicevaluationvsalternativeapproaches.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-12-31-gwern-mentalgymnasticsmeme-dynamicevaluationvsalternativeapproaches.jpg "Gwern 2023"){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-hardt-figure5-bitsperbytegpt2performanceimprovementwhentrainingon50nearestneighborexamples.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-hardt-figure5-bitsperbytegpt2performanceimprovementwhentrainingon50nearestneighborexamples.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-hardt-figure6-perplexitiesdecreasewhentrainingonincreasinglymoreneighborsusinggpt2onthepile.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-hardt-figure6-perplexitiesdecreasewhentrainingonincreasinglymoreneighborsusinggpt2onthepile.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-hardt-figure7-bitesperbyteforgpt2large.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-hardt-figure7-bitesperbyteforgpt2large.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-hardt-figure8-bitesperbyteforgptneo.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-hardt-figure8-bitesperbyteforgptneo.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2023-hardt-figure9-trainingcostofdynamicevaluationonnearestneighborlookups.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2023-hardt-figure9-trainingcostofdynamicevaluationonnearestneighborlookups.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2022-clark-figure2-fwldynamicevaluationimprovesmostonrareorrepeatedtokens.jpg`{=html}](/doc/ai/nn/dynamic-evaluation/2022-clark-figure2-fwldynamicevaluationimprovesmostonrareorrepeatedtokens.jpg){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`/doc/ai/nn/dynamic-evaluation/2017-krause-figure2-dynamicevaluationrnnpredictionofwikipediaandspanishtextshowingtesttimeadaptation.png`{=html}](/doc/ai/nn/dynamic-evaluation/2017-krause-figure2-dynamicevaluationrnnpredictionofwikipediaandspanishtextshowingtesttimeadaptation.png){.link-annotated-partial .id-not .include-annotation link-icon="image" link-icon-type="svg"}** - **[`https://arxiv.org/pdf/2102.01951.pdf#page=18&=deepmind`{=html}](https://arxiv.org/pdf/2102.01951.pdf#page=18&=deepmind){.id-not .include-annotation link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}[  :]{.mobile-not}** ::: aux-links-transclude-file ::: {.collapse .mobile-not} **View PDF**: [`/doc/www/arxiv.org/2b42979ae6f5f70c53a29c1350667ad5d50118d4.pdf#page=18&=deepmind`](/doc/www/arxiv.org/2b42979ae6f5f70c53a29c1350667ad5d50118d4.pdf#page=18&=deepmind){.id-not .link-annotated-not .include-content .include-lazy link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"} ::: ::: - **[`https://benkrause.github.io/blog/human-level-text-prediction/`{=html}](https://benkrause.github.io/blog/human-level-text-prediction/){.link-annotated-partial .id-not .include-annotation}** - **[`https://www.latent.space/p/fastai#%C2%A7replacing-fine-tuning-with-continued-pre-training`{=html}](https://www.latent.space/p/fastai#%C2%A7replacing-fine-tuning-with-continued-pre-training){.id-not .include-annotation}** ::: aux-links-transclude-file ::: collapse **View HTML**: [`/doc/www/www.latent.space/1b54d898ad24b02d0faeab34ab6e834adec475d7.html#%C2%A7replacing-fine-tuning-with-continued-pre-training`](/doc/www/www.latent.space/1b54d898ad24b02d0faeab34ab6e834adec475d7.html#%C2%A7replacing-fine-tuning-with-continued-pre-training){.id-not .link-annotated-not .include-content .include-lazy link-icon="internet-archive" link-icon-type="svg"} ::: ::: - **[`https://x.com/AndrewLampinen/status/1918347839232721123`{=html}](https://x.com/AndrewLampinen/status/1918347839232721123 "AndrewLampinen 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** - **[`https://x.com/corefpark/status/1919811638435201107`{=html}](https://x.com/corefpark/status/1919811638435201107 "corefpark 2026"){.link-annotated-partial .id-not .include-annotation link-icon="twitter" link-icon-type="svg" link-icon-color="#1da1f2"}** # Bibliography {#link-bibliography-section} 1. `https://arxiv.org/abs/2510.15020`: [`“The Coverage Principle: How Pre-Training Enables Post-Training”`{=html}](https://arxiv.org/abs/2510.15020 "'The Coverage Principle: How Pre-Training Enables Post-Training', Chen et al 2025"){#chen-et-al-2025 .link-annotated}, [Fan Chen, Audrey Huang, Noah Golowich[, Sadhika Malladi, Adam Block, Jordan T. Ash, Akshay Krishnamurthy, Dylan J. Foster]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2510.15020.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2510.15020"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 2. `https://arxiv.org/abs/2505.00661#google`: [`“On the Generalization of Language Models from In-Context Learning and Finetuning: a Controlled Study”`{=html}](https://arxiv.org/abs/2505.00661#google "'On the generalization of language models from in-context learning and finetuning: a controlled study', Lampinen et al 2025"){#lampinen-et-al-2025 .link-annotated}, [Andrew K. Lampinen, Arslan Chaudhry, Stephanie C. Y. Chan[, [Cody Wild](https://en.wikipedia.org/wiki/Cody_Wild "Cody Wild"){.id-not}, Diane Wan, Alex Ku, Jörg Bornschein, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}, [Murray Shanahan](https://en.wikipedia.org/wiki/Murray_Shanahan "Murray Shanahan"){.id-not}, [James L. McClelland](https://en.wikipedia.org/wiki/James_L._McClelland "James L. McClelland"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2505.00661%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2505.00661#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 3. `https://arxiv.org/abs/2504.05298#nvidia`: [`“One-Minute Video Generation With Test-Time Training”`{=html}](https://arxiv.org/abs/2504.05298#nvidia "'One-Minute Video Generation with Test-Time Training', Dalal et al 2025"){#dalal-et-al-2025 .link-annotated}, [Karan Dalal, Daniel Koceja, Gashon Hussein[, Jiarui Xu, Yue Zhao, Youjin Song, Shihao Han, Ka Chun Cheung, Jan Kautz, Carlos Guestrin, [Tatsunori Hashimoto](https://thashim.github.io/ "Tatsunori Hashimoto"){.link-annotated-partial .id-not}, Sanmi Koyejo, [Yejin Choi](https://en.wikipedia.org/wiki/Yejin_Choi "Yejin Choi"){.id-not}, [Yu Sun](https://yueatsprograms.github.io/ "Yu Sun"){.link-annotated-partial .id-not}, Xiaolong Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2504.05298%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2504.05298#nvidia"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 4. `https://arxiv.org/abs/2503.14456`: [`“RWKV-7 ‘Goose’ With Expressive Dynamic State Evolution”`{=html}](https://arxiv.org/abs/2503.14456 "'RWKV-7 ‘Goose’ with Expressive Dynamic State Evolution', Peng et al 2025"){#peng-et-al-2025 .link-annotated}, [[Bo Peng](https://en.wikipedia.org/wiki/Bo_Peng "Bo Peng"){.id-not}, Ruichong Zhang, Daniel Goldstein[, Eric Alcaide, Xingjian Du, Haowen Hou, Jiaju Lin, Jiaxing Liu, Janna Lu, William Merrill, Guangyu Song, Kaifeng Tan, Saiteja Utpala, Nathan Wilce, Johan S. Wind, Tianyi Wu, Daniel Wuttke, Christian Zhou-Zheng]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2503.14456.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2503.14456"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 5. `https://arxiv.org/abs/2410.00179`: [`“Evaluating the Fairness of Task-Adaptive Pretraining on Unlabeled Test Data Before Few-Shot Text Classification”`{=html}](https://arxiv.org/abs/2410.00179 "'Evaluating the fairness of task-adaptive pretraining on unlabeled test data before few-shot text classification', Dubey 2024"){#dubey-2024 .link-annotated}, [Kush Dubey]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2410.00179.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2410.00179"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 6. `https://arxiv.org/abs/2409.17141`: [`“FineZip: Pushing the Limits of Large Language Models for Practical Lossless Text Compression”`{=html}](https://arxiv.org/abs/2409.17141 "'FineZip: Pushing the Limits of Large Language Models for Practical Lossless Text Compression', Mittu et al 2024"){#mittu-et-al-2024 .link-annotated}, [Fazal Mittu, Yihuan Bu, Akshat Gupta[, Ashok Devireddy, Alp Eren Ozdarendeli, Anant Singh, Gopala Anumanchipalli]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2409.17141.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2409.17141"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 7. `https://lab42.global/community-interview-jack-cole/`: [`“Test-Time Augmentation to Solve ARC”`{=html}](https://lab42.global/community-interview-jack-cole/ "'Test-Time Augmentation to solve ARC', Cole 2024"){#cole-2024 .link-annotated}, [Jack Cole]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Flab42.global%252Fcommunity-interview-jack-cole%252F.html "Directory-tag link-bibliography for link https://lab42.global/community-interview-jack-cole/"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 8. `https://arxiv.org/abs/2403.01518#deepmind`: [`“Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models”`{=html}](https://arxiv.org/abs/2403.01518#deepmind "'Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models', Rannen-Triki et al 2024"){#rannen-triki-et-al-2024 .link-annotated}, [Amal Rannen-Triki, Jorg Bornschein, [Razvan Pascanu](https://sites.google.com/view/razp/home){.link-annotated-partial .id-not}[, [Marcus Hutter](https://en.wikipedia.org/wiki/Marcus_Hutter "Marcus Hutter"){.id-not}, Andras György, Alexandre Galashov, [Yee Whye Teh](https://en.wikipedia.org/wiki/Yee_Whye_Teh "Yee Whye Teh"){.id-not}, Michalis K. Titsias]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2403.01518%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2403.01518#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 9. `abstract`: [`“Dynamic Evaluation”`{=html}](/doc/ai/nn/dynamic-evaluation/abstract "'Dynamic Evaluation', Gwern 2023"){#gwern-doc-ai-nn-dynamic-evaluation-abstract-2 .link-annotated-partial}, [[Gwern](/index#abstract "'Gwern Bibliography', Gwern 2010"){.link-annotated-partial .id-not}]{.author .cite-author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fnn%252Fdynamic-evaluation%252Fabstract.html "Directory-tag link-bibliography for link /doc/ai/nn/dynamic-evaluation/abstract"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 10. `https://arxiv.org/abs/2309.06275`: [`“Re2: Re-Reading Improves Reasoning in Large Language Models”`{=html}](https://arxiv.org/abs/2309.06275 "'Re2: Re-Reading Improves Reasoning in Large Language Models', Xu et al 2023"){#xu-et-al-2023 .link-annotated}, [Xiaohan Xu, Chongyang Tao, Tao Shen[, [Can Xu](https://en.wikipedia.org/wiki/Can_Xu "Can Xu"){.id-not}, Hongbo Xu, Guodong Long, Jian-guang Lou, Shuai Ma]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2309.06275.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2309.06275"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 11. `https://arxiv.org/abs/2307.05014`: [`“Test-Time Training on Video Streams”`{=html}](https://arxiv.org/abs/2307.05014 "'Test-Time Training on Video Streams', Wang et al 2023"){#wang-et-al-2023-13 .link-annotated}, [Renhao Wang, [Yu Sun](https://yueatsprograms.github.io/ "Yu Sun"){.link-annotated-partial .id-not}, Yossi Gandelsman[, [Xinlei Chen](https://xinleic.xyz/ "Xinlei Chen"){.link-annotated-partial .id-not}, [Alexei A. Efros](https://en.wikipedia.org/wiki/Alexei_A._Efros "Alexei A. Efros"){.id-not}, Xiaolong Wang]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2307.05014.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2307.05014"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 12. `https://arxiv.org/abs/2305.18466`: [`“TTT-NN: Test-Time Training on Nearest Neighbors for Large Language Models”`{=html}](https://arxiv.org/abs/2305.18466 "'TTT-NN: Test-Time Training on Nearest Neighbors for Large Language Models', Hardt & Sun 2023"){#hardt-sun-2023 .link-annotated}, [Moritz Hardt, [Yu Sun](https://yueatsprograms.github.io/ "Yu Sun"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2305.18466.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2305.18466"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 13. `https://arxiv.org/abs/2212.02475#google`: [`“FWL: Meta-Learning Fast Weight Language Models”`{=html}](https://arxiv.org/abs/2212.02475#google "'FWL: Meta-Learning Fast Weight Language Models', Clark et al 2022"){#clark-et-al-2022 .link-annotated}, [Kevin Clark, Kelvin Guu, Ming-Wei Chang[, Panupong Pasupat, [Geoffrey Hinton](https://en.wikipedia.org/wiki/Geoffrey_Hinton "Geoffrey Hinton"){.link-annotated-partial .id-not}, [Mohammad Norouzi](https://en.wikipedia.org/wiki/Mohammad_Norouzi "Mohammad Norouzi"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2212.02475%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2212.02475#google"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 14. `https://arxiv.org/abs/2206.05314#deepmind`: [`“Large-Scale Retrieval for Reinforcement Learning”`{=html}](https://arxiv.org/abs/2206.05314#deepmind "'Large-Scale Retrieval for Reinforcement Learning', Humphreys et al 2022"){#humphreys-et-al-2022-1 .link-annotated}, [Peter C. Humphreys, [Arthur Guez](https://scholar.google.com/citations?user=iyD9aw8AAAAJ "Arthur Guez"){.id-not}, Olivier Tieleman[, [Laurent Sifre](https://scholar.google.com/citations?user=0kVh58wAAAAJ "Laurent Sifre"){.id-not}, Théophane Weber, [Timothy Lillicrap](https://en.wikipedia.org/wiki/Timothy_Lillicrap "Timothy Lillicrap"){.id-not}]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.05314%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.05314#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 15. `https://arxiv.org/abs/2112.08653`: [`“Reconsidering the Past: Optimizing Hidden States in Language Models”`{=html}](https://arxiv.org/abs/2112.08653 "'Reconsidering the Past: Optimizing Hidden States in Language Models', Yoshida & Gimpel 2021"){#yoshida-gimpel-2021 .link-annotated}, [Davis Yoshida, [Kevin Gimpel](https://home.ttic.edu/~kgimpel/ "Kevin Gimpel"){.link-annotated-partial .id-not}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2112.08653.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2112.08653"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 16. `https://arxiv.org/abs/2102.01951#scaling&org=deepmind`: [`“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling”`{=html}](https://arxiv.org/abs/2102.01951#scaling&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Scaling', Lazaridou et al 2021"){#lazaridou-et-al-2021 .link-annotated}, [Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya[, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, Kris Cao, Susannah Young, Phil Blunsom]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2102.01951%2523scaling%2526org%253Ddeepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2102.01951#scaling&org=deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 17. `https://arxiv.org/pdf/2102.01951#page=7&org=deepmind`: [`“Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation”`{=html}](https://arxiv.org/pdf/2102.01951#page=7&org=deepmind "'Mind the Gap: Assessing Temporal Generalization in Neural Language Models § Dynamic Evaluation', Lazaridou et al 2021 (page 7 org deepmind)"){#lazaridou-et-al-2021-page-7-org-deepmind .link-annotated}, [Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya[, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d'Autume, Tomas Kocisky, Sebastian Ruder, [Dani Yogatama](https://dyogatama.github.io/ "Dani Yogatama"){.link-annotated-partial .id-not}, Kris Cao, Susannah Young, Phil Blunsom]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fpdf%252F2102.01951%2523page%253D7%2526org%253Ddeepmind.html "Directory-tag link-bibliography for link https://arxiv.org/pdf/2102.01951#page=7&org=deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 18. `https://arxiv.org/abs/1909.01792#deepmind`: [`“Mogrifier LSTM”`{=html}](https://arxiv.org/abs/1909.01792#deepmind "'Mogrifier LSTM', Melis et al 2019"){#melis-et-al-2019 .link-annotated}, [Gábor Melis, Tomáš Kočiský, Phil Blunsom]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1909.01792%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1909.01792#deepmind"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 19. `https://arxiv.org/abs/1904.08378`: [`“Dynamic Evaluation of Transformer Language Models”`{=html}](https://arxiv.org/abs/1904.08378 "'Dynamic Evaluation of Transformer Language Models', Krause et al 2019"){#krause-et-al-2019 .link-annotated}, [Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1904.08378.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1904.08378"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 20. `https://arxiv.org/abs/1709.07432`: [`“Dynamic Evaluation of Neural Sequence Models”`{=html}](https://arxiv.org/abs/1709.07432 "'Dynamic Evaluation of Neural Sequence Models', Krause et al 2017"){#krause-et-al-2017 .link-annotated}, [Ben Krause, Emmanuel Kahembwe, Iain Murray, Steve Renals]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1709.07432.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1709.07432"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block} 21. `2010-mikolov.pdf#page=2`: [`“Recurrent Neural Network Based Language Model § Dynamic Evaluation”`{=html}](/doc/ai/nn/rnn/2010-mikolov.pdf#page=2 "'Recurrent Neural Network Based Language Model § Dynamic Evaluation', Mikolov et al 2010 (page 2)"){#mikolov-et-al-2010-page-2 .link-annotated}, [[Tomas Mikolov](https://en.wikipedia.org/wiki/Tomas_Mikolov "Tomas Mikolov"){.id-not}, [Martin Karafiat](https://en.wikipedia.org/wiki/Martin_Karafiat "Martin Karafiat"){.id-not}, Lukas Burget[, Jan Cernocky, Sanjeev Khudanpur]{.collapse}]{.author} [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Fai%252Fnn%252Frnn%252F2010-mikolov.pdf%2523page%253D2.html "Directory-tag link-bibliography for link /doc/ai/nn/rnn/2010-mikolov.pdf#page=2"){.id-not .include-even-when-collapsed}]{.collapse .tag-index-link-bibliography-block}