---
title: '‘model-based RL’ directory'
description: "Bibliography for directory <code>reinforcement-learning/model</code>, most recent first: 7 <a class='icon-not' href='/doc/reinforcement-learning/model/index#see-alsos'>related tags</a>, 268 <a class='icon-not' href='/doc/reinforcement-learning/model/index#links'>annotations</a>, & 33 <a class='icon-not' href='/doc/reinforcement-learning/model/index#miscellaneous'>links</a> (<a href='/doc/reinforcement-learning/index' class='link-page link-tag directory-indexes-upwards link-annotated' data-link-icon='arrow-up-left' data-link-icon-type='svg' rel='tag' title='Link to parent directory'>parent</a>)."
thumbnail: /doc/reinforcement-learning/meta-learning/2025-elkishky-figure6-o3learnedselftestingstrategy.jpg
thumbnail-text: 'Figure 6: o3 testing its own solution. This reflects a sophisticated reasoning strategy that partially implements the hand-designed test-time strategy used for o1-ioi in IOI 2024.'
thumbnail-css: "outline"
created: 2019-08-31
modified: 2026-09-12
status: in progress
previous: /doc/reinforcement-learning/model-free/oa5/index
next: /doc/reinforcement-learning/model/alphago/index
confidence: log
importance: 0
css-extension: dropcaps-not
index: True
backlink: False
...

# See Also

::: {#see-alsos .directory-indexes .columns}
-   [Parent ('*`RL`{=html}*' tag)](/doc/reinforcement-learning/index "Link to parent directory 'doc/reinforcement-learning/' (ascending)"){#_wui3VZh5
    .link-annotated .link-tag .directory-indexes-upwards rel="tag"}

-   [*`AlphaGo`{=html}*](/doc/reinforcement-learning/model/alphago/index "‘AlphaGo’ directory"){#_vbPIBTE6
    .link-annotated .link-tag .directory-indexes-downwards rel="tag"}

-   [*`Decision Transformer`{=html}*](/doc/reinforcement-learning/model/decision-transformer/index "‘Decision Transformer’ directory"){#_e4c3cmDO
    .link-annotated .link-tag .directory-indexes-downwards rel="tag"}

-   [*`MuZero`{=html}*](/doc/reinforcement-learning/model/muzero/index "‘MuZero’ directory"){#_AkwcamB6
    .link-annotated .link-tag .directory-indexes-downwards rel="tag"}

-   [*`autoencoder NN`{=html}*](/doc/ai/nn/vae/index "‘autoencoder NN’ directory"){#_3e9iLz8B
    .link-annotated .link-tag .directory-indexes-sideways rel="tag"}

-   [*`video generation`{=html}*](/doc/ai/video/generation/index "‘video generation’ directory"){#_FVaGQXtH
    .link-annotated .link-tag .directory-indexes-sideways rel="tag"}

-   [*`AI chess`{=html}*](/doc/reinforcement-learning/chess/index "‘AI chess’ directory"){#_BZwl5DsP
    .link-annotated .link-tag .directory-indexes-sideways rel="tag"}

-   [*`preference learning`{=html}*](/doc/reinforcement-learning/preference-learning/index "‘preference learning’ directory"){#_3OnUDklb
    .link-annotated .link-tag .directory-indexes-sideways rel="tag"}
:::

# Gwern {.display-pop-not}

## `“‘Try, Score, Change’: Reinforcement Learning for Children”, Pro et al 2026`{=html} {#pro-et-al-2026-section}

**[`‘Try, Score, Change’: Reinforcement Learning for Children`{=html}](/rl-children "'‘Try, Score, Change’: Reinforcement Learning for Children', Pro et al 2026"){.link-annotated
.id-not .include-annotation}**

## `“The Kelly Coin-Flipping Game: Exact Solutions”, Gwern et al 2017`{=html} {#gwern-et-al-2017-section}

**[`The Kelly Coin-Flipping Game: Exact Solutions`{=html}](/coin-flip "'The Kelly Coin-Flipping Game: Exact Solutions', Gwern et al 2017"){.link-annotated
.id-not .include-annotation}**

## `“Resorting Media Ratings”, Gwern 2015`{=html} {#gwern-resorter-section}

**[`Resorting Media Ratings`{=html}](/resorter "'Resorting Media Ratings', Gwern 2015"){.link-annotated
.id-not .include-annotation}**

# Links {.display-pop-not}

## `“Advancing Mathematics Research With AI-Driven Formal Proof Search”, Tsoukalas et al 2026`{=html} {#tsoukalas-et-al-2026-section}

**[`Advancing Mathematics Research with AI-Driven Formal Proof Search`{=html}](https://arxiv.org/abs/2605.22763#deepmind "'Advancing Mathematics Research with AI-Driven Formal Proof Search', Tsoukalas et al 2026"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Imperfect World Models Are Exploitable”, Bhamidipaty et al 2026`{=html} {#bhamidipaty-et-al-2026-section}

**[`Imperfect World Models are Exploitable`{=html}](https://arxiv.org/abs/2605.15960 "'Imperfect World Models are Exploitable', Bhamidipaty et al 2026"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Biggest <em>Trackmania</em> Pathfinding Competition”, Wirtual 2026`{=html} {#wirtual-2026-section}

**[`The Biggest <em>Trackmania</em> Pathfinding Competition`{=html}](https://www.youtube.com/watch?v=3f5Wu4UY6iQ "'The Biggest <em>Trackmania</em> Pathfinding Competition', Wirtual 2026"){.link-annotated-partial
.id-not .include-annotation link-icon="youtube" link-icon-type="svg"
link-icon-color="#ff0033"}**

## `“Optimal <em>Caverna</em> Gameplay via Formal Methods”, Diehl 2026`{=html} {#diehl-2026-section}

**[`Optimal <em>Caverna</em> Gameplay via Formal Methods`{=html}](https://www.stephendiehl.com/posts/caverna/ "'Optimal <em>Caverna</em> Gameplay via Formal Methods', Diehl 2026"){.link-annotated-partial
.id-not .include-annotation}**

## `“I Spent the Last Month and a Half Building a Model That Visualizes Strategic Golf: A Way to Actually See the Ideas Hidden in Golf Course Architecture”, Schoolfield 2026`{=html} {#schoolfield-2026-section}

**[`I Spent the Last Month and a Half Building a Model that Visualizes Strategic Golf: A way to actually see the ideas hidden in golf course architecture`{=html}](https://golfcoursewiki.substack.com/p/i-spent-the-last-month-and-a-half "'I Spent the Last Month and a Half Building a Model that Visualizes Strategic Golf: A way to actually see the ideas hidden in golf course architecture', Schoolfield 2026"){.link-annotated-partial
.id-not .include-annotation link-icon="substack" link-icon-type="svg"
link-icon-color="#ff6719"}**

## `“The Waymo World Model: A New Frontier For Autonomous Driving Simulation”, Waymo 2026`{=html} {#waymo-2026-section}

**[`The Waymo World Model: A New Frontier For Autonomous Driving Simulation`{=html}](https://waymo.com/blog/2026/02/the-waymo-world-model-a-new-frontier-for-autonomous-driving-simulation "'The Waymo World Model: A New Frontier For Autonomous Driving Simulation', Waymo 2026"){.link-annotated-partial
.id-not .include-annotation}**

## `“The Anticipation of Imminent Events Is Time-Scale Invariant”, Grabenhorst et al 2026`{=html} {#grabenhorst-et-al-2026-section}

**[`The anticipation of imminent events is time-scale invariant`{=html}](https://www.pnas.org/doi/abs/10.1073/pnas.2518982123 "'The anticipation of imminent events is time-scale invariant', Grabenhorst et al 2026"){.id-not
.include-annotation link-icon="PNAS" link-icon-type="text,quad"
link-icon-color="#1f75b9"}**

## `“How Gemini 3 Pro Beat <em>Pokemon Crystal</em> (And 2.5 Pro Didn’t)”, Zhang 2025`{=html} {#zhang-2025-section}

**[`How Gemini 3 Pro Beat <em>Pokemon Crystal</em> (and 2.5 Pro didn’t)`{=html}](https://blog.jcz.dev/gemini-3-pro-vs-25-pro-in-pokemon-crystal "'How Gemini 3 Pro Beat <em>Pokemon Crystal</em> (and 2.5 Pro didn’t)', Zhang 2025"){.link-annotated-partial
.id-not .include-annotation}**

## `“TreeGRPO: Tree-Advantage GRPO for Online RL Post-Training of Diffusion Models”, Ding & Ye 2025`{=html} {#ding-ye-2025-section}

**[`TreeGRPO: Tree-Advantage GRPO for Online RL Post-Training of Diffusion Models`{=html}](https://arxiv.org/abs/2512.08153 "'TreeGRPO: Tree-Advantage GRPO for Online RL Post-Training of Diffusion Models', Ding & Ye 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“SIMA 2: A Generalist Embodied Agent for Virtual Worlds”, team et al 2025`{=html} {#team-et-al-2025-section}

**[`SIMA 2: A Generalist Embodied Agent for Virtual Worlds`{=html}](https://arxiv.org/abs/2512.04797#deepmind "'SIMA 2: A Generalist Embodied Agent for Virtual Worlds', team et al 2025"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Effective Harnesses for Long-Running Agents: Agents Still Face Challenges Working across Many Context Windows. We Looked to Human Engineers for Inspiration in Creating a More Effective Harness for Long-Running Agents”, Young 2025`{=html} {#young-2025-section}

**[`Effective harnesses for long-running agents: Agents still face challenges working across many context windows. We looked to human engineers for inspiration in creating a more effective harness for long-running agents`{=html}](https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents "'Effective harnesses for long-running agents: Agents still face challenges working across many context windows. We looked to human engineers for inspiration in creating a more effective harness for long-running agents', Young 2025"){.link-annotated-partial
.id-not .include-annotation link-icon="anthropic" link-icon-type="svg"
link-icon-color="#d4a27f"}**

## `“Reasoning With Sampling: Your Base Model Is Smarter Than You Think”, Karan & Du 2025`{=html} {#karan-du-2025-section}

**[`Reasoning with Sampling: Your Base Model is Smarter Than You Think`{=html}](https://arxiv.org/abs/2510.14901 "'Reasoning with Sampling: Your Base Model is Smarter Than You Think', Karan & Du 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Spooky Collusion at a Distance With Superrational AI”, bira 2025`{=html} {#bira-2025-section}

**[`Spooky Collusion at a Distance with Superrational AI`{=html}](https://www.lesswrong.com/posts/JEtAWvp2sAe8nqpfy/spooky-collusion-at-a-distance-with-superrational-ai "'Spooky Collusion at a Distance with Superrational AI', bira 2025"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Tree-GRPO: Tree Search for LLM Agent Reinforcement Learning”, Ji et al 2025`{=html} {#ji-et-al-2025-section}

**[`Tree-GRPO: Tree Search for LLM Agent Reinforcement Learning`{=html}](https://arxiv.org/abs/2509.21240#alibaba "'Tree-GRPO: Tree Search for LLM Agent Reinforcement Learning', Ji et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Graph Theory in State-Space [For Klotski Sliding-Block Puzzle]”, 2swap 2025`{=html} {#2swap-2025-section}

**[`Graph Theory in State-Space [for Klotski sliding-block puzzle]`{=html}](https://www.youtube.com/watch?v=YGLNyHd2w10 "'Graph Theory in State-Space [for Klotski sliding-block puzzle]', 2swap 2025"){.link-annotated-partial
.id-not .include-annotation link-icon="youtube" link-icon-type="svg"
link-icon-color="#ff0033"}**

## `“Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation”, Rozet et al 2025`{=html} {#rozet-et-al-2025-section}

**[`Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation`{=html}](https://arxiv.org/abs/2507.02608 "'Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation', Rozet et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Strategic Intelligence in Large Language Models: Evidence from Evolutionary Game Theory”, Payne & Alloui-Cros 2025`{=html} {#payne-alloui-cros-2025-section}

**[`Strategic Intelligence in Large Language Models: Evidence from evolutionary Game Theory`{=html}](https://arxiv.org/abs/2507.02618 "'Strategic Intelligence in Large Language Models: Evidence from evolutionary Game Theory', Payne & Alloui-Cros 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning”, Assran et al 2025`{=html} {#assran-et-al-2025-section}

**[`V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning`{=html}](https://arxiv.org/abs/2506.09985#facebook "'V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning', Assran et al 2025"){.link-annotated
.id-not .include-annotation link-icon="facebook" link-icon-type="svg"
link-icon-color="#1877f2"}**

## `“Visual Planning: Let’s Think Only With Images”, Xu et al 2025`{=html} {#xu-et-al-2025-section}

**[`Visual Planning: Let’s Think Only with Images`{=html}](https://arxiv.org/abs/2505.11409 "'Visual Planning: Let’s Think Only with Images', Xu et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Reinforcement Learning Finetunes Small Subnetworks in Large Language Models”, Mukherjee et al 2025`{=html} {#mukherjee-et-al-2025-section}

**[`Reinforcement Learning Finetunes Small Subnetworks in Large Language Models`{=html}](https://arxiv.org/abs/2505.11711 "'Reinforcement Learning Finetunes Small Subnetworks in Large Language Models', Mukherjee et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“<em>XX</em><sup>t</sup> Can Be Faster”, Rybin et al 2025`{=html} {#rybin-et-al-2025-section}

**[`<em>XX</em><sup>t</sup> Can Be Faster`{=html}](https://arxiv.org/abs/2505.09814 "'<em>XX</em><sup>t</sup> Can Be Faster', Rybin et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“LLMs Get Lost In Multi-Turn Conversation”, Laban et al 2025`{=html} {#laban-et-al-2025-section}

**[`LLMs Get Lost In Multi-Turn Conversation`{=html}](https://arxiv.org/abs/2505.06120 "'LLMs Get Lost In Multi-Turn Conversation', Laban et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Reinforcement Learning for Reasoning in Large Language Models With One Training Example”, Wang et al 2025`{=html} {#wang-et-al-2025-section}

**[`Reinforcement Learning for Reasoning in Large Language Models with One Training Example`{=html}](https://arxiv.org/abs/2504.20571 "'Reinforcement Learning for Reasoning in Large Language Models with One Training Example', Wang et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Tina: Tiny Reasoning Models via LoRA”, Wang et al 2025`{=html} {#wang-et-al-2025-section}

**[`Tina: Tiny Reasoning Models via LoRA`{=html}](https://arxiv.org/abs/2504.15777 "'Tina: Tiny Reasoning Models via LoRA', Wang et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Geometry of Self-Verification in a Task-Specific Reasoning Model”, Lee et al 2025`{=html} {#lee-et-al-2025-section}

**[`The Geometry of Self-Verification in a Task-Specific Reasoning Model`{=html}](https://arxiv.org/abs/2504.14379 "'The Geometry of Self-Verification in a Task-Specific Reasoning Model', Lee et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Is Google Gemini-2.5-Pro Now Better Than Claude at <em>Pokémon</em>? [Probably]”, Bradshaw 2025`{=html} {#bradshaw-2025-section}

**[`Is Google Gemini-2.5-pro now better than Claude at <em>Pokémon</em>? [probably]`{=html}](https://www.lesswrong.com/posts/7mqp8uRnnPdbBzJZE/is-gemini-now-better-than-claude-at-pokemon "'Is Google Gemini-2.5-pro now better than Claude at <em>Pokémon</em>? [probably]', Bradshaw 2025"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?”, Yue et al 2025`{=html} {#yue-et-al-2025-section}

**[`Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?`{=html}](https://arxiv.org/abs/2504.13837 "'Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?', Yue et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Interpreting Emergent Planning in Model-Free Reinforcement Learning”, Bush et al 2025`{=html} {#bush-et-al-2025-section}

**[`Interpreting Emergent Planning in Model-Free Reinforcement Learning`{=html}](https://arxiv.org/abs/2504.01871 "'Interpreting Emergent Planning in Model-Free Reinforcement Learning', Bush et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Video-T1: Test-Time Scaling for Video Generation”, Liu et al 2025`{=html} {#liu-et-al-2025-section}

**[`Video-T1: Test-Time Scaling for Video Generation`{=html}](https://arxiv.org/abs/2503.18942 "'Video-T1: Test-Time Scaling for Video Generation', Liu et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“(How) Do Language Models Track State?”, Li et al 2025`{=html} {#li-et-al-2025-section}

**[`(How) Do Language Models Track State?`{=html}](https://arxiv.org/abs/2503.02854 "'(How) Do Language Models Track State?', Li et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“NaturalReasoning: Reasoning in the Wild With 2.8M Challenging Questions”, Yuan et al 2025`{=html} {#yuan-et-al-2025-naturalreasoning-section}

**[`NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions`{=html}](https://arxiv.org/abs/2502.13124#facebook "'NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions', Yuan et al 2025"){.link-annotated
.id-not .include-annotation link-icon="facebook" link-icon-type="svg"
link-icon-color="#1877f2"}**

## `“Training Language Models for Social Deduction With Multi-Agent Reinforcement Learning”, Sarkar et al 2025`{=html} {#sarkar-et-al-2025-section}

**[`Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning`{=html}](https://arxiv.org/abs/2502.06060 "'Training Language Models for Social Deduction with Multi-Agent Reinforcement Learning', Sarkar et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Competitive Programming With Large Reasoning Models”, El-Kishky et al 2025`{=html} {#el-kishky-et-al-2025-section}

**[`Competitive Programming with Large Reasoning Models`{=html}](https://arxiv.org/abs/2502.06807#openai "'Competitive Programming with Large Reasoning Models', El-Kishky et al 2025"){.link-annotated
.id-not .include-annotation link-icon="openai" link-icon-type="svg"}**

## `“MR.Q: Towards General-Purpose Model-Free Reinforcement Learning”, Fujimoto et al 2025`{=html} {#fujimoto-et-al-2025-section}

**[`MR.Q: Towards General-Purpose Model-Free Reinforcement Learning`{=html}](https://arxiv.org/abs/2501.16142#facebook "'MR.Q: Towards General-Purpose Model-Free Reinforcement Learning', Fujimoto et al 2025"){.link-annotated
.id-not .include-annotation link-icon="facebook" link-icon-type="svg"
link-icon-color="#1877f2"}**

## `“DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning”, Guo et al 2025`{=html} {#guo-et-al-2025-section}

**[`DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning`{=html}](https://arxiv.org/abs/2501.12948#deepseek "'DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning', Guo et al 2025"){.link-annotated
.id-not .include-annotation link-icon="deepseek" link-icon-type="svg"
link-icon-color="#4d6bfe"}**

## `“Deployment of an Aerial Multi-Agent System for Automated Task Execution in Large-Scale Underground Mining Environments”, Dahlquist et al 2025`{=html} {#dahlquist-et-al-2025-section}

**[`Deployment of an Aerial Multi-agent System for Automated Task Execution in Large-scale Underground Mining Environments`{=html}](https://arxiv.org/abs/2501.10262 "'Deployment of an Aerial Multi-agent System for Automated Task Execution in Large-scale Underground Mining Environments', Dahlquist et al 2025"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Generalized Dijkstra in Haskell”, flupe 2024`{=html} {#flupe-2024-section}

**[`Generalized Dijkstra in Haskell`{=html}](https://acatalepsie.fr/posts/haskell-dijkstra.html "'Generalized Dijkstra in Haskell', flupe 2024"){.link-annotated-partial
.id-not .include-annotation}**

## `“Proposing and Solving Olympiad Geometry With Guided Tree Search”, Zhang et al 2024`{=html} {#zhang-et-al-2024-section}

**[`Proposing and solving olympiad geometry with guided tree search`{=html}](https://arxiv.org/abs/2412.10673 "'Proposing and solving olympiad geometry with guided tree search', Zhang et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Centaur: a Foundation Model of Human Cognition”, Binz et al 2024`{=html} {#binz-et-al-2024-section}

**[`Centaur: a foundation model of human cognition`{=html}](https://arxiv.org/abs/2410.20268 "'Centaur: a foundation model of human cognition', Binz et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making”, Li et al 2024`{=html} {#li-et-al-2024-1-section}

**[`Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making`{=html}](https://arxiv.org/abs/2410.07166 "'Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making', Li et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Interpretable Contrastive Monte Carlo Tree Search Reasoning”, Gao et al 2024`{=html} {#gao-et-al-2024-section}

**[`Interpretable Contrastive Monte Carlo Tree Search Reasoning`{=html}](https://arxiv.org/abs/2410.01707 "'Interpretable Contrastive Monte Carlo Tree Search Reasoning', Gao et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“OpenAI Co-Founder Sutskever’s New Safety-Focused AI Startup SSI Raises $1 Billion”, Cai et al 2024`{=html} {#cai-et-al-2024-section}

**[`OpenAI co-founder Sutskever’s new safety-focused AI startup SSI raises $1 billion`{=html}](https://www.reuters.com/technology/artificial-intelligence/openai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04/ "'OpenAI co-founder Sutskever’s new safety-focused AI startup SSI raises $1 billion', Cai et al 2024"){.link-annotated
.id-not .include-annotation link-icon="R" link-icon-type="text,sans"
link-icon-color="#e56218"}**

## `“The Brain Simulates Actions and Their Consequences during REM Sleep”, Senzai & Scanziani 2024`{=html} {#senzai-scanziani-2024-section}

**[`The brain simulates actions and their consequences during REM sleep`{=html}](https://www.biorxiv.org/content/10.1101/2024.08.13.607810.full "'The brain simulates actions and their consequences during REM sleep', Senzai & Scanziani 2024"){.link-annotated
.id-not .include-annotation link-icon="chi-dna" link-icon-type="svg"
link-icon-color="#bd2736"}**

## `“Solving <em>Path of Exile</em> Item Crafting With Value Iteration”, Britz 2024`{=html} {#britz-2024-section}

**[`Solving <em>Path of Exile</em> item crafting with Value Iteration`{=html}](https://dennybritz.com/posts/poe-crafting/ "'Solving <em>Path of Exile</em> item crafting with Value Iteration', Britz 2024"){.link-annotated-partial
.id-not .include-annotation}**

## `“Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models”, Denison et al 2024`{=html} {#denison-et-al-2024-section}

**[`Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models`{=html}](https://arxiv.org/abs/2406.10162#anthropic "'Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Models', Denison et al 2024"){.link-annotated
.id-not .include-annotation link-icon="anthropic" link-icon-type="svg"
link-icon-color="#d4a27f"}**

## `“DT-VIN: Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning”, Wang et al 2024`{=html} {#wang-et-al-2024-03-section}

**[`DT-VIN: Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning`{=html}](https://arxiv.org/abs/2406.08404#schmidhuber "'DT-VIN: Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning', Wang et al 2024"){.link-annotated
.id-not .include-annotation link-icon="SMDH"
link-icon-type="text,quad,sans"}**

## `“MCTSr: Accessing GPT-4 Level Mathematical Olympiad Solutions via Monte Carlo Tree Self-Refine With LLaMA-3-8B”, Zhang et al 2024`{=html} {#zhang-et-al-2024-02-section}

**[`MCTSr: Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMA-3-8B`{=html}](https://arxiv.org/abs/2406.07394 "'MCTSr: Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMA-3-8B', Zhang et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Safety Alignment Should Be Made More Than Just a Few Tokens Deep”, Qi et al 2024`{=html} {#qi-et-al-2024-section}

**[`Safety Alignment Should Be Made More Than Just a Few Tokens Deep`{=html}](https://arxiv.org/abs/2406.05946 "'Safety Alignment Should Be Made More Than Just a Few Tokens Deep', Qi et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Can Language Models Serve As Text-Based World Simulators?”, Wang et al 2024`{=html} {#wang-et-al-2024-04-section}

**[`Can Language Models Serve as Text-Based World Simulators?`{=html}](https://arxiv.org/abs/2406.06485 "'Can Language Models Serve as Text-Based World Simulators?', Wang et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Evaluating the World Model Implicit in a Generative Model”, Vafa et al 2024`{=html} {#vafa-et-al-2024-section}

**[`Evaluating the World Model Implicit in a Generative Model`{=html}](https://arxiv.org/abs/2406.03689 "'Evaluating the World Model Implicit in a Generative Model', Vafa et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“OmegaPRM: Improve Mathematical Reasoning in Language Models by Automated Process Supervision”, Luo et al 2024`{=html} {#luo-et-al-2024-section}

**[`OmegaPRM: Improve Mathematical Reasoning in Language Models by Automated Process Supervision`{=html}](https://arxiv.org/abs/2406.06592#deepmind "'OmegaPRM: Improve Mathematical Reasoning in Language Models by Automated Process Supervision', Luo et al 2024"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Diffusion On Syntax Trees For Program Synthesis”, Kapur et al 2024`{=html} {#kapur-et-al-2024-section}

**[`Diffusion On Syntax Trees For Program Synthesis`{=html}](https://arxiv.org/abs/2405.20519 "'Diffusion On Syntax Trees For Program Synthesis', Kapur et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“A Pontryagin Perspective on Reinforcement Learning”, Eberhard et al 2024`{=html} {#eberhard-et-al-2024-section}

**[`A Pontryagin Perspective on Reinforcement Learning`{=html}](https://arxiv.org/abs/2405.18100 "'A Pontryagin Perspective on Reinforcement Learning', Eberhard et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches With TikZ”, Belouadi et al 2024`{=html} {#belouadi-et-al-2024-section}

**[`DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ`{=html}](https://arxiv.org/abs/2405.15306 "'DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ', Belouadi et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data”, Xin et al 2024`{=html} {#xin-et-al-2024-section}

**[`DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data`{=html}](https://arxiv.org/abs/2405.14333#deepseek "'DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data', Xin et al 2024"){.link-annotated
.id-not .include-annotation link-icon="deepseek" link-icon-type="svg"
link-icon-color="#4d6bfe"}**

## `“Amit’s A<sup>✱</sup> Pages”, Patel 2024`{=html} {#patel-2024-2-section}

**[`Amit’s A<sup>✱</sup> Pages`{=html}](https://theory.stanford.edu/~amitp/GameProgramming/ "'Amit’s A<sup>✱</sup> Pages', Patel 2024"){.id-not
.include-annotation}**

## `“From <em>r</em> to <em>Q</em><sup>✱</sup>: Your Language Model Is Secretly a Q-Function”, Rafailov et al 2024`{=html} {#rafailov-et-al-2024-section}

**[`From <em>r</em> to <em>Q</em><sup>✱</sup>: Your Language Model is Secretly a Q-Function`{=html}](https://arxiv.org/abs/2404.12358 "'From <em>r</em> to <em>Q</em><sup>✱</sup>: Your Language Model is Secretly a Q-Function', Rafailov et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Algorithmic Collusion by Large Language Models”, Fish et al 2024`{=html} {#fish-et-al-2024-section}

**[`Algorithmic Collusion by Large Language Models`{=html}](https://arxiv.org/abs/2404.00806 "'Algorithmic Collusion by Large Language Models', Fish et al 2024"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Identifying General Reaction Conditions by Bandit Optimization”, Wang et al 2024b`{=html} {#wang-et-al-2024b-section}

**[`Identifying general reaction conditions by bandit optimization`{=html}](/doc/reinforcement-learning/model/2024-wang-2.pdf "'Identifying general reaction conditions by bandit optimization', Wang et al 2024b"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Gradient-Based Planning With World Models”, V et al 2023`{=html} {#v-et-al-2023-section}

**[`Gradient-based Planning with World Models`{=html}](https://arxiv.org/abs/2312.17227 "'Gradient-based Planning with World Models', V et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“ReCoRe: Regularized Contrastive Representation Learning of World Model”, Poudel et al 2023`{=html} {#poudel-et-al-2023-section}

**[`ReCoRe: Regularized Contrastive Representation Learning of World Model`{=html}](https://arxiv.org/abs/2312.09056 "'ReCoRe: Regularized Contrastive Representation Learning of World Model', Poudel et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Can a Transformer Represent a Kalman Filter?”, Goel & Bartlett 2023`{=html} {#goel-bartlett-2023-section}

**[`Can a Transformer Represent a Kalman Filter?`{=html}](https://arxiv.org/abs/2312.06937 "'Can a Transformer Represent a Kalman Filter?', Goel & Bartlett 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Self-Supervised Behavior Cloned Transformers Are Path Crawlers for Text Games”, Wang & Jansen 2023`{=html} {#wang-jansen-2023-section}

**[`Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games`{=html}](https://arxiv.org/abs/2312.04657 "'Self-Supervised Behavior Cloned Transformers are Path Crawlers for Text Games', Wang & Jansen 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Why Won’t OpenAI Say What the Q<sup>✱</sup> Algorithm Is? Supposed AI Breakthroughs Are Frequently Veiled in Secrecy, Hindering Scientific Consensus”, Hao 2023`{=html} {#hao-2023-section}

**[`Why Won’t OpenAI Say What the Q<sup>✱</sup> Algorithm Is? Supposed AI breakthroughs are frequently veiled in secrecy, hindering scientific consensus`{=html}](https://www.theatlantic.com/technology/archive/2023/11/openai-sam-altman-q-algorithm-breakthrough-project/676163/ "'Why Won’t OpenAI Say What the Q<sup>✱</sup> Algorithm Is? Supposed AI breakthroughs are frequently veiled in secrecy, hindering scientific consensus', Hao 2023"){.link-annotated
.id-not .include-annotation link-icon="A" link-icon-type="text,italic"
link-icon-color="#e7131a"}**

## `“Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations”, Hong et al 2023`{=html} {#hong-et-al-2023-section}

**[`Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations`{=html}](https://arxiv.org/abs/2311.05584 "'Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversations', Hong et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Neural Basis of Mental Navigation in Rats: A Brain–machine Interface Demonstrates Volitional Control of Hippocampal Activity”, Coulter & Kemere 2023`{=html} {#coulter-kemere-2023-section}

**[`The neural basis of mental navigation in rats: A brain–machine interface demonstrates volitional control of hippocampal activity`{=html}](/doc/psychology/neuroscience/2023-coulter.pdf "'The neural basis of mental navigation in rats: A brain–machine interface demonstrates volitional control of hippocampal activity', Coulter & Kemere 2023"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Volitional Activation of Remote Place Representations With a Hippocampal Brain–machine Interface”, Lai et al 2023`{=html} {#lai-et-al-2023-2-section}

**[`Volitional activation of remote place representations with a hippocampal brain–machine interface`{=html}](/doc/psychology/neuroscience/2023-lai.pdf "'Volitional activation of remote place representations with a hippocampal brain–machine interface', Lai et al 2023"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion”, Zhang et al 2023`{=html} {#zhang-et-al-2023-08-section}

**[`Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion`{=html}](https://arxiv.org/abs/2311.01017 "'Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion', Zhang et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Self-AIXI: Self-Predictive Universal AI”, Catt et al 2023`{=html} {#catt-et-al-2023-section}

**[`Self-AIXI: Self-Predictive Universal AI`{=html}](https://openreview.net/forum?id=psXVkKO9No#deepmind "'Self-AIXI: Self-Predictive Universal AI', Catt et al 2023"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Othello Is Solved”, Takizawa 2023`{=html} {#takizawa-2023-section}

**[`Othello is Solved`{=html}](https://arxiv.org/abs/2310.19387 "'Othello is Solved', Takizawa 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Course Correcting Koopman Representations”, Fathi et al 2023`{=html} {#fathi-et-al-2023-1-section}

**[`Course Correcting Koopman Representations`{=html}](https://arxiv.org/abs/2310.15386 "'Course Correcting Koopman Representations', Fathi et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Predictive Auxiliary Objectives in Deep RL Mimic Learning in the Brain”, Fang & Stachenfeld 2023`{=html} {#fang-stachenfeld-2023-section}

**[`Predictive auxiliary objectives in deep RL mimic learning in the brain`{=html}](https://arxiv.org/abs/2310.06089 "'Predictive auxiliary objectives in deep RL mimic learning in the brain', Fang & Stachenfeld 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models”, Zhou et al 2023`{=html} {#zhou-et-al-2023-04-section}

**[`Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models`{=html}](https://arxiv.org/abs/2310.04406 "'Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models', Zhou et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Comparative Study of Model-Based and Model-Free Reinforcement Learning Control Performance in HVAC Systems”, Gao & Wang 2023`{=html} {#gao-wang-2023-section}

**[`Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems`{=html}](/doc/reinforcement-learning/model/2023-gao.pdf "'Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems', Gao & Wang 2023"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Learning to Model the World With Language”, Lin et al 2023`{=html} {#lin-et-al-2023-6-section}

**[`Learning to Model the World with Language`{=html}](https://arxiv.org/abs/2308.01399 "'Learning to Model the World with Language', Lin et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations”, Chen et al 2023`{=html} {#chen-et-al-2023-section}

**[`Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations`{=html}](https://arxiv.org/abs/2307.08678 "'Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations', Chen et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Fighting Uncertainty With Gradients: Offline Reinforcement Learning via Diffusion Score Matching”, Suh et al 2023`{=html} {#suh-et-al-2023-section}

**[`Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching`{=html}](https://arxiv.org/abs/2306.14079 "'Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matching', Suh et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought”, Wong et al 2023`{=html} {#wong-et-al-2023-2-section}

**[`From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought`{=html}](https://arxiv.org/abs/2306.12672 "'From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought', Wong et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Improving Long-Horizon Imitation Through Instruction Prediction”, Hejna et al 2023`{=html} {#hejna-et-al-2023-section}

**[`Improving Long-Horizon Imitation Through Instruction Prediction`{=html}](https://arxiv.org/abs/2306.12554 "'Improving Long-Horizon Imitation Through Instruction Prediction', Hejna et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming (CDHF)”, Mozannar et al 2023`{=html} {#mozannar-et-al-2023-section}

**[`When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming (CDHF)`{=html}](https://arxiv.org/abs/2306.04930#microsoft "'When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming (CDHF)', Mozannar et al 2023"){.link-annotated
.id-not .include-annotation link-icon="MS"
link-icon-type="text,sans,italic"}**

## `“Reasoning With Language Model Is Planning With World Model”, Hao et al 2023`{=html} {#hao-et-al-2023-section}

**[`Reasoning with Language Model is Planning with World Model`{=html}](https://arxiv.org/abs/2305.14992 "'Reasoning with Language Model is Planning with World Model', Hao et al 2023"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Micromouse: The Fastest Maze-Solving Competition On Earth”, Veritasium 2023`{=html} {#veritasium-2023-section}

**[`Micromouse: The Fastest Maze-Solving Competition On Earth`{=html}](https://www.youtube.com/watch?v=ZMQbHMgK2rw "'Micromouse: The Fastest Maze-Solving Competition On Earth', Veritasium 2023"){.link-annotated-partial
.id-not .include-annotation link-icon="youtube" link-icon-type="svg"
link-icon-color="#ff0033"}**

## `“Long-Term Value of Exploration: Measurements, Findings and Algorithms”, Su et al 2023`{=html} {#su-et-al-2023-section}

**[`Long-Term Value of Exploration: Measurements, Findings and Algorithms`{=html}](https://arxiv.org/abs/2305.07764#google "'Long-Term Value of Exploration: Measurements, Findings and Algorithms', Su et al 2023"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Emergence of Belief-Like Representations through Reinforcement Learning”, Hennig et al 2023`{=html} {#hennig-et-al-2023-section}

**[`Emergence of belief-like representations through reinforcement learning`{=html}](https://www.biorxiv.org/content/10.1101/2023.04.04.535512.full "'Emergence of belief-like representations through reinforcement learning', Hennig et al 2023"){.link-annotated
.id-not .include-annotation link-icon="chi-dna" link-icon-type="svg"
link-icon-color="#bd2736"}**

## `“Six Experiments in Action Minimization”, Greydanus 2023`{=html} {#greydanus-2023-physicsbackprop-1-section}

**[`Six Experiments in Action Minimization`{=html}](https://greydanus.github.io/2023/03/12/ncf-six-experiments/ "'Six Experiments in Action Minimization', Greydanus 2023"){.id-not
.include-annotation}**

::: aux-links-transclude-file
::: collapse
**View External Link**:

[`https://greydanus.github.io/2023/03/12/ncf-six-experiments/`](https://greydanus.github.io/2023/03/12/ncf-six-experiments/){.id-not
.link-annotated-not .include-content .include-lazy}
:::
:::

## `“Finding Paths of Least Action With Gradient Descent”, Greydanus 2023`{=html} {#greydanus-2023-2023-physicsbackprop-2-section}

**[`Finding Paths of Least Action with Gradient Descent`{=html}](https://greydanus.github.io/2023/03/05/ncf-tutorial/ "'Finding Paths of Least Action with Gradient Descent', Greydanus 2023"){.id-not
.include-annotation}**

::: aux-links-transclude-file
::: collapse
**View External Link**:

[`https://greydanus.github.io/2023/03/05/ncf-tutorial/`](https://greydanus.github.io/2023/03/05/ncf-tutorial/){.id-not
.link-annotated-not .include-content .include-lazy}
:::
:::

## `“MimicPlay: Long-Horizon Imitation Learning by Watching Human Play”, Wang et al 2023`{=html} {#wang-et-al-2023-16-section}

**[`MimicPlay: Long-Horizon Imitation Learning by Watching Human Play`{=html}](https://arxiv.org/abs/2302.12422#nvidia "'MimicPlay: Long-Horizon Imitation Learning by Watching Human Play', Wang et al 2023"){.link-annotated
.id-not .include-annotation link-icon="n"
link-icon-type="text,sans,italic" link-icon-color="#77ba00"}**

## `“Graph Schemas As Abstractions for Transfer Learning, Inference, and Planning”, Guntupalli et al 2023`{=html} {#guntupalli-et-al-2023-section}

**[`Graph schemas as abstractions for transfer learning, inference, and planning`{=html}](https://arxiv.org/abs/2302.07350#deepmind "'Graph schemas as abstractions for transfer learning, inference, and planning', Guntupalli et al 2023"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“John Carmack’s ‘Different Path’ to Artificial General Intelligence”, Carmack 2023`{=html} {#carmack-2023-section}

**[`John Carmack’s ‘Different Path’ to Artificial General Intelligence`{=html}](https://dallasinnovates.com/exclusive-qa-john-carmacks-different-path-to-artificial-general-intelligence/ "'John Carmack’s ‘Different Path’ to Artificial General Intelligence', Carmack 2023"){.link-annotated
.id-not .include-annotation}**

## `“DreamerV3: Mastering Diverse Domains through World Models”, Hafner et al 2023`{=html} {#hafner-et-al-2023-section}

**[`DreamerV3: Mastering Diverse Domains through World Models`{=html}](https://arxiv.org/abs/2301.04104#deepmind "'DreamerV3: Mastering Diverse Domains through World Models', Hafner et al 2023"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Merging Enzymatic and Synthetic Chemistry With Computational Synthesis Planning”, Levin et al 2022`{=html} {#levin-et-al-2022-1-section}

**[`Merging enzymatic and synthetic chemistry with computational synthesis planning`{=html}](https://www.nature.com/articles/s41467-022-35422-y "'Merging enzymatic and synthetic chemistry with computational synthesis planning', Levin et al 2022"){.link-annotated
.id-not .include-annotation link-icon="n" link-icon-type="text"}**

## `“PALMER: Perception-Action Loop With Memory for Long-Horizon Planning”, Beker et al 2022`{=html} {#beker-et-al-2022-section}

**[`PALMER: Perception-Action Loop with Memory for Long-Horizon Planning`{=html}](https://arxiv.org/abs/2212.04581 "'PALMER: Perception-Action Loop with Memory for Long-Horizon Planning', Beker et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Space Is a Latent [CSCG] Sequence: Structured Sequence Learning As a Unified Theory of Representation in the Hippocampus”, Raju et al 2022`{=html} {#raju-et-al-2022-section}

**[`Space is a latent [CSCG] sequence: Structured sequence learning as a unified theory of representation in the hippocampus`{=html}](https://arxiv.org/abs/2212.01508#deepmind "'Space is a latent [CSCG] sequence: Structured sequence learning as a unified theory of representation in the hippocampus', Raju et al 2022"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“CICERO: Human-Level Play in the Game of <em>Diplomacy</em> by Combining Language Models With Strategic Reasoning”, Bakhtin et al 2022`{=html} {#bakhtin-et-al-2022-2-section}

**[`CICERO: Human-level play in the game of <em>Diplomacy</em> by combining language models with strategic reasoning`{=html}](/doc/reinforcement-learning/imperfect-information/diplomacy/2022-bakhtin.pdf "'CICERO: Human-level play in the game of <em>Diplomacy</em> by combining language models with strategic reasoning', Bakhtin et al 2022"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Online Learning and Bandits With Queried Hints”, Bhaskara et al 2022`{=html} {#bhaskara-et-al-2022-section}

**[`Online Learning and Bandits with Queried Hints`{=html}](https://arxiv.org/abs/2211.02703 "'Online Learning and Bandits with Queried Hints', Bhaskara et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“E3B: Exploration via Elliptical Episodic Bonuses”, Henaff et al 2022`{=html} {#henaff-et-al-2022-section}

**[`E3B: Exploration via Elliptical Episodic Bonuses`{=html}](https://arxiv.org/abs/2210.05805#facebook "'E3B: Exploration via Elliptical Episodic Bonuses', Henaff et al 2022"){.link-annotated
.id-not .include-annotation link-icon="facebook" link-icon-type="svg"
link-icon-color="#1877f2"}**

## `“Creating a Dynamic Quadrupedal Robotic Goalkeeper With Reinforcement Learning”, Huang et al 2022`{=html} {#huang-et-al-2022-3-section}

**[`Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning`{=html}](https://arxiv.org/abs/2210.04435 "'Creating a Dynamic Quadrupedal Robotic Goalkeeper with Reinforcement Learning', Huang et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Top-Down Design of Protein Nanomaterials With Reinforcement Learning”, Lutz et al 2022`{=html} {#lutz-et-al-2022-section}

**[`Top-down design of protein nanomaterials with reinforcement learning`{=html}](https://www.biorxiv.org/content/10.1101/2022.09.25.509419.full "'Top-down design of protein nanomaterials with reinforcement learning', Lutz et al 2022"){.link-annotated
.id-not .include-annotation link-icon="chi-dna" link-icon-type="svg"
link-icon-color="#bd2736"}**

## `“Simplifying Model-Based RL: Learning Representations, Latent-Space Models, and Policies With One Objective (ALM)”, Ghugare et al 2022`{=html} {#ghugare-et-al-2022-section}

**[`Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective (ALM)`{=html}](https://arxiv.org/abs/2209.08466 "'Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective (ALM)', Ghugare et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“IRIS: Transformers Are Sample-Efficient World Models”, Micheli et al 2022`{=html} {#micheli-et-al-2022-section}

**[`IRIS: Transformers are Sample-Efficient World Models`{=html}](https://arxiv.org/abs/2209.00588 "'IRIS: Transformers are Sample-Efficient World Models', Micheli et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“LGE: Cell-Free Latent Go-Explore”, Gallouédec & Dellandréa 2022`{=html} {#gallouédec-dellandréa-2022-section}

**[`LGE: Cell-Free Latent Go-Explore`{=html}](https://arxiv.org/abs/2208.14928 "'LGE: Cell-Free Latent Go-Explore', Gallouédec & Dellandréa 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“LaTTe: Language Trajectory TransformEr”, Bucker et al 2022`{=html} {#bucker-et-al-2022-section}

**[`LaTTe: Language Trajectory TransformEr`{=html}](https://arxiv.org/abs/2208.02918#microsoft "'LaTTe: Language Trajectory TransformEr', Bucker et al 2022"){.link-annotated
.id-not .include-annotation link-icon="MS"
link-icon-type="text,sans,italic"}**

## `“PI-ARS: Accelerating Evolution-Learned Visual-Locomotion With Predictive Information Representations”, Lee et al 2022`{=html} {#lee-et-al-2022-06-section}

**[`PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations`{=html}](https://arxiv.org/abs/2207.13224#google "'PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations', Lee et al 2022"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Learning With Combinatorial Optimization Layers: a Probabilistic Approach”, Dalle et al 2022`{=html} {#dalle-et-al-2022-section}

**[`Learning with Combinatorial Optimization Layers: a Probabilistic Approach`{=html}](https://arxiv.org/abs/2207.13513 "'Learning with Combinatorial Optimization Layers: a Probabilistic Approach', Dalle et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Spatial Representation by Ramping Activity of Neurons in the Retrohippocampal Cortex”, Tennant et al 2022`{=html} {#tennant-et-al-2022-section}

**[`Spatial representation by ramping activity of neurons in the retrohippocampal cortex`{=html}](https://www.biorxiv.org/content/10.1101/2021.03.15.435518.full "'Spatial representation by ramping activity of neurons in the retrohippocampal cortex', Tennant et al 2022"){.link-annotated
.id-not .include-annotation link-icon="chi-dna" link-icon-type="svg"
link-icon-color="#bd2736"}**

## `“Inner Monologue: Embodied Reasoning through Planning With Language Models”, Huang et al 2022`{=html} {#huang-et-al-2022-5-section}

**[`Inner Monologue: Embodied Reasoning through Planning with Language Models`{=html}](https://arxiv.org/abs/2207.05608#google "'Inner Monologue: Embodied Reasoning through Planning with Language Models', Huang et al 2022"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“LM-Nav: Robotic Navigation With Large Pre-Trained Models of Language, Vision, and Action”, Shah et al 2022`{=html} {#shah-et-al-2022-section}

**[`LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action`{=html}](https://arxiv.org/abs/2207.04429 "'LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action', Shah et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DayDreamer: World Models for Physical Robot Learning”, Wu et al 2022`{=html} {#wu-et-al-2022-06-section}

**[`DayDreamer: World Models for Physical Robot Learning`{=html}](https://arxiv.org/abs/2206.14176 "'DayDreamer: World Models for Physical Robot Learning', Wu et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos”, Baker et al 2022`{=html} {#baker-et-al-2022-2-section}

**[`Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos`{=html}](https://arxiv.org/abs/2206.11795#openai "'Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos', Baker et al 2022"){.link-annotated
.id-not .include-annotation link-icon="openai" link-icon-type="svg"}**

## `“GODEL: Large-Scale Pre-Training for Goal-Directed Dialog”, Peng et al 2022`{=html} {#peng-et-al-2022-2-section}

**[`GODEL: Large-Scale Pre-Training for Goal-Directed Dialog`{=html}](https://arxiv.org/abs/2206.11309#microsoft "'GODEL: Large-Scale Pre-Training for Goal-Directed Dialog', Peng et al 2022"){.link-annotated
.id-not .include-annotation link-icon="MS"
link-icon-type="text,sans,italic"}**

## `“BYOL-Explore: Exploration by Bootstrapped Prediction”, Guo et al 2022`{=html} {#guo-et-al-2022-4-section}

**[`BYOL-Explore: Exploration by Bootstrapped Prediction`{=html}](https://arxiv.org/abs/2206.08332#deepmind "'BYOL-Explore: Exploration by Bootstrapped Prediction', Guo et al 2022"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Director: Deep Hierarchical Planning from Pixels”, Hafner et al 2022`{=html} {#hafner-et-al-2022-section}

**[`Director: Deep Hierarchical Planning from Pixels`{=html}](https://arxiv.org/abs/2206.04114#google "'Director: Deep Hierarchical Planning from Pixels', Hafner et al 2022"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Flexible Diffusion Modeling of Long Videos”, Harvey et al 2022`{=html} {#harvey-et-al-2022-section}

**[`Flexible Diffusion Modeling of Long Videos`{=html}](https://arxiv.org/abs/2205.11495 "'Flexible Diffusion Modeling of Long Videos', Harvey et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Housekeep: Tidying Virtual Households Using Commonsense Reasoning”, Kant et al 2022`{=html} {#kant-et-al-2022-section}

**[`Housekeep: Tidying Virtual Households using Commonsense Reasoning`{=html}](https://arxiv.org/abs/2205.10712 "'Housekeep: Tidying Virtual Households using Commonsense Reasoning', Kant et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Semantic Exploration from Language Abstractions and Pretrained Representations”, Tam et al 2022`{=html} {#tam-et-al-2022-section}

**[`Semantic Exploration from Language Abstractions and Pretrained Representations`{=html}](https://arxiv.org/abs/2204.05080#deepmind "'Semantic Exploration from Language Abstractions and Pretrained Representations', Tam et al 2022"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning”, Valassakis et al 2022`{=html} {#valassakis-et-al-2022-section}

**[`Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning`{=html}](https://arxiv.org/abs/2204.02863 "'Demonstrate Once, Imitate Immediately (DOME): Learning Visual Servoing for One-Shot Imitation Learning', Valassakis et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances”, Ahn et al 2022`{=html} {#ahn-et-al-2022-section}

**[`Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances`{=html}](https://arxiv.org/abs/2204.01691#google "'Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances', Ahn et al 2022"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Reinforcement Learning With Action-Free Pre-Training from Videos”, Seo et al 2022`{=html} {#seo-et-al-2022-section}

**[`Reinforcement Learning with Action-Free Pre-Training from Videos`{=html}](https://arxiv.org/abs/2203.13880 "'Reinforcement Learning with Action-Free Pre-Training from Videos', Seo et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“On-The-Fly Strategy Adaptation for Ad-Hoc Agent Coordination”, Zand et al 2022`{=html} {#zand-et-al-2022-section}

**[`On-the-fly Strategy Adaptation for ad-hoc Agent Coordination`{=html}](https://arxiv.org/abs/2203.08015 "'On-the-fly Strategy Adaptation for ad-hoc Agent Coordination', Zand et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“VAPO: Affordance Learning from Play for Sample-Efficient Policy Learning”, Borja-Diaz et al 2022`{=html} {#borja-diaz-et-al-2022-section}

**[`VAPO: Affordance Learning from Play for Sample-Efficient Policy Learning`{=html}](https://arxiv.org/abs/2203.00352 "'VAPO: Affordance Learning from Play for Sample-Efficient Policy Learning', Borja-Diaz et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning Synthetic Environments and Reward Networks for Reinforcement Learning”, Ferreira et al 2022`{=html} {#ferreira-et-al-2022-2-section}

**[`Learning Synthetic Environments and Reward Networks for Reinforcement Learning`{=html}](https://arxiv.org/abs/2202.02790 "'Learning Synthetic Environments and Reward Networks for Reinforcement Learning', Ferreira et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“How to Build a Cognitive Map: Insights from Models of the Hippocampal Formation”, Whittington et al 2022`{=html} {#whittington-et-al-2022-section}

**[`How to build a cognitive map: insights from models of the hippocampal formation`{=html}](https://arxiv.org/abs/2202.01682 "'How to build a cognitive map: insights from models of the hippocampal formation', Whittington et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“LID: Pre-Trained Language Models for Interactive Decision-Making”, Li et al 2022`{=html} {#li-et-al-2022-19-section}

**[`LID: Pre-Trained Language Models for Interactive Decision-Making`{=html}](https://arxiv.org/abs/2202.01771 "'LID: Pre-Trained Language Models for Interactive Decision-Making', Li et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Rotting Infinitely Many-Armed Bandits”, Kim et al 2022`{=html} {#kim-et-al-2022-8-section}

**[`Rotting Infinitely Many-armed Bandits`{=html}](https://arxiv.org/abs/2201.12975 "'Rotting Infinitely Many-armed Bandits', Kim et al 2022"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Language Models As Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents”, Huang et al 2022`{=html} {#huang-et-al-2022-6-section}

**[`Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents`{=html}](https://arxiv.org/abs/2201.07207#google "'Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents', Huang et al 2022"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“What Is the Point of Computers? A Question for Pure Mathematicians”, Buzzard 2021`{=html} {#buzzard-2021-section}

**[`What is the point of computers? A question for pure mathematicians`{=html}](https://arxiv.org/abs/2112.11598 "'What is the point of computers? A question for pure mathematicians', Buzzard 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“An Experimental Design Perspective on Model-Based Reinforcement Learning”, Mehta et al 2021`{=html} {#mehta-et-al-2021-2-section}

**[`An Experimental Design Perspective on Model-Based Reinforcement Learning`{=html}](https://arxiv.org/abs/2112.05244 "'An Experimental Design Perspective on Model-Based Reinforcement Learning', Mehta et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Residual Pathway Priors for Soft Equivariance Constraints”, Finzi et al 2021`{=html} {#finzi-et-al-2021-section}

**[`Residual Pathway Priors for Soft Equivariance Constraints`{=html}](https://arxiv.org/abs/2112.01388 "'Residual Pathway Priors for Soft Equivariance Constraints', Finzi et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates”, Kantack 2021`{=html} {#kantack-2021-section}

**[`Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates`{=html}](https://arxiv.org/abs/2111.09800 "'Reinforcement Learning on Human Decision Models for Uniquely Collaborative AI Teammates', Kantack 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning Representations for Pixel-Based Control: What Matters and Why?”, Tomar et al 2021`{=html} {#tomar-et-al-2021-section}

**[`Learning Representations for Pixel-based Control: What Matters and Why?`{=html}](https://arxiv.org/abs/2111.07775 "'Learning Representations for Pixel-based Control: What Matters and Why?', Tomar et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning Behaviors through Physics-Driven Latent Imagination”, Richard et al 2021`{=html} {#richard-et-al-2021-section}

**[`Learning Behaviors through Physics-driven Latent Imagination`{=html}](https://openreview.net/forum?id=YwDvofEWlEx "'Learning Behaviors through Physics-driven Latent Imagination', Richard et al 2021"){.link-annotated
.id-not .include-annotation link-icon="OR" link-icon-type="text,sans"
link-icon-color="#8c1b13"}**

## `“Is Bang-Bang Control All You Need? Solving Continuous Control With Bernoulli Policies”, Seyde et al 2021`{=html} {#seyde-et-al-2021-section}

**[`Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies`{=html}](https://arxiv.org/abs/2111.02552 "'Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies', Seyde et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Skill Induction and Planning With Latent Language”, Sharma et al 2021`{=html} {#sharma-et-al-2021-2-section}

**[`Skill Induction and Planning with Latent Language`{=html}](https://arxiv.org/abs/2110.01517 "'Skill Induction and Planning with Latent Language', Sharma et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks”, Wu et al 2021`{=html} {#wu-et-al-2021-09-section}

**[`Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks`{=html}](https://arxiv.org/abs/2109.10312 "'Example-Driven Model-Based Reinforcement Learning for Solving Long-Horizon Visuomotor Tasks', Wu et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“TrufLL: Learning Natural Language Generation from Scratch”, Donati et al 2021`{=html} {#donati-et-al-2021-2-section}

**[`TrufLL: Learning Natural Language Generation from Scratch`{=html}](https://arxiv.org/abs/2109.09371 "'TrufLL: Learning Natural Language Generation from Scratch', Donati et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Dropout’s Dream Land: Generalization from Learned Simulators to Reality”, Wellmer & Kwok 2021`{=html} {#wellmer-kwok-2021-section}

**[`Dropout’s Dream Land: Generalization from Learned Simulators to Reality`{=html}](https://arxiv.org/abs/2109.08342 "'Dropout’s Dream Land: Generalization from Learned Simulators to Reality', Wellmer & Kwok 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning [EMPA]”, Tsividis et al 2021`{=html} {#tsividis-et-al-2021-section}

**[`Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning [EMPA]`{=html}](https://arxiv.org/abs/2107.12544 "'Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning [EMPA]', Tsividis et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“FitVid: Overfitting in Pixel-Level Video Prediction”, Babaeizadeh et al 2021`{=html} {#babaeizadeh-et-al-2021-section}

**[`FitVid: Overfitting in Pixel-Level Video Prediction`{=html}](https://arxiv.org/abs/2106.13195#google "'FitVid: Overfitting in Pixel-Level Video Prediction', Babaeizadeh et al 2021"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation”, Freeman et al 2021`{=html} {#freeman-et-al-2021-section}

**[`Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation`{=html}](https://arxiv.org/abs/2106.13281#google "'Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation', Freeman et al 2021"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“A Graph Placement Methodology for Fast Chip Design”, Mirhoseini et al 2021`{=html} {#mirhoseini-et-al-2021-section}

**[`A graph placement methodology for fast chip design`{=html}](/doc/reinforcement-learning/model/2021-mirhoseini.pdf#google "'A graph placement methodology for fast chip design', Mirhoseini et al 2021"){.link-annotated-partial
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Planning for Novelty: Width-Based Algorithms for Common Problems in Control, Planning and Reinforcement Learning”, Lipovetzky 2021`{=html} {#lipovetzky-2021-section}

**[`Planning for Novelty: Width-Based Algorithms for Common Problems in Control, Planning and Reinforcement Learning`{=html}](https://arxiv.org/abs/2106.04866 "'Planning for Novelty: Width-Based Algorithms for Common Problems in Control, Planning and Reinforcement Learning', Lipovetzky 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Whole Prefrontal Cortex Is Premotor Cortex”, Fine & Hayden 2021`{=html} {#fine-hayden-2021-section}

**[`The whole prefrontal cortex is premotor cortex`{=html}](https://arxiv.org/abs/2106.04651 "'The whole prefrontal cortex is premotor cortex', Fine & Hayden 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World”, Zellers et al 2021`{=html} {#zellers-et-al-2021-section}

**[`PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World`{=html}](https://arxiv.org/abs/2106.00188 "'PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World', Zellers et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Constructions in Combinatorics via Neural Networks”, Wagner 2021`{=html} {#wagner-2021-section}

**[`Constructions in combinatorics via neural networks`{=html}](https://arxiv.org/abs/2104.14516 "'Constructions in combinatorics via neural networks', Wagner 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Machine Translation Decoding beyond Beam Search”, Leblond et al 2021`{=html} {#leblond-et-al-2021-section}

**[`Machine Translation Decoding beyond Beam Search`{=html}](https://arxiv.org/abs/2104.05336#deepmind "'Machine Translation Decoding beyond Beam Search', Leblond et al 2021"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Learning What To Do by Simulating the Past”, Lindner et al 2021`{=html} {#lindner-et-al-2021-section}

**[`Learning What To Do by Simulating the Past`{=html}](https://arxiv.org/abs/2104.03946 "'Learning What To Do by Simulating the Past', Lindner et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Waymo Simulated Driving Behavior in Reconstructed Fatal Crashes within an Autonomous Vehicle Operating Domain”, Scanlon et al 2021`{=html} {#scanlon-et-al-2021-section}

**[`Waymo Simulated Driving Behavior in Reconstructed Fatal Crashes within an Autonomous Vehicle Operating Domain`{=html}](/doc/reinforcement-learning/model/2021-scanlon.pdf "'Waymo Simulated Driving Behavior in Reconstructed Fatal Crashes within an Autonomous Vehicle Operating Domain', Scanlon et al 2021"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing”, Brunnbauer et al 2021`{=html} {#brunnbauer-et-al-2021-section}

**[`Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing`{=html}](https://arxiv.org/abs/2103.04909 "'Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing', Brunnbauer et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Replaying Real Life: How the Waymo Driver Avoids Fatal Human Crashes”, Waymo 2021`{=html} {#waymo-2021-section}

**[`Replaying real life: how the Waymo Driver avoids fatal human crashes`{=html}](https://waymo.com/blog/2021/03/replaying-real-life/ "'Replaying real life: how the Waymo Driver avoids fatal human crashes', Waymo 2021"){.link-annotated
.id-not .include-annotation}**

## `“Learning Chess Blindfolded: Evaluating Language Models on State Tracking”, Toshniwal et al 2021`{=html} {#toshniwal-et-al-2021-section}

**[`Learning Chess Blindfolded: Evaluating Language Models on State Tracking`{=html}](https://arxiv.org/abs/2102.13249 "'Learning Chess Blindfolded: Evaluating Language Models on State Tracking', Toshniwal et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“COMBO: Conservative Offline Model-Based Policy Optimization”, Yu et al 2021`{=html} {#yu-et-al-2021-3-section}

**[`COMBO: Conservative Offline Model-Based Policy Optimization`{=html}](https://arxiv.org/abs/2102.08363 "'COMBO: Conservative Offline Model-Based Policy Optimization', Yu et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“A<sup>✱</sup> Search Without Expansions: Learning Heuristic Functions With Deep Q-Networks”, Agostinelli et al 2021`{=html} {#agostinelli-et-al-2021-section}

**[`A<sup>✱</sup> Search Without Expansions: Learning Heuristic Functions with Deep Q-Networks`{=html}](https://arxiv.org/abs/2102.04518 "'A<sup>✱</sup> Search Without Expansions: Learning Heuristic Functions with Deep Q-Networks', Agostinelli et al 2021"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“ViNG: Learning Open-World Navigation With Visual Goals”, Shah et al 2020`{=html} {#shah-et-al-2020-section}

**[`ViNG: Learning Open-World Navigation with Visual Goals`{=html}](https://arxiv.org/abs/2012.09812 "'ViNG: Learning Open-World Navigation with Visual Goals', Shah et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Inductive Biases for Deep Learning of Higher-Level Cognition”, Goyal & Bengio 2020`{=html} {#goyal-bengio-2020-section}

**[`Inductive Biases for Deep Learning of Higher-Level Cognition`{=html}](https://arxiv.org/abs/2011.15091 "'Inductive Biases for Deep Learning of Higher-Level Cognition', Goyal & Bengio 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Multimodal Dynamics Modeling for Off-Road Autonomous Vehicles”, Tremblay et al 2020`{=html} {#tremblay-et-al-2020-section}

**[`Multimodal dynamics modeling for off-road autonomous vehicles`{=html}](https://arxiv.org/abs/2011.11751 "'Multimodal dynamics modeling for off-road autonomous vehicles', Tremblay et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Targeting for Long-Term Outcomes”, Yang et al 2020`{=html} {#yang-et-al-2020-3-section}

**[`Targeting for long-term outcomes`{=html}](https://arxiv.org/abs/2010.15835 "'Targeting for long-term outcomes', Yang et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“What Are the Statistical Limits of Offline RL With Linear Function Approximation?”, Wang et al 2020`{=html} {#wang-et-al-2020-06-section}

**[`What are the Statistical Limits of Offline RL with Linear Function Approximation?`{=html}](https://arxiv.org/abs/2010.11895 "'What are the Statistical Limits of Offline RL with Linear Function Approximation?', Wang et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“A Time Leap Challenge for SAT Solving”, Fichte et al 2020`{=html} {#fichte-et-al-2020-section}

**[`A Time Leap Challenge for SAT Solving`{=html}](https://arxiv.org/abs/2008.02215 "'A Time Leap Challenge for SAT Solving', Fichte et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Overfitted Brain: Dreams Evolved to Assist Generalization”, Hoel 2020`{=html} {#hoel-2020-section}

**[`The Overfitted Brain: Dreams evolved to assist generalization`{=html}](https://arxiv.org/abs/2007.09560 "'The Overfitted Brain: Dreams evolved to assist generalization', Hoel 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning”, Gulcehre et al 2020`{=html} {#gulcehre-et-al-2020-section}

**[`RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning`{=html}](https://arxiv.org/abs/2006.13888 "'RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning', Gulcehre et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Mathematical Reasoning via Self-Supervised Skip-Tree Training”, Rabe et al 2020`{=html} {#rabe-et-al-2020-section}

**[`Mathematical Reasoning via Self-supervised Skip-tree Training`{=html}](https://arxiv.org/abs/2006.04757 "'Mathematical Reasoning via Self-supervised Skip-tree Training', Rabe et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“MOPO: Model-Based Offline Policy Optimization”, Yu et al 2020`{=html} {#yu-et-al-2020-1-section}

**[`MOPO: Model-based Offline Policy Optimization`{=html}](https://arxiv.org/abs/2005.13239v6 "'MOPO: Model-based Offline Policy Optimization', Yu et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Simulate Dynamic Environments With GameGAN”, Kim et al 2020`{=html} {#kim-et-al-2020-gamegan-paper-section}

**[`Learning to Simulate Dynamic Environments with GameGAN`{=html}](https://arxiv.org/abs/2005.12126#nvidia "'Learning to Simulate Dynamic Environments with GameGAN', Kim et al 2020"){.link-annotated
.id-not .include-annotation link-icon="n"
link-icon-type="text,sans,italic" link-icon-color="#77ba00"}**

## `“Planning to Explore via Self-Supervised World Models”, Sekar et al 2020`{=html} {#sekar-et-al-2020-section}

**[`Planning to Explore via Self-Supervised World Models`{=html}](https://arxiv.org/abs/2005.05960 "'Planning to Explore via Self-Supervised World Models', Sekar et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Simulate Dynamic Environments With GameGAN [Homepage]”, Kim et al 2020`{=html} {#kim-et-al-2020-gamegan-repo-section}

**[`Learning to Simulate Dynamic Environments with GameGAN [homepage]`{=html}](https://nv-tlabs.github.io/gameGAN/#nvidia "'Learning to Simulate Dynamic Environments with GameGAN [homepage]', Kim et al 2020"){.link-annotated
.id-not .include-annotation link-icon="n"
link-icon-type="text,sans,italic" link-icon-color="#77ba00"}**

## `“Reinforcement Learning With Augmented Data”, Laskin et al 2020`{=html} {#laskin-et-al-2020-2-section}

**[`Reinforcement Learning with Augmented Data`{=html}](https://arxiv.org/abs/2004.14990 "'Reinforcement Learning with Augmented Data', Laskin et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Fly via Deep Model-Based Reinforcement Learning”, Becker-Ehmck et al 2020`{=html} {#becker-ehmck-et-al-2020-section}

**[`Learning to Fly via Deep Model-Based Reinforcement Learning`{=html}](https://arxiv.org/abs/2003.08876 "'Learning to Fly via Deep Model-Based Reinforcement Learning', Becker-Ehmck et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Introducing Dreamer: Scalable Reinforcement Learning Using World Models”, Hafner 2020`{=html} {#hafner-2020-section}

**[`Introducing Dreamer: Scalable Reinforcement Learning Using World Models`{=html}](https://research.google/blog/introducing-dreamer-scalable-reinforcement-learning-using-world-models/ "'Introducing Dreamer: Scalable Reinforcement Learning Using World Models', Hafner 2020"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Reinforcement Learning for Combinatorial Optimization: A Survey”, Mazyavkina et al 2020`{=html} {#mazyavkina-et-al-2020-section}

**[`Reinforcement Learning for Combinatorial Optimization: A Survey`{=html}](https://arxiv.org/abs/2003.03600 "'Reinforcement Learning for Combinatorial Optimization: A Survey', Mazyavkina et al 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Prove Theorems by Learning to Generate Theorems”, Wang & Deng 2020`{=html} {#wang-deng-2020-section}

**[`Learning to Prove Theorems by Learning to Generate Theorems`{=html}](https://arxiv.org/abs/2002.07019 "'Learning to Prove Theorems by Learning to Generate Theorems', Wang & Deng 2020"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Gambler’s Problem and Beyond”, Wang et al 2019`{=html} {#wang-et-al-2019-1-section}

**[`The Gambler’s Problem and Beyond`{=html}](https://arxiv.org/abs/2001.00102 "'The Gambler’s Problem and Beyond', Wang et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Combining Q-Learning and Search With Amortized Value Estimates”, Hamrick et al 2019`{=html} {#hamrick-et-al-2019-section}

**[`Combining Q-Learning and Search with Amortized Value Estimates`{=html}](https://arxiv.org/abs/1912.02807#deepmind "'Combining Q-Learning and Search with Amortized Value Estimates', Hamrick et al 2019"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Dream to Control: Learning Behaviors by Latent Imagination”, Hafner et al 2019`{=html} {#hafner-et-al-2019-section}

**[`Dream to Control: Learning Behaviors by Latent Imagination`{=html}](https://arxiv.org/abs/1912.01603#googledeepmind "'Dream to Control: Learning Behaviors by Latent Imagination', Hafner et al 2019"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Approximate Inference in Discrete Distributions With Monte Carlo Tree Search and Value Functions”, Buesing et al 2019`{=html} {#buesing-et-al-2019-section}

**[`Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions`{=html}](https://arxiv.org/abs/1910.06862 "'Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functions', Buesing et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?”, Du et al 2019`{=html} {#du-et-al-2019-4-section}

**[`Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?`{=html}](https://arxiv.org/abs/1910.03016 "'Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?', Du et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Designing Agent Incentives to Avoid Reward Tampering”, Everitt et al 2019`{=html} {#everitt-et-al-2019-1-section}

**[`Designing agent incentives to avoid reward tampering`{=html}](https://deepmindsafetyresearch.medium.com/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd "'Designing agent incentives to avoid reward tampering', Everitt et al 2019"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“An Application of Reinforcement Learning to Aerobatic Helicopter Flight”, Abbeel et al 2019`{=html} {#abbeel-et-al-2019-section}

**[`An Application of Reinforcement Learning to Aerobatic Helicopter Flight`{=html}](https://ai.stanford.edu/~pabbeel/pubs/AbbeelCoatesQuigleyNg_aaorltahf_nips2006.pdf "'An Application of Reinforcement Learning to Aerobatic Helicopter Flight', Abbeel et al 2019"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“When to Trust Your Model: Model-Based Policy Optimization (MOPO)”, Janner et al 2019`{=html} {#janner-et-al-2019-section}

**[`When to Trust Your Model: Model-Based Policy Optimization (MOPO)`{=html}](https://arxiv.org/abs/1906.08253 "'When to Trust Your Model: Model-Based Policy Optimization (MOPO)', Janner et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“VISR: Fast Task Inference With Variational Intrinsic Successor Features”, Hansen et al 2019`{=html} {#hansen-et-al-2019-section}

**[`VISR: Fast Task Inference with Variational Intrinsic Successor Features`{=html}](https://arxiv.org/abs/1906.05030 "'VISR: Fast Task Inference with Variational Intrinsic Successor Features', Hansen et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Write, Execute, Assess: Program Synthesis With a REPL”, Ellis et al 2019`{=html} {#ellis-et-al-2019-section}

**[`Write, Execute, Assess: Program Synthesis with a REPL`{=html}](https://arxiv.org/abs/1906.04604 "'Write, Execute, Assess: Program Synthesis with a REPL', Ellis et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Reason in Large Theories without Imitation”, Bansal et al 2019`{=html} {#bansal-et-al-2019-section}

**[`Learning to Reason in Large Theories without Imitation`{=html}](https://arxiv.org/abs/1905.10501 "'Learning to Reason in Large Theories without Imitation', Bansal et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Biasing MCTS With Features for General Games”, Soemers et al 2019`{=html} {#soemers-et-al-2019-section}

**[`Biasing MCTS with Features for General Games`{=html}](https://arxiv.org/abs/1903.08942 "'Biasing MCTS with Features for General Games', Soemers et al 2019"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DRC: An Investigation of Model-Free Planning”, Guez et al 2019`{=html} {#guez-et-al-2019-section}

**[`DRC: An investigation of model-free planning`{=html}](https://arxiv.org/abs/1901.03559#deepmind "'DRC: An investigation of model-free planning', Guez et al 2019"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“The Credit Assignment Problem”, Demski 2019`{=html} {#demski-2019-section}

**[`The Credit Assignment Problem`{=html}](https://www.lesswrong.com/posts/Ajcq9xWi2fmgn8RBJ/the-credit-assignment-problem "'The Credit Assignment Problem', Demski 2019"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Bayesian Layers: A Module for Neural Network Uncertainty”, Tran et al 2018`{=html} {#tran-et-al-2018-section}

**[`Bayesian Layers: A Module for Neural Network Uncertainty`{=html}](https://arxiv.org/abs/1812.03973#google "'Bayesian Layers: A Module for Neural Network Uncertainty', Tran et al 2018"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“PlaNet: Learning Latent Dynamics for Planning from Pixels”, Hafner et al 2018`{=html} {#hafner-et-al-2018-section}

**[`PlaNet: Learning Latent Dynamics for Planning from Pixels`{=html}](https://arxiv.org/abs/1811.04551#google "'PlaNet: Learning Latent Dynamics for Planning from Pixels', Hafner et al 2018"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning”, Foerster et al 2018`{=html} {#foerster-et-al-2018-section}

**[`Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning`{=html}](https://arxiv.org/abs/1811.01458#deepmind "'Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learning', Foerster et al 2018"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Human-Like Playtesting With Deep Learning”, Gudmundsson et al 2018`{=html} {#gudmundsson-et-al-2018-section}

**[`Human-Like Playtesting with Deep Learning`{=html}](/doc/reinforcement-learning/imitation-learning/2018-gudmundsson.pdf "'Human-Like Playtesting with Deep Learning', Gudmundsson et al 2018"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“General Value Function Networks”, Schlegel et al 2018`{=html} {#schlegel-et-al-2018-section}

**[`General Value Function Networks`{=html}](https://arxiv.org/abs/1807.06763 "'General Value Function Networks', Schlegel et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search”, Zela et al 2018`{=html} {#zela-et-al-2018-section}

**[`Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search`{=html}](https://arxiv.org/abs/1807.06906 "'Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search', Zela et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Alignment Problem for Bayesian History-Based Reinforcement Learners”, Everitt & Hutter 2018`{=html} {#everitt-hutter-2018-section}

**[`The Alignment Problem for Bayesian History-Based Reinforcement Learners`{=html}](/doc/reinforcement-learning/model/2018-everitt.pdf "'The Alignment Problem for Bayesian History-Based Reinforcement Learners', Everitt & Hutter 2018"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Neural Scene Representation and Rendering”, Eslami et al 2018`{=html} {#eslami-et-al-2018-section}

**[`Neural scene representation and rendering`{=html}](/doc/reinforcement-learning/model/2018-eslami.pdf#deepmind "'Neural scene representation and rendering', Eslami et al 2018"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Deep Reinforcement Learning in a Handful of Trials Using Probabilistic Dynamics Models”, Kurtl et al 2018`{=html} {#kurtl-et-al-2018-section}

**[`Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models`{=html}](https://arxiv.org/abs/1805.12114 "'Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models', Kurtl et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Mining Gold from Implicit Models to Improve Likelihood-Free Inference”, Brehmer et al 2018`{=html} {#brehmer-et-al-2018-section}

**[`Mining gold from implicit models to improve likelihood-free inference`{=html}](https://arxiv.org/abs/1805.12244 "'Mining gold from implicit models to improve likelihood-free inference', Brehmer et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning to Optimize Tensor Programs”, Chen et al 2018`{=html} {#chen-et-al-2018-learningtensorprograms-section}

**[`Learning to Optimize Tensor Programs`{=html}](https://arxiv.org/abs/1805.08166 "'Learning to Optimize Tensor Programs', Chen et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Reinforcement Learning and Control As Probabilistic Inference: Tutorial and Review”, Levine 2018`{=html} {#levine-2018-section}

**[`Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review`{=html}](https://arxiv.org/abs/1805.00909 "'Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review', Levine 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks With Existing Applications”, Hadash et al 2018`{=html} {#hadash-et-al-2018-section}

**[`Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications`{=html}](https://arxiv.org/abs/1804.09028#ibm "'Estimate and Replace: A Novel Approach to Integrating Deep Neural Networks with Existing Applications', Hadash et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“World Models”, Ha & Schmidhuber 2018`{=html} {#ha-schmidhuber-2018-section}

**[`World Models`{=html}](https://arxiv.org/abs/1803.10122#google "'World Models', Ha & Schmidhuber 2018"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling”, Riquelme et al 2018`{=html} {#riquelme-et-al-2018-section}

**[`Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling`{=html}](https://arxiv.org/abs/1802.09127#google "'Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Sampling', Riquelme et al 2018"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Differentiable Dynamic Programming for Structured Prediction and Attention”, Mensch & Blondel 2018`{=html} {#mensch-blondel-2018-section}

**[`Differentiable Dynamic Programming for Structured Prediction and Attention`{=html}](https://arxiv.org/abs/1802.03676 "'Differentiable Dynamic Programming for Structured Prediction and Attention', Mensch & Blondel 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Planning With Pixels in (Almost) Real Time”, Bandres et al 2018`{=html} {#bandres-et-al-2018-section}

**[`Planning with Pixels in (Almost) Real Time`{=html}](https://arxiv.org/abs/1801.03354 "'Planning with Pixels in (Almost) Real Time', Bandres et al 2018"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“How to Explore Chemical Space Using Algorithms and Automation”, Gromski et al 2018`{=html} {#gromski-et-al-2018-section}

**[`How to explore chemical space using algorithms and automation`{=html}](/doc/reinforcement-learning/model/2018-gromski.pdf "'How to explore chemical space using algorithms and automation', Gromski et al 2018"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Planning Chemical Syntheses With Deep Neural Networks and Symbolic AI”, Segler et al 2018`{=html} {#segler-et-al-2018-section}

**[`Planning chemical syntheses with deep neural networks and symbolic AI`{=html}](/doc/reinforcement-learning/model/2018-segler.pdf "'Planning chemical syntheses with deep neural networks and symbolic AI', Segler et al 2018"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Generalization Guides Human Exploration in Vast Decision Spaces”, Wu et al 2018`{=html} {#wu-et-al-2018-3-section}

**[`Generalization guides human exploration in vast decision spaces`{=html}](/doc/reinforcement-learning/model/2018-wu.pdf "'Generalization guides human exploration in vast decision spaces', Wu et al 2018"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Safe Policy Search With Gaussian Process Models”, Polymenakos et al 2017`{=html} {#polymenakos-et-al-2017-section}

**[`Safe Policy Search with Gaussian Process Models`{=html}](https://arxiv.org/abs/1712.05556 "'Safe Policy Search with Gaussian Process Models', Polymenakos et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics”, Chatzilygeroudis & Mouret 2017`{=html} {#chatzilygeroudis-mouret-2017-section}

**[`Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics`{=html}](https://arxiv.org/abs/1709.06917 "'Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics', Chatzilygeroudis & Mouret 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Analogical-Based Bayesian Optimization”, Le et al 2017`{=html} {#le-et-al-2017-section}

**[`Analogical-based Bayesian Optimization`{=html}](https://arxiv.org/abs/1709.06390 "'Analogical-based Bayesian Optimization', Le et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“A Game-Theoretic Analysis of the Off-Switch Game”, Wängberg et al 2017`{=html} {#wängberg-et-al-2017-section}

**[`A Game-Theoretic Analysis of the Off-Switch Game`{=html}](https://arxiv.org/abs/1708.03871 "'A Game-Theoretic Analysis of the Off-Switch Game', Wängberg et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Neural Network Dynamics for Model-Based Deep Reinforcement Learning With Model-Free Fine-Tuning”, Nagabandi et al 2017`{=html} {#nagabandi-et-al-2017-section}

**[`Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning`{=html}](https://arxiv.org/abs/1708.02596 "'Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning', Nagabandi et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Learning Transferable Architectures for Scalable Image Recognition”, Zoph et al 2017`{=html} {#zoph-et-al-2017-section}

**[`Learning Transferable Architectures for Scalable Image Recognition`{=html}](https://arxiv.org/abs/1707.07012#google "'Learning Transferable Architectures for Scalable Image Recognition', Zoph et al 2017"){.link-annotated
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Learning Model-Based Planning from Scratch”, Pascanu et al 2017`{=html} {#pascanu-et-al-2017-section}

**[`Learning model-based planning from scratch`{=html}](https://arxiv.org/abs/1707.06170 "'Learning model-based planning from scratch', Pascanu et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Value Prediction Network”, Oh et al 2017`{=html} {#oh-et-al-2017-section}

**[`Value Prediction Network`{=html}](https://arxiv.org/abs/1707.03497#deepmind "'Value Prediction Network', Oh et al 2017"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Path Integral Networks: End-To-End Differentiable Optimal Control”, Okada et al 2017`{=html} {#okada-et-al-2017-section}

**[`Path Integral Networks: End-to-End Differentiable Optimal Control`{=html}](https://arxiv.org/abs/1706.09597 "'Path Integral Networks: End-to-End Differentiable Optimal Control', Okada et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Visual Semantic Planning Using Deep Successor Representations”, Zhu et al 2017`{=html} {#zhu-et-al-2017-1-section}

**[`Visual Semantic Planning using Deep Successor Representations`{=html}](https://arxiv.org/abs/1705.08080 "'Visual Semantic Planning using Deep Successor Representations', Zhu et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“AIXIjs: A Software Demo for General Reinforcement Learning”, Aslanides 2017`{=html} {#aslanides-2017-section}

**[`AIXIjs: A Software Demo for General Reinforcement Learning`{=html}](https://arxiv.org/abs/1705.07615 "'AIXIjs: A Software Demo for General Reinforcement Learning', Aslanides 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Metacontrol for Adaptive Imagination-Based Optimization”, Hamrick et al 2017`{=html} {#hamrick-et-al-2017-section}

**[`Metacontrol for Adaptive Imagination-Based Optimization`{=html}](https://arxiv.org/abs/1705.02670 "'Metacontrol for Adaptive Imagination-Based Optimization', Hamrick et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DeepArchitect: Automatically Designing and Training Deep Architectures”, Negrinho & Gordon 2017`{=html} {#negrinho-gordon-2017-section}

**[`DeepArchitect: Automatically Designing and Training Deep Architectures`{=html}](https://arxiv.org/abs/1704.08792 "'DeepArchitect: Automatically Designing and Training Deep Architectures', Negrinho & Gordon 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Stochastic Constraint Programming As Reinforcement Learning”, Prestwich et al 2017`{=html} {#prestwich-et-al-2017-section}

**[`Stochastic Constraint Programming as Reinforcement Learning`{=html}](https://arxiv.org/abs/1704.07183 "'Stochastic Constraint Programming as Reinforcement Learning', Prestwich et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Recurrent Environment Simulators”, Chiappa et al 2017`{=html} {#chiappa-et-al-2017-section}

**[`Recurrent Environment Simulators`{=html}](https://arxiv.org/abs/1704.02254 "'Recurrent Environment Simulators', Chiappa et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World”, Tobin et al 2017`{=html} {#tobin-et-al-2017-section}

**[`Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World`{=html}](https://arxiv.org/abs/1703.06907 "'Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World', Tobin et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Prediction and Control With Temporal Segment Models”, Mishra et al 2017`{=html} {#mishra-et-al-2017-section}

**[`Prediction and Control with Temporal Segment Models`{=html}](https://arxiv.org/abs/1703.04070 "'Prediction and Control with Temporal Segment Models', Mishra et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Rotting Bandits”, Levine et al 2017`{=html} {#levine-et-al-2017-section}

**[`Rotting Bandits`{=html}](https://arxiv.org/abs/1702.07274 "'Rotting Bandits', Levine et al 2017"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Hippocampus As a Predictive Map”, Stachenfeld et al 2017`{=html} {#stachenfeld-et-al-2017-section}

**[`The hippocampus as a predictive map`{=html}](/doc/reinforcement-learning/model/2017-stachenfeld.pdf "'The hippocampus as a predictive map', Stachenfeld et al 2017"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“The Predictron: End-To-End Learning and Planning”, Silver et al 2016`{=html} {#silver-et-al-2016-2-section}

**[`The Predictron: End-To-End Learning and Planning`{=html}](https://arxiv.org/abs/1612.08810#deepmind "'The Predictron: End-To-End Learning and Planning', Silver et al 2016"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Model-Based Adversarial Imitation Learning”, Baram et al 2016`{=html} {#baram-et-al-2016-section}

**[`Model-based Adversarial Imitation Learning`{=html}](https://arxiv.org/abs/1612.02179 "'Model-based Adversarial Imitation Learning', Baram et al 2016"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“DeepMath: Deep Sequence Models for Premise Selection”, Alemi et al 2016`{=html} {#alemi-et-al-2016-section}

**[`DeepMath: Deep Sequence Models for Premise Selection`{=html}](https://arxiv.org/abs/1606.04442 "'DeepMath: Deep Sequence Models for Premise Selection', Alemi et al 2016"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Death and Suicide in Universal Artificial Intelligence”, Martin et al 2016`{=html} {#martin-et-al-2016-section}

**[`Death and Suicide in Universal Artificial Intelligence`{=html}](https://arxiv.org/abs/1606.00652 "'Death and Suicide in Universal Artificial Intelligence', Martin et al 2016"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Value Iteration Networks”, Tamar et al 2016`{=html} {#tamar-et-al-2016-section}

**[`Value Iteration Networks`{=html}](https://arxiv.org/abs/1602.02867#deepmind "'Value Iteration Networks', Tamar et al 2016"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models”, Schmidhuber 2015`{=html} {#schmidhuber-2015-section}

**[`On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models`{=html}](https://arxiv.org/abs/1511.09249#schmidhuber "'On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models', Schmidhuber 2015"){.link-annotated
.id-not .include-annotation link-icon="SMDH"
link-icon-type="text,quad,sans"}**

## `“Classical Planning Algorithms on the Atari Video Games”, Lipovetzky et al 2015`{=html} {#lipovetzky-et-al-2015-section}

**[`Classical Planning Algorithms on the Atari Video Games`{=html}](/doc/reinforcement-learning/model/2015-lipovetzky.pdf "'Classical Planning Algorithms on the Atari Video Games', Lipovetzky et al 2015"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-Armed Bandit Problem With Multiple Plays”, Komiyama et al 2015`{=html} {#komiyama-et-al-2015-section}

**[`Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays`{=html}](https://arxiv.org/abs/1506.00779 "'Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays', Komiyama et al 2015"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Compress and Control”, Veness et al 2014`{=html} {#veness-et-al-2014-section}

**[`Compress and Control`{=html}](https://arxiv.org/abs/1411.5326#deepmind "'Compress and Control', Veness et al 2014"){.link-annotated
.id-not .include-annotation link-icon="deepmind" link-icon-type="svg"
link-icon-color="#4185f4"}**

## `“Learning to Win by Reading Manuals in a Monte-Carlo Framework”, Branavan et al 2014`{=html} {#branavan-et-al-2014-section}

**[`Learning to Win by Reading Manuals in a Monte-Carlo Framework`{=html}](https://arxiv.org/abs/1401.5390 "'Learning to Win by Reading Manuals in a Monte-Carlo Framework', Branavan et al 2014"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science”, Clark 2013`{=html} {#clark-2013-section}

**[`Whatever next? Predictive brains, situated agents, and the future of cognitive science`{=html}](/doc/ai/nn/2013-clark.pdf "'Whatever next? Predictive brains, situated agents, and the future of cognitive science', Clark 2013"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Model-Based Bayesian Exploration”, Dearden et al 2013`{=html} {#dearden-et-al-2013-section}

**[`Model-Based Bayesian Exploration`{=html}](https://arxiv.org/abs/1301.6690 "'Model-Based Bayesian Exploration', Dearden et al 2013"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“PUCT: Continuous Upper Confidence Trees With Polynomial Exploration-Consistency”, Auger et al 2013`{=html} {#auger-et-al-2013-section}

**[`PUCT: Continuous Upper Confidence Trees with Polynomial Exploration-Consistency`{=html}](/doc/reinforcement-learning/model/2013-auger.pdf "'PUCT: Continuous Upper Confidence Trees with Polynomial Exploration-Consistency', Auger et al 2013"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Planning As Satisfiability: Heuristics”, Rintanen 2012`{=html} {#rintanen-2012-section}

**[`Planning as satisfiability: Heuristics`{=html}](/doc/ai/tabular/2012-rintanen.pdf "'Planning as satisfiability: Heuristics', Rintanen 2012"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Width and Serialization of Classical Planning Problems”, Lipovetzky & Geffner 2012`{=html} {#lipovetzky-geffner-2012-section}

**[`Width and Serialization of Classical Planning Problems`{=html}](/doc/reinforcement-learning/model/2012-lipovetzky.pdf "'Width and Serialization of Classical Planning Problems', Lipovetzky & Geffner 2012"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“An Empirical Evaluation of Thompson Sampling”, Chapelle & Li 2011`{=html} {#chapelle-li-2011-section}

**[`An Empirical Evaluation of Thompson Sampling`{=html}](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.831.5818&rep=rep1&type=pdf "'An Empirical Evaluation of Thompson Sampling', Chapelle & Li 2011"){.id-not
.include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“POMCP: Monte-Carlo Planning in Large POMDPs”, Silver & Veness 2010`{=html} {#silver-veness-2010-section}

**[`POMCP: Monte-Carlo Planning in Large POMDPs`{=html}](/doc/reinforcement-learning/model/2010-silver.pdf "'POMCP: Monte-Carlo Planning in Large POMDPs', Silver & Veness 2010"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“A Monte Carlo AIXI Approximation”, Veness et al 2009`{=html} {#veness-et-al-2009-section}

**[`A Monte Carlo AIXI Approximation`{=html}](https://arxiv.org/abs/0909.0801 "'A Monte Carlo AIXI Approximation', Veness et al 2009"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“The Neural Processes Underpinning Episodic Memory”, Hassabis 2009`{=html} {#hassabis-2009-section}

**[`The Neural Processes Underpinning Episodic Memory`{=html}](/doc/psychology/neuroscience/memory/2009-hassabis.pdf "'The Neural Processes Underpinning Episodic Memory', Hassabis 2009"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Evolution And Episodic Memory: An Analysis And Demonstration Of A Social Function Of Episodic Recollection”, Klein et al 2009`{=html} {#klein-et-al-2009-section}

**[`Evolution And Episodic Memory: An Analysis And Demonstration Of A Social Function Of Episodic Recollection`{=html}](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3559008/ "'Evolution And Episodic Memory: An Analysis And Demonstration Of A Social Function Of Episodic Recollection', Klein et al 2009"){.link-annotated
.id-not .include-annotation link-icon="nlm-ncbi" link-icon-type="svg"
link-icon-color="#20558a"}**

## `“Resilient Machines Through Continuous Self-Modeling”, Bongard et al 2006`{=html} {#bongard-et-al-2006-section}

**[`Resilient Machines Through Continuous Self-Modeling`{=html}](/doc/reinforcement-learning/robot/2006-bongard.pdf "'Resilient Machines Through Continuous Self-Modeling', Bongard et al 2006"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements”, Schmidhuber 2003`{=html} {#schmidhuber-2003-section}

**[`Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements`{=html}](https://arxiv.org/abs/cs/0309048 "'Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements', Schmidhuber 2003"){.link-annotated
.id-not .include-annotation link-icon="𝛘" link-icon-type="text"
link-icon-color="#b31b1b"}**

## `“Policy Mining: Learning Decision Policies from Fixed Sets of Data”, Zadrozny 2003`{=html} {#zadrozny-2003-section}

**[`Policy Mining: Learning Decision Policies from Fixed Sets of Data`{=html}](/doc/statistics/decision/2003-zadrozny.pdf "'Policy Mining: Learning Decision Policies from Fixed Sets of Data', Zadrozny 2003"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions”, Schmidhuber 2002`{=html} {#schmidhuber-2002-2-section}

**[`The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions`{=html}](/doc/reinforcement-learning/model/2002-schmidhuber.pdf "'The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions', Schmidhuber 2002"){.link-annotated-partial
.id-not .include-annotation link-icon="SMDH"
link-icon-type="text,quad,sans"}**

## `“Iterative Widening”, Cazenave 2001`{=html} {#cazenave-2001-section}

**[`Iterative widening`{=html}](/doc/reinforcement-learning/model/2001-cazenave.pdf "'Iterative widening', Cazenave 2001"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Abstract Proof Search”, Cazenave 2000`{=html} {#cazenave-2000-section}

**[`Abstract Proof Search`{=html}](/doc/reinforcement-learning/model/2000-cazenave.pdf "'Abstract Proof Search', Cazenave 2000"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Paul J. Werbos Interview”, Werbos 1998`{=html} {#werbos-1998-section}

**[`Paul J. Werbos interview`{=html}](/doc/ai/nn/rnn/1998-werbos.pdf "'Paul J. Werbos interview', Werbos 1998"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“A Critique of Pure Reason”, McDermott 1987`{=html} {#mcdermott-1987-section}

**[`A critique of pure reason`{=html}](/doc/ai/1987-mcdermott.pdf "'A critique of pure reason', McDermott 1987"){.link-annotated
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Human Window on the World”, Michie 1985`{=html} {#michie-1985-section}

**[`Human Window on the World`{=html}](/doc/ai/1985-michie.pdf "'Human Window on the World', Michie 1985"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Why the Law of Effect Will Not Go Away”, Dennett 1974`{=html} {#dennett-1974-section}

**[`Why the Law of Effect will not Go Away`{=html}](/doc/philosophy/mind/1974-dennett.pdf "'Why the Law of Effect will not Go Away', Dennett 1974"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“OpenAI’s GPT-6 Astra on ARC-AGI-3”`{=html} {#_OJB0zb1Z-section}

**[`OpenAI’s GPT-6 Astra on ARC-AGI-3`{=html}](https://arcprize.org/blog/astra "OpenAI’s GPT-6 Astra on ARC-AGI-3"){.link-modified-recently
.link-annotated-partial .id-not .include-annotation}**

## `“First Place in <em>Tetris 99</em> Using Computer Vision, Classical AI, and a Whole Lot of Free Time”`{=html} {#_ddasRsHN-section}

**[`First place in <em>Tetris 99</em> using computer vision, classical AI, and a whole lot of free time`{=html}](https://bpinzone.github.io/TetrisAI/ "First place in <em>Tetris 99</em> using computer vision, classical AI, and a whole lot of free time"){.link-annotated-partial
.id-not .include-annotation}**

## `“Getting the World Record in HATETRIS”`{=html} {#_r8-3y0Lw-section}

**[`Getting the World Record in HATETRIS`{=html}](https://hallofdreams.org/posts/hatetris/ "Getting the World Record in HATETRIS"){.link-annotated-partial
.id-not .include-annotation}**

## `“Solving Probabilistic Tic-Tac-Toe”, Abraham 2026`{=html} {#_uu9Dez4H-section}

**[`Solving Probabilistic Tic-Tac-Toe`{=html}](https://louisabraham.github.io/articles/probabilistic-tic-tac-toe "'Solving Probabilistic Tic-Tac-Toe', Abraham 2026"){.id-not
.include-annotation}**

::: aux-links-transclude-file
::: collapse
**View External Link**:

[`https://louisabraham.github.io/articles/probabilistic-tic-tac-toe`](https://louisabraham.github.io/articles/probabilistic-tic-tac-toe){.id-not
.link-annotated-not .include-content .include-lazy}
:::
:::

## `“Approximate Bayes Optimal Policy Search Using Neural Networks”`{=html} {#_QEuSe7iu-section}

**[`Approximate Bayes Optimal Policy Search using Neural Networks`{=html}](https://orbi.uliege.be/bitstream/2268/204410/1/ANN-BRL_final.pdf "Approximate Bayes Optimal Policy Search using Neural Networks"){.link-annotated-partial
.id-not .include-annotation link-icon="pdf" link-icon-type="svg"
link-icon-color="#f40f02"}**

## `“Embodying Addiction: A Predictive Processing Account”`{=html} {#_2ddEhECq-section}

**[`Embodying addiction: A predictive processing account`{=html}](https://pmc.ncbi.nlm.nih.gov/articles/PMC6983939/#ab005 "Embodying addiction: A predictive processing account"){.link-annotated-partial
.id-not .include-annotation link-icon="nlm-ncbi" link-icon-type="svg"
link-icon-color="#20558a"}**

## `“Joel Veness”`{=html} {#_z87sQP7F-section}

**[`Joel Veness`{=html}](https://scholar.google.co.uk/citations?user=_iYrAxEAAAAJ "Joel Veness"){.link-annotated-partial
.id-not .include-annotation link-icon="alphabet" link-icon-type="svg"
link-icon-color="#4285f4"}**

## `“Introducing ‘Computer Use’, a New Claude-3.5-Sonnet, and Claude 3.5 Haiku”, Anthropic 2026`{=html} {#_A69mOJ6X-section}

**[`Introducing ‘computer use’, a new Claude-3.5-sonnet, and Claude 3.5 Haiku`{=html}](https://www.anthropic.com/news/3-5-models-and-computer-use "'Introducing ‘computer use’, a new Claude-3.5-sonnet, and Claude 3.5 Haiku', Anthropic 2026"){.link-annotated-partial
.id-not .include-annotation link-icon="anthropic" link-icon-type="svg"
link-icon-color="#d4a27f"}**

## `“Developing a Computer Use Model”, Anthropic 2026`{=html} {#_zrVBZLPx-section}

**[`Developing a computer use model`{=html}](https://www.anthropic.com/research/developing-computer-use "'Developing a computer use model', Anthropic 2026"){.link-annotated-partial
.id-not .include-annotation link-icon="anthropic" link-icon-type="svg"
link-icon-color="#d4a27f"}**

## `“AIXIjs: General Reinforcement Learning in the Browser”`{=html} {#_J_Ophtlm-section}

**[`AIXIjs: General reinforcement learning in the browser`{=html}](https://www.aslanides.io/aixijs/demo.html "AIXIjs: General reinforcement learning in the browser"){.link-annotated-partial
.id-not .include-annotation}**

## `“Why Rationalists Get Depressed”`{=html} {#_ieK_Kody-section}

**[`Why rationalists get depressed`{=html}](https://www.lesswrong.com/posts/eD29LDrjvS4Lrhxkk/why-rationalists-get-depressed-1 "Why rationalists get depressed"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Best-Of-<em>n</em> With Misaligned Reward Models for Math Reasoning”`{=html} {#_kFxovLr1-section}

**[`Best-of-<em>n</em> with misaligned reward models for Math reasoning`{=html}](https://www.lesswrong.com/posts/iJjFRrGQkCxaqKrEo/best-of-n-with-misaligned-reward-models-for-math-reasoning "Best-of-<em>n</em> with misaligned reward models for Math reasoning"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Did Claude 3 Opus Align Itself via Gradient Hacking?”`{=html} {#_k4iqFsKE-section}

**[`Did Claude 3 Opus align itself via gradient hacking?`{=html}](https://www.lesswrong.com/posts/ioZxrP7BhS5ArK59w/did-claude-3-opus-align-itself-via-gradient-hacking "Did Claude 3 Opus align itself via gradient hacking?"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“A Year Late, Claude Finally Beats Pokémon”`{=html} {#_me6Ol9PD-section}

**[`A Year Late, Claude Finally Beats Pokémon`{=html}](https://www.lesswrong.com/posts/sehJYg5Yny9fvpbpt/a-year-late-claude-finally-beats-pokemon "A Year Late, Claude Finally Beats Pokémon"){.link-annotated-partial
.id-not .include-annotation link-icon="LW" link-icon-type="text"
link-icon-color="#7faf83"}**

## `“Claude Plays <em>Pokemon</em>”`{=html} {#_91-NByFY-section}

**[`Claude Plays <em>Pokemon</em>`{=html}](https://www.twitch.tv/claudeplayspokemon "Claude Plays <em>Pokemon</em>"){.id-not
.include-annotation}**

## Sort By Magic

::: {demo-type="sort-by-magic-preface"}
Annotations sorted by machine learning into [inferred \'tags\'](/design#future-tag-features){#gwern-design--future-tag-features
.link-annotated-partial}. This provides an alternative way to browse: instead of by *date* order, one can browse in *topic* order. The \'sorted\' list has been automatically clustered into multiple sections & auto-labeled for easier browsing.

Beginning with the newest annotation, it uses the embedding of each annotation to attempt to create a list of nearest-neighbor annotations, creating a progression of topics. For more details, see the link.
:::

### `episodic-memory` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#hassabis-2009-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#klein-et-al-2009-section){.include
.include-even-when-collapsed}

### `neuroscience` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#buzzard-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fine-hayden-2021-section){.include
.include-even-when-collapsed}

### `differentiable-physics` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#freeman-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#takizawa-2023-section){.include
.include-even-when-collapsed}

### `satisfaction-heuristics` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#soemers-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#rintanen-2012-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fichte-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#rybin-et-al-2025-section){.include
.include-even-when-collapsed}

### `bayesian-uncertainty` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#goel-bartlett-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tran-et-al-2018-section){.include
.include-even-when-collapsed}

### `human-feedback` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#binz-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mozannar-et-al-2023-section){.include
.include-even-when-collapsed}

### `ai-ethics` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#hao-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#cai-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#carmack-2023-section){.include
.include-even-when-collapsed}

### `automation-mining` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#kant-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#dahlquist-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fish-et-al-2024-section){.include
.include-even-when-collapsed}

### `transfer-learning` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#wagner-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#guntupalli-et-al-2023-section){.include
.include-even-when-collapsed}

### `protein-design` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#wang-et-al-2024b-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#levin-et-al-2022-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lutz-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#belouadi-et-al-2024-section){.include
.include-even-when-collapsed}

### `model-based-control` {.collapse title="Machine-generated tag name for the following cluster of links."}

[\[see previous entry\]](#mcdermott-1987-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#schmidhuber-2003-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#catt-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#veness-et-al-2009-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#aslanides-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tsividis-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lindner-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hafner-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hafner-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#brunnbauer-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#richard-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wu-et-al-2022-06-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hafner-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hafner-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#baker-et-al-2022-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#assran-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#seo-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#borja-diaz-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2023-16-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#baram-et-al-2016-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#nagabandi-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kurtl-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#levine-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#eberhard-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#yu-et-al-2020-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#yu-et-al-2021-3-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#janner-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mehta-et-al-2021-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#dearden-et-al-2013-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#riquelme-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#komiyama-et-al-2015-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bhaskara-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#su-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tam-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#li-et-al-2022-19-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#shah-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#shah-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#xu-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zhu-et-al-2017-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#pascanu-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#guez-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bush-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#sekar-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#guo-et-al-2022-4-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#henaff-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#gallouédec-dellandréa-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ji-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ding-ye-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kapur-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ellis-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#schmidhuber-2015-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hamrick-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#negrinho-gordon-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zela-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zoph-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#eslami-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kim-et-al-2020-gamegan-paper-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kim-et-al-2020-gamegan-repo-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#gudmundsson-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#branavan-et-al-2014-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lin-et-al-2023-6-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#huang-et-al-2022-6-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#sharma-et-al-2021-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hejna-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hong-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zhou-et-al-2023-04-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hao-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wong-et-al-2023-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#chen-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#vafa-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lee-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#guo-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#yue-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#karan-du-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#luo-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zhang-et-al-2024-02-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zhang-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tsoukalas-et-al-2026-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#xin-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-deng-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#alemi-et-al-2016-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bansal-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#rabe-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#gao-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#el-kishky-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#huang-et-al-2022-5-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ahn-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hafner-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#team-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2024-04-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-jansen-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#toshniwal-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#li-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#laban-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mukherjee-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#rafailov-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hennig-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fang-stachenfeld-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#goyal-bengio-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#clark-2013-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bongard-et-al-2006-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hoel-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#senzai-scanziani-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lai-et-al-2023-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#coulter-kemere-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#whittington-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#raju-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tennant-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lee-et-al-2022-06-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#chatzilygeroudis-mouret-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#becker-ehmck-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#abbeel-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#huang-et-al-2022-3-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#poudel-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#v-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#okada-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tamar-et-al-2016-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2024-03-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#schlegel-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#oh-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#silver-et-al-2016-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#beker-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wu-et-al-2021-09-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#valassakis-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tobin-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#laskin-et-al-2020-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tomar-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#du-et-al-2019-4-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2020-06-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fathi-et-al-2023-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mishra-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#tremblay-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zhang-et-al-2023-08-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#rozet-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#harvey-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#babaeizadeh-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#liu-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#yuan-et-al-2025-naturalreasoning-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#peng-et-al-2022-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#payne-alloui-cros-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#denison-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#qi-et-al-2024-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#everitt-hutter-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#everitt-et-al-2019-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bhamidipaty-et-al-2026-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ha-schmidhuber-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wellmer-kwok-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ferreira-et-al-2022-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#chiappa-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hansen-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#micheli-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lipovetzky-et-al-2015-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bandres-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lipovetzky-geffner-2012-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#lipovetzky-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#cazenave-2001-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#cazenave-2000-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#silver-veness-2010-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#auger-et-al-2013-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#buesing-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#le-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#brehmer-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#leblond-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#agostinelli-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hamrick-et-al-2019-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#ghugare-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#fujimoto-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#finzi-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#prestwich-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mazyavkina-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#dalle-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#mensch-blondel-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#suh-et-al-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#gulcehre-et-al-2020-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#gao-wang-2023-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#chen-et-al-2018-learningtensorprograms-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#hadash-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#li-et-al-2024-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zellers-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bucker-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#donati-et-al-2021-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#sarkar-et-al-2025-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#foerster-et-al-2018-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kantack-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zand-et-al-2022-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#bakhtin-et-al-2022-2-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wängberg-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#martin-et-al-2016-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#waymo-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#scanlon-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#polymenakos-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#veness-et-al-2014-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#seyde-et-al-2021-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#wang-et-al-2019-1-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#kim-et-al-2022-8-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#levine-et-al-2017-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#yang-et-al-2020-3-section){.include
.include-even-when-collapsed}

[\[see previous entry\]](#zadrozny-2003-section){.include
.include-even-when-collapsed}

## Wikipedia (10) {#titled-links-wikipedia .collapse}

1.  **[`Algorithmic information theory`{=html}](https://en.wikipedia.org/wiki/Algorithmic_information_theory "Algorithmic information theory"){.link-annotated-partial
    .id-not .include-annotation link-icon="wikipedia"
    link-icon-type="svg"}**

2.  **[`Control of chaos`{=html}](https://en.wikipedia.org/wiki/Control_of_chaos "Control of chaos"){.id-not
    .include-annotation link-icon="wikipedia" link-icon-type="svg"}**

    ::: aux-links-transclude-file
    [`https://en.wikipedia.org/wiki/Control_of_chaos`](https://en.wikipedia.org/wiki/Control_of_chaos "Control of chaos"){.id-not
    .include-content include-template="$annotationFileIncludeTemplate"}
    :::

3.  **[`Expectation–maximization algorithm`{=html}](https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm "Expectation–maximization algorithm"){.link-modified-recently
    .link-annotated-partial .id-not .include-annotation
    link-icon="wikipedia" link-icon-type="svg"}**

4.  **[`Good regulator`{=html}](https://en.wikipedia.org/wiki/Good_regulator "Good regulator"){.id-not
    .include-annotation link-icon="wikipedia" link-icon-type="svg"}**

    ::: aux-links-transclude-file
    [`https://en.wikipedia.org/wiki/Good_regulator`](https://en.wikipedia.org/wiki/Good_regulator "Good regulator"){.id-not
    .include-content include-template="$annotationFileIncludeTemplate"}
    :::

5.  **[`Iterative deepening depth-first search`{=html}](https://en.wikipedia.org/wiki/Iterative_deepening_depth-first_search "Iterative deepening depth-first search"){.link-annotated-partial
    .id-not .include-annotation link-icon="wikipedia"
    link-icon-type="svg"}**

6.  **[`Linear-quadratic-Gaussian control`{=html}](https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic%E2%80%93Gaussian_control "Linear-quadratic-Gaussian control"){.id-not
    .include-annotation link-icon="wikipedia" link-icon-type="svg"}**

    ::: aux-links-transclude-file
    [`https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic%E2%80%93Gaussian_control`](https://en.wikipedia.org/wiki/Linear%E2%80%93quadratic%E2%80%93Gaussian_control "Linear-quadratic-Gaussian control"){.id-not
    .include-content include-template="$annotationFileIncludeTemplate"}
    :::

7.  **[`Micromouse competition`{=html}](https://en.wikipedia.org/wiki/Micromouse "Micromouse competition"){.link-annotated-partial
    .id-not .include-annotation link-icon="wikipedia"
    link-icon-type="svg"}**

8.  **[`Model predictive control`{=html}](https://en.wikipedia.org/wiki/Model_predictive_control "Model predictive control"){.link-annotated-partial
    .id-not .include-annotation link-icon="wikipedia"
    link-icon-type="svg"}**

9.  **[`Turnpike theory`{=html}](https://en.wikipedia.org/wiki/Turnpike_theory "Turnpike theory"){.link-annotated-partial
    .id-not .include-annotation link-icon="wikipedia"
    link-icon-type="svg"}**

10. **[`Variety (cybernetics) § Law of requisite variety`{=html}](https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_requisite_variety "Variety (cybernetics) § Law of requisite variety"){.id-not
    .include-annotation link-icon="wikipedia" link-icon-type="svg"}**

    ::: aux-links-transclude-file
    [`https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_requisite_variety`](https://en.wikipedia.org/wiki/Variety_(cybernetics)#Law_of_requisite_variety "Variety (cybernetics) § Law of requisite variety"){.id-not
    .include-content include-template="$annotationFileIncludeTemplate"}
    :::

# Miscellaneous

-   **[`<code>/doc/reinforcement-learning/model/2021-scanlon-waymoaccidentavoidance-worldsimreconstruction-case_AZ1796255_2_surfels_noagent_cropped-2021-03-05_12_56_39.mp4</code>`{=html}](/doc/reinforcement-learning/model/2021-scanlon-waymoaccidentavoidance-worldsimreconstruction-case_AZ1796255_2_surfels_noagent_cropped-2021-03-05_12_56_39.mp4){.link-annotated-partial
    .id-not .include-annotation link-icon="file-video"
    link-icon-type="svg"}**

-   **[`<code>/doc/reinforcement-learning/model/2020-hafner-dreamer-learninganimation.mp4</code>`{=html}](/doc/reinforcement-learning/model/2020-hafner-dreamer-learninganimation.mp4){.link-annotated-partial
    .id-not .include-annotation link-icon="file-video"
    link-icon-type="svg"}**

-   **[`<code>/doc/reinforcement-learning/model/2020-hafner-dreamer-modelpredictions.png</code>`{=html}](/doc/reinforcement-learning/model/2020-hafner-dreamer-modelpredictions.png){.link-annotated-partial
    .id-not .include-annotation link-icon="image"
    link-icon-type="svg"}**

-   **[`<code>/doc/reinforcement-learning/model/2020-hafner-dreamer-threephasearchitecture.png</code>`{=html}](/doc/reinforcement-learning/model/2020-hafner-dreamer-threephasearchitecture.png){.link-annotated-partial
    .id-not .include-annotation link-icon="image"
    link-icon-type="svg"}**

-   **[`<code>/doc/reinforcement-learning/model/2010-silver-figure1-illustrationofpomcpmctssearchoverapomdp.jpg</code>`{=html}](/doc/reinforcement-learning/model/2010-silver-figure1-illustrationofpomcpmctssearchoverapomdp.jpg){.link-annotated-partial
    .id-not .include-annotation link-icon="image"
    link-icon-type="svg"}**

-   **[`<code>http://www.alpha60.de/research/programming_enter/DavidLink_ProgrammingEnter_ComputerResurrection60_2012.pdf</code>`{=html}](http://www.alpha60.de/research/programming_enter/DavidLink_ProgrammingEnter_ComputerResurrection60_2012.pdf){.link-annotated-partial
    .id-not .include-annotation link-icon="pdf" link-icon-type="svg"
    link-icon-color="#f40f02"}**

-   **[`<code>https://blog.evjang.com/2018/08/dijkstras.html</code>`{=html}](https://blog.evjang.com/2018/08/dijkstras.html){.link-annotated-partial
    .id-not .include-annotation}**

-   **[`<code>https://github.com/KeeyanGhoreshi/PokemonFireredSingleSequence</code>`{=html}](https://github.com/KeeyanGhoreshi/PokemonFireredSingleSequence){.id-not
    .include-annotation link-icon="github" link-icon-type="svg"}**

-   **[`<code>https://github.com/Significant-Gravitas/AutoGPT</code>`{=html}](https://github.com/Significant-Gravitas/AutoGPT){.link-annotated-partial
    .id-not .include-annotation link-icon="github"
    link-icon-type="svg"}**

-   **[`<code>https://github.com/sanjeevanahilan/nanoChatGPT</code>`{=html}](https://github.com/sanjeevanahilan/nanoChatGPT){.link-annotated-partial
    .id-not .include-annotation link-icon="github"
    link-icon-type="svg"}**

-   **[`<code>https://iagoleal.com/posts/value-iteration-haskell/</code>`{=html}](https://iagoleal.com/posts/value-iteration-haskell/){.id-not
    .include-annotation}**

-   **[`<code>https://if50.substack.com/p/christopher-strachey-and-the-dawn</code>`{=html}](https://if50.substack.com/p/christopher-strachey-and-the-dawn){.id-not
    .include-annotation link-icon="substack" link-icon-type="svg"
    link-icon-color="#ff6719"}**

-   **[`<code>https://journals.sagepub.com/doi/10.1177/17456916231204811</code>`{=html}](https://journals.sagepub.com/doi/10.1177/17456916231204811){.link-annotated-partial
    .id-not .include-annotation link-icon="SAGE"
    link-icon-type="text,quad,sans" link-icon-color="#046ff8"}**

-   **[`<code>https://madebyoll.in/posts/game_emulation_via_dnn/</code>`{=html}](https://madebyoll.in/posts/game_emulation_via_dnn/){.id-not
    .include-annotation}**

-   **[`<code>https://netflixtechblog.com/artwork-personalization-c589f074ad76</code>`{=html}](https://netflixtechblog.com/artwork-personalization-c589f074ad76){.id-not
    .include-annotation}**

-   **[`<code>https://openai.com/research/vpt</code>`{=html}](https://openai.com/research/vpt){.link-annotated-partial
    .id-not .include-annotation link-icon="openai"
    link-icon-type="svg"}**

-   **[`<code>https://proceedings.neurips.cc/paper_files/paper/2009/file/00411460f7c92d2124a67ea0f4cb5f85-Paper.pdf</code>`{=html}](https://proceedings.neurips.cc/paper_files/paper/2009/file/00411460f7c92d2124a67ea0f4cb5f85-Paper.pdf){.link-annotated-partial
    .id-not .include-annotation link-icon="NIPS"
    link-icon-type="text,sans,quad" link-icon-color="#0d6efd"}**

-   **[`<code>https://www.aboutwayfair.com/careers/tech-blog/contextual-bandit-for-marketing-treatment-optimization</code>`{=html}](https://www.aboutwayfair.com/careers/tech-blog/contextual-bandit-for-marketing-treatment-optimization){.id-not
    .include-annotation}**

-   **[`<code>https://www.bkgm.com/articles/Berliner/ComputerBackgammon/index.html</code>`{=html}](https://www.bkgm.com/articles/Berliner/ComputerBackgammon/index.html){.link-annotated-partial
    .id-not .include-annotation}**

-   **[`<code>https://www.dwarkesh.com/p/demis-hassabis#%C2%A7timestamps</code>`{=html}](https://www.dwarkesh.com/p/demis-hassabis#%C2%A7timestamps){.link-annotated-partial
    .id-not .include-annotation link-icon="Dwkh"
    link-icon-type="text,sans,quad" link-icon-color="#f3c016"}**

-   **[`<code>https://www.everything2.net/index.pl?node_id=1190642</code>`{=html}](https://www.everything2.net/index.pl?node_id=1190642){.link-annotated-partial
    .id-not .include-annotation}**

-   **[`<code>https://www.freepatentsonline.com/y2024/0104353.html#deepmind</code>`{=html}](https://www.freepatentsonline.com/y2024/0104353.html#deepmind){.id-not
    .include-annotation link-icon="deepmind" link-icon-type="svg"
    link-icon-color="#4185f4"}**

    ::: aux-links-transclude-file
    ::: collapse
    **View HTML**:

    [`/doc/www/www.freepatentsonline.com/60589a2ab3b4503180e5d189b4a77b0c00730e73.html#deepmind`](/doc/www/www.freepatentsonline.com/60589a2ab3b4503180e5d189b4a77b0c00730e73.html#deepmind){.id-not
    .link-annotated-not .include-content .include-lazy
    link-icon="deepmind" link-icon-type="svg" link-icon-color="#4185f4"}
    :::
    :::

-   **[`<code>https://www.instacart.com/company/how-its-made/using-contextual-bandit-models-in-large-action-spaces-at-instacart/</code>`{=html}](https://www.instacart.com/company/how-its-made/using-contextual-bandit-models-in-large-action-spaces-at-instacart/){.id-not
    .include-annotation}**

-   **[`<code>https://www.lesswrong.com/posts/S54HKhxQyttNLATKu/deconfusing-direct-vs-amortised-optimization</code>`{=html}](https://www.lesswrong.com/posts/S54HKhxQyttNLATKu/deconfusing-direct-vs-amortised-optimization){.link-annotated-partial
    .id-not .include-annotation link-icon="LW" link-icon-type="text"
    link-icon-color="#7faf83"}**

-   **[`<code>https://www.lesswrong.com/posts/ZwshvqiqCvXPsZEct/the-learning-theoretic-agenda-status-2023</code>`{=html}](https://www.lesswrong.com/posts/ZwshvqiqCvXPsZEct/the-learning-theoretic-agenda-status-2023){.link-annotated-partial
    .id-not .include-annotation link-icon="LW" link-icon-type="text"
    link-icon-color="#7faf83"}**

-   **[`<code>https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world</code>`{=html}](https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world){.id-not
    .include-annotation link-icon="LW" link-icon-type="text"
    link-icon-color="#7faf83"}**

    ::: aux-links-transclude-file
    ::: collapse
    **View External Link**:

    [`https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world`](https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world){.id-not
    .link-annotated-not .include-content .include-lazy link-icon="LW"
    link-icon-type="text" link-icon-color="#7faf83"}
    :::
    :::

-   **[`<code>https://www.quantamagazine.org/electric-ripples-in-the-resting-brain-tag-memories-for-storage-20240521/</code>`{=html}](https://www.quantamagazine.org/electric-ripples-in-the-resting-brain-tag-memories-for-storage-20240521/){.link-annotated-partial
    .id-not .include-annotation link-icon="quanta" link-icon-type="svg"
    link-icon-color="#ff8600"}**

-   **[`<code>https://www.reddit.com/gallery/1d6w6b4</code>`{=html}](https://www.reddit.com/gallery/1d6w6b4){.id-not
    .include-annotation link-icon="reddit" link-icon-type="svg"
    link-icon-color="#ff4500"}**

-   **[`<code>https://www.youtube.com/watch?v=g3lc8BxTPiU</code>`{=html}](https://www.youtube.com/watch?v=g3lc8BxTPiU){.id-not
    .include-annotation link-icon="youtube" link-icon-type="svg"
    link-icon-color="#ff0033"}**

    ::: aux-links-transclude-file
    [`https://www.youtube.com/watch?v=g3lc8BxTPiU`](https://www.youtube.com/watch?v=g3lc8BxTPiU){.link-annotated-not
    .include-content .width-full}
    :::

-   **[`<code>https://x.com/Shalev_lif/status/1883214799938396356</code>`{=html}](https://x.com/Shalev_lif/status/1883214799938396356 "Shalev_lif 2026"){.link-annotated-partial
    .id-not .include-annotation link-icon="twitter" link-icon-type="svg"
    link-icon-color="#1da1f2"}**

-   **[`<code>https://x.com/jeremyphoward/status/1801037736968913128</code>`{=html}](https://x.com/jeremyphoward/status/1801037736968913128 "Howard 2026"){.id-not
    .include-annotation link-icon="twitter" link-icon-type="svg"
    link-icon-color="#1da1f2"}**

    ::: aux-links-transclude-file
    [`https://x.com/jeremyphoward/status/1801037736968913128`](https://x.com/jeremyphoward/status/1801037736968913128 "Howard 2026"){.id-not
    .include-content}
    :::

-   **[`<code>https://x.com/moyix/status/1795284112791703735</code>`{=html}](https://x.com/moyix/status/1795284112791703735 "moyix 2026"){.id-not
    .include-annotation link-icon="twitter" link-icon-type="svg"
    link-icon-color="#1da1f2"}**

    ::: aux-links-transclude-file
    [`https://x.com/moyix/status/1795284112791703735`](https://x.com/moyix/status/1795284112791703735 "moyix 2026"){.id-not
    .include-content}
    :::

-   **[`<code>https://x.com/sand7one/status/1912012664269930896</code>`{=html}](https://x.com/sand7one/status/1912012664269930896 "sand7one 2026"){.link-annotated-partial
    .id-not .include-annotation link-icon="twitter" link-icon-type="svg"
    link-icon-color="#1da1f2"}**

# Bibliography {#link-bibliography-section}

1.  `https://arxiv.org/abs/2512.08153`:
    [`“TreeGRPO: Tree-Advantage GRPO for Online RL Post-Training of Diffusion Models”`{=html}](https://arxiv.org/abs/2512.08153 "'TreeGRPO: Tree-Advantage GRPO for Online RL Post-Training of Diffusion Models', Ding & Ye 2025"){#ding-ye-2025
    .link-annotated}, [Zheng Ding, Weirui Ye]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2512.08153.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2512.08153"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

2.  `https://arxiv.org/abs/2512.04797#deepmind`:
    [`“SIMA 2: A Generalist Embodied Agent for Virtual Worlds”`{=html}](https://arxiv.org/abs/2512.04797#deepmind "'SIMA 2: A Generalist Embodied Agent for Virtual Worlds', team et al 2025"){#team-et-al-2025
    .link-annotated},
    [SIMA team, Adrian Bolton, Alexander Lerchner[, Alexandra Cordell, Alexandre Moufarek, [Andrew Bolt](https://en.wikipedia.org/wiki/Andrew_Bolt "Andrew Bolt"){.id-not}, Andrew Lampinen, Anna Mitenkova, Arne Olav Hallingstad, Bojan Vujatovic, Bonnie Li, Cong Lu, [Daan Wierstra](https://scholar.google.com/citations?user=aDbsf28AAAAJ "Daan Wierstra"){.id-not}, Daniel P. Sawyer, Daniel Slater, [David Reichert](https://en.wikipedia.org/wiki/David_Reichert "David Reichert"){.id-not}, Davide Vercelli, [Demis Hassabis](https://en.wikipedia.org/wiki/Demis_Hassabis "Demis Hassabis"){.link-annotated-partial
    .id-not}, Drew A. Hudson, Duncan Williams, Ed Hirst, Fabio Pardo, [Felix Hill](https://fh295.github.io/ "Language Understanding Grounded in Perception and Action"){.link-annotated-partial
    .id-not}, Frederic Besse, Hannah Openshaw, Harris Chan, Hubert Soyer, Jane X. Wang, [Jeff Clune](http://jeffclune.com/ "Jeff Clune—Professor—Computer Science—University of British Columbia"){.link-annotated-partial
    .id-not}, John Agapiou, John Reid, Joseph Marino, Junkyung Kim, Karol Gregor, Kaustubh Sridhar, Kay McKinney, Laura Kampis, Lei M. Zhang, Loic Matthey, Luyu Wang, Maria Abi Raad, Maria Loks-Thompson, Martin Engelcke, Matija Kecman, Matthew Jackson, Maxime Gazeau, Ollie Purkiss, Oscar Knagg, Peter Stys, Piermaria Mendolicchio, Raia Hadsell, Rosemary Ke, Ryan Faulkner, Sarah Chakera, Satinder Singh Baveja, [Shane Legg](https://en.wikipedia.org/wiki/Shane_Legg "Shane Legg"){.link-annotated-partial
    .id-not}, Sheleem Kashem, Tayfun Terzi, Thomas Keck, Tim Harley, Tim Scholtes, Tyson Roberts, Volodymyr Mnih, Yulan Liu, [Zhengdong Wang](https://zhengdongwang.com/ "'Zhengdong Wang homepage', Wang 2026"){.link-annotated-partial
    .id-not}, [Zoubin Ghahramani](https://en.wikipedia.org/wiki/Zoubin_Ghahramani "Zoubin Ghahramani"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2512.04797%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2512.04797#deepmind"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

3.  `https://arxiv.org/abs/2506.09985#facebook`:
    [`“V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning”`{=html}](https://arxiv.org/abs/2506.09985#facebook "'V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning', Assran et al 2025"){#assran-et-al-2025
    .link-annotated},
    [Mido Assran, Adrien Bardes, David Fan[, Quentin Garrido, Russell Howes, Mojtaba, Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, [Piotr Bojanowski](https://scholar.google.com/citations?user=lJ_oh2EAAAAJ "Piotr Bojanowski"){.id-not}, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier, [Yann LeCun](https://en.wikipedia.org/wiki/Yann_LeCun "Yann LeCun"){.id-not}, Michael Rabbat, Nicolas Ballas]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2506.09985%2523facebook.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2506.09985#facebook"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

4.  `https://arxiv.org/abs/2505.11711`:
    [`“Reinforcement Learning Finetunes Small Subnetworks in Large Language Models”`{=html}](https://arxiv.org/abs/2505.11711 "'Reinforcement Learning Finetunes Small Subnetworks in Large Language Models', Mukherjee et al 2025"){#mukherjee-et-al-2025
    .link-annotated},
    [Sagnik Mukherjee, [Lifan Yuan](https://en.wikipedia.org/wiki/Lifan_Yuan "Lifan Yuan"){.id-not}, Dilek Hakkani-Tur, Hao Peng]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2505.11711.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2505.11711"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

5.  `https://arxiv.org/abs/2504.20571`:
    [`“Reinforcement Learning for Reasoning in Large Language Models With One Training Example”`{=html}](https://arxiv.org/abs/2504.20571 "'Reinforcement Learning for Reasoning in Large Language Models with One Training Example', Wang et al 2025"){#wang-et-al-2025
    .link-annotated},
    [Yiping Wang, Qing Yang, Zhiyuan Zeng[, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/ "Jianfeng Gao at Microsoft Research"){.link-annotated-partial
    .id-not}, [Weizhu Chen](https://scholar.google.com/citations?user=LG_E-4EAAAAJ "Weizhu Chen"){.id-not}, Shuohang Wang, Simon Shaolei Du, Yelong Shen]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2504.20571.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2504.20571"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

6.  `https://arxiv.org/abs/2504.14379`:
    [`“The Geometry of Self-Verification in a Task-Specific Reasoning Model”`{=html}](https://arxiv.org/abs/2504.14379 "'The Geometry of Self-Verification in a Task-Specific Reasoning Model', Lee et al 2025"){#lee-et-al-2025
    .link-annotated},
    [Andrew Lee, Lihao Sun, Chris Wendler[, [Fernanda Viégas](https://en.wikipedia.org/wiki/Fernanda_Vi%C3%A9gas "Fernanda Viégas"){.id-not}, [Martin M. Wattenberg](https://en.wikipedia.org/wiki/Martin_M._Wattenberg "Martin M. Wattenberg"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2504.14379.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2504.14379"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

7.  `https://arxiv.org/abs/2502.06807#openai`:
    [`“Competitive Programming With Large Reasoning Models”`{=html}](https://arxiv.org/abs/2502.06807#openai "'Competitive Programming with Large Reasoning Models', El-Kishky et al 2025"){#el-kishky-et-al-2025
    .link-annotated},
    [Ahmed El-Kishky, Alexander Wei, Andre Saraiva[, Borys Minaev, [Daniel Selsam](https://scholar.google.com/citations?user=yaSqFaEAAAAJ "Daniel Selsam"){.id-not}, [David Dohan](https://scholar.google.com/citations?user=iZ5cY0AAAAAJ&view_op=list_works&sortby=pubdate "David Dohan"){.id-not}, Francis Song, Hunter Lightman, Ignasi Clavera, [Jakub Pachocki](https://en.wikipedia.org/wiki/Jakub_Pachocki "Jakub Pachocki"){.id-not}, Jerry Tworek, Lorenz Kuhn, [Łukasz Kaiser](https://scholar.google.com/citations?user=JWmiQR0AAAAJ "Łukasz Kaiser"){.id-not}, [Mark Chen](https://event.technologyreview.com/emtech-mit-2023/speaker/901826/mark-chen "Speaker Details: EmTech MIT 2023"){.link-annotated-partial
    .id-not}, Max Schwarzer, Mostafa Rohaninejad, [Nat McAleese](https://scholar.google.com/citations?user=crw6TeIAAAAJ "Nat McAleese"){.id-not}, o3 contributors, Oleg Mürk, Rhythm Garg, Rui Shu, Szymon Sidor, Vineet Kosaraju, Wenda Zhou]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2502.06807%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2502.06807#openai"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

8.  `https://arxiv.org/abs/2410.01707`:
    [`“Interpretable Contrastive Monte Carlo Tree Search Reasoning”`{=html}](https://arxiv.org/abs/2410.01707 "'Interpretable Contrastive Monte Carlo Tree Search Reasoning', Gao et al 2024"){#gao-et-al-2024
    .link-annotated},
    [Zitian Gao, Boye Niu, Xuzheng He[, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, Lijie Wen]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2410.01707.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2410.01707"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

9.  `https://www.reuters.com/technology/artificial-intelligence/openai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04/`:
    [`“OpenAI Co-Founder Sutskever’s New Safety-Focused AI Startup SSI Raises $1 Billion”`{=html}](https://www.reuters.com/technology/artificial-intelligence/openai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04/ "'OpenAI co-founder Sutskever’s new safety-focused AI startup SSI raises $1 billion', Cai et al 2024"){#cai-et-al-2024
    .link-annotated},
    [Kenrick Cai, [Krystal Hu](https://www.reuters.com/authors/krystal-hu/){.id-not}, [Anna Tong](https://www.reuters.com/authors/anna-tong/){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.reuters.com%252Ftechnology%252Fartificial-intelligence%252Fopenai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04%252F.html "Directory-tag link-bibliography for link https://www.reuters.com/technology/artificial-intelligence/openai-co-founder-sutskevers-new-safety-focused-ai-startup-ssi-raises-1-billion-2024-09-04/"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

10. `https://arxiv.org/abs/2406.08404#schmidhuber`:
    [`“DT-VIN: Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning”`{=html}](https://arxiv.org/abs/2406.08404#schmidhuber "'DT-VIN: Scaling Value Iteration Networks to 5000 Layers for Extreme Long-Term Planning', Wang et al 2024"){#wang-et-al-2024-03
    .link-annotated},
    [Yuhui Wang, Qingyuan Wu, Weida Li[, Dylan R. Ashley, Francesco Faccio, Chao Huang, [Jürgen Schmidhuber](https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber "Jürgen Schmidhuber"){.link-annotated-partial
    .id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.08404%2523schmidhuber.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.08404#schmidhuber"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

11. `https://arxiv.org/abs/2406.07394`:
    [`“MCTSr: Accessing GPT-4 Level Mathematical Olympiad Solutions via Monte Carlo Tree Self-Refine With LLaMA-3-8B”`{=html}](https://arxiv.org/abs/2406.07394 "'MCTSr: Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMA-3-8B', Zhang et al 2024"){#zhang-et-al-2024-02
    .link-annotated},
    [Di Zhang, Xiaoshui Huang, Dongzhan Zhou[, Yuqiang Li, Wanli Ouyang]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.07394.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.07394"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

12. `https://arxiv.org/abs/2406.03689`:
    [`“Evaluating the World Model Implicit in a Generative Model”`{=html}](https://arxiv.org/abs/2406.03689 "'Evaluating the World Model Implicit in a Generative Model', Vafa et al 2024"){#vafa-et-al-2024
    .link-annotated},
    [Keyon Vafa, Justin Y. Chen, [Jon Kleinberg](https://en.wikipedia.org/wiki/Jon_Kleinberg "Jon Kleinberg"){.id-not}[, [Sendhil Mullainathan](https://en.wikipedia.org/wiki/Sendhil_Mullainathan "Sendhil Mullainathan"){.id-not}, Ashesh Rambachan]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2406.03689.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2406.03689"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

13. `https://arxiv.org/abs/2405.15306`:
    [`“DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches With TikZ”`{=html}](https://arxiv.org/abs/2405.15306 "'DeTikZify: Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ', Belouadi et al 2024"){#belouadi-et-al-2024
    .link-annotated},
    [Jonas Belouadi, Simone Paolo Ponzetto, Steffen Eger]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2405.15306.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2405.15306"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

14. `https://arxiv.org/abs/2404.12358`:
    [`“From <em>r</em> to <em>Q</em><sup>✱</sup>: Your Language Model Is Secretly a Q-Function”`{=html}](https://arxiv.org/abs/2404.12358 "'From <em>r</em> to <em>Q</em><sup>✱</sup>: Your Language Model is Secretly a Q-Function', Rafailov et al 2024"){#rafailov-et-al-2024
    .link-annotated},
    [Rafael Rafailov, Joey Hejna, Ryan Park, [Chelsea Finn](https://en.wikipedia.org/wiki/Chelsea_Finn "Chelsea Finn"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2404.12358.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2404.12358"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

15. `2024-wang-2.pdf`:
    [`“Identifying General Reaction Conditions by Bandit Optimization”`{=html}](/doc/reinforcement-learning/model/2024-wang-2.pdf "'Identifying general reaction conditions by bandit optimization', Wang et al 2024b"){#wang-et-al-2024b
    .link-annotated},
    [Jason Y. Wang, Jason M. Stevens, Stavros K. Kariofillis[, Mai-Jan Tom, Dung L. Golden, Jun Li, Jose E. Tabora, Marvin Parasram, Benjamin J. Shields, David N. Primer, Bo Hao, David Valle, Stacey DiSomma, Ariel Furman, G. Greg Zipp, [Sergey Melnikov](https://en.wikipedia.org/wiki/Sergey_Melnikov "Sergey Melnikov"){.id-not}, [James Paulson](https://en.wikipedia.org/wiki/James_Paulson "James Paulson"){.id-not}, Abigail G. Doyle]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fmodel%252F2024-wang-2.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/model/2024-wang-2.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

16. `https://arxiv.org/abs/2311.01017`:
    [`“Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion”`{=html}](https://arxiv.org/abs/2311.01017 "'Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion', Zhang et al 2023"){#zhang-et-al-2023-08
    .link-annotated},
    [Lunjun Zhang, Yuwen Xiong, Ze Yang[, [Sergio Casas](https://en.wikipedia.org/wiki/Sergio_Casas "Sergio Casas"){.id-not}, Rui Hu, [Raquel Urtasun](https://en.wikipedia.org/wiki/Raquel_Urtasun "Raquel Urtasun"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2311.01017.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2311.01017"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

17. `https://openreview.net/forum?id=psXVkKO9No#deepmind`:
    [`“Self-AIXI: Self-Predictive Universal AI”`{=html}](https://openreview.net/forum?id=psXVkKO9No#deepmind "'Self-AIXI: Self-Predictive Universal AI', Catt et al 2023"){#catt-et-al-2023
    .link-annotated},
    [Elliot Catt, Jordi Grau-Moya, [Marcus Hutter](https://en.wikipedia.org/wiki/Marcus_Hutter "Marcus Hutter"){.id-not}[, Matthew Aitchison, Tim Genewein, Gregoire Deletang, Li Kevin Wenliang, [Joel Veness](https://scholar.google.co.uk/citations?user=_iYrAxEAAAAJ "Joel Veness"){.link-annotated-partial
    .id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fopenreview.net%252Fforum%253Fid%253DpsXVkKO9No%2523deepmind.html "Directory-tag link-bibliography for link https://openreview.net/forum?id=psXVkKO9No#deepmind"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

18. `https://arxiv.org/abs/2310.04406`:
    [`“Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models”`{=html}](https://arxiv.org/abs/2310.04406 "'Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models', Zhou et al 2023"){#zhou-et-al-2023-04
    .link-annotated},
    [Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman[, Haohan Wang, Yu-Xiong Wang]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2310.04406.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2310.04406"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

19. `2023-gao.pdf`:
    [`“Comparative Study of Model-Based and Model-Free Reinforcement Learning Control Performance in HVAC Systems”`{=html}](/doc/reinforcement-learning/model/2023-gao.pdf "'Comparative study of model-based and model-free reinforcement learning control performance in HVAC systems', Gao & Wang 2023"){#gao-wang-2023
    .link-annotated}, [Cheng Gao, Dan Wang]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fmodel%252F2023-gao.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/model/2023-gao.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

20. `https://arxiv.org/abs/2306.04930#microsoft`:
    [`“When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming (CDHF)”`{=html}](https://arxiv.org/abs/2306.04930#microsoft "'When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming (CDHF)', Mozannar et al 2023"){#mozannar-et-al-2023
    .link-annotated},
    [Hussein Mozannar, Gagan Bansal, Adam Fourney, [Eric Horvitz](https://en.wikipedia.org/wiki/Eric_Horvitz "Eric Horvitz"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2306.04930%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2306.04930#microsoft"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

21. `https://arxiv.org/abs/2302.12422#nvidia`:
    [`“MimicPlay: Long-Horizon Imitation Learning by Watching Human Play”`{=html}](https://arxiv.org/abs/2302.12422#nvidia "'MimicPlay: Long-Horizon Imitation Learning by Watching Human Play', Wang et al 2023"){#wang-et-al-2023-16
    .link-annotated},
    [Chen Wang, Linxi Fan, Jiankai Sun[, Ruohan Zhang, [Li Fei-Fei](https://en.wikipedia.org/wiki/Li_Fei-Fei "Li Fei-Fei"){.id-not}, Danfei Xu, Yuke Zhu, [Anima Anandkumar](https://en.wikipedia.org/wiki/Anima_Anandkumar "Anima Anandkumar"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2302.12422%2523nvidia.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2302.12422#nvidia"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

22. `https://arxiv.org/abs/2301.04104#deepmind`:
    [`“DreamerV3: Mastering Diverse Domains through World Models”`{=html}](https://arxiv.org/abs/2301.04104#deepmind "'DreamerV3: Mastering Diverse Domains through World Models', Hafner et al 2023"){#hafner-et-al-2023
    .link-annotated},
    [Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, [Timothy Lillicrap](https://en.wikipedia.org/wiki/Timothy_Lillicrap "Timothy Lillicrap"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2301.04104%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2301.04104#deepmind"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

23. `https://www.nature.com/articles/s41467-022-35422-y`:
    [`“Merging Enzymatic and Synthetic Chemistry With Computational Synthesis Planning”`{=html}](https://www.nature.com/articles/s41467-022-35422-y "'Merging enzymatic and synthetic chemistry with computational synthesis planning', Levin et al 2022"){#levin-et-al-2022-1
    .link-annotated},
    [Itai Levin, Mengjie Liu, Christopher A. Voigt, Connor W. Coley]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwww.nature.com%252Farticles%252Fs41467-022-35422-y.html "Directory-tag link-bibliography for link https://www.nature.com/articles/s41467-022-35422-y"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

24. `2022-bakhtin.pdf`:
    [`“CICERO: Human-Level Play in the Game of <em>Diplomacy</em> by Combining Language Models With Strategic Reasoning”`{=html}](/doc/reinforcement-learning/imperfect-information/diplomacy/2022-bakhtin.pdf "'CICERO: Human-level play in the game of <em>Diplomacy</em> by combining language models with strategic reasoning', Bakhtin et al 2022"){#bakhtin-et-al-2022-2
    .link-annotated},
    [Anton Bakhtin, [Noam Brown](https://scholar.google.com/citations?user=RLDbLcUAAAAJ "Noam Brown"){.id-not}, Emily Dinan[, Gabriele Farina, [Colin Flaherty](https://en.wikipedia.org/wiki/Colin_Flaherty "Colin Flaherty"){.id-not}, [Daniel Fried](https://en.wikipedia.org/wiki/Daniel_Fried "Daniel Fried"){.id-not}, [Andrew Goff](https://en.wikipedia.org/wiki/Andrew_Goff "Andrew Goff"){.id-not}, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, [Adam Lerer](https://adamlerer.github.io/ "Adam Lerer"){.id-not}, [Mike Lewis](https://scholar.google.com/citations?user=SnQnQicAAAAJ "Mike Lewis"){.id-not}, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, Markus Zijlstra]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fimperfect-information%252Fdiplomacy%252F2022-bakhtin.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/imperfect-information/diplomacy/2022-bakhtin.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

25. `https://arxiv.org/abs/2209.08466`:
    [`“Simplifying Model-Based RL: Learning Representations, Latent-Space Models, and Policies With One Objective (ALM)”`{=html}](https://arxiv.org/abs/2209.08466 "'Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objective (ALM)', Ghugare et al 2022"){#ghugare-et-al-2022
    .link-annotated},
    [Raj Ghugare, Homanga Bharadhwaj, Benjamin Eysenbach[, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, [Ruslan Salakhutdinov](https://en.wikipedia.org/wiki/Ruslan_Salakhutdinov "Ruslan Salakhutdinov"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2209.08466.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2209.08466"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

26. `https://arxiv.org/abs/2209.00588`:
    [`“IRIS: Transformers Are Sample-Efficient World Models”`{=html}](https://arxiv.org/abs/2209.00588 "'IRIS: Transformers are Sample-Efficient World Models', Micheli et al 2022"){#micheli-et-al-2022
    .link-annotated},
    [Vincent Micheli, Eloi Alonso, François Fleuret]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2209.00588.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2209.00588"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

27. `https://arxiv.org/abs/2207.05608#google`:
    [`“Inner Monologue: Embodied Reasoning through Planning With Language Models”`{=html}](https://arxiv.org/abs/2207.05608#google "'Inner Monologue: Embodied Reasoning through Planning with Language Models', Huang et al 2022"){#huang-et-al-2022-5
    .link-annotated},
    [Wenlong Huang, [Fei Xia](https://en.wikipedia.org/wiki/Fei_Xia "Fei Xia"){.id-not}, Ted Xiao[, Harris Chan, Jacky Liang, Pete Florence, [Andy Zeng](https://en.wikipedia.org/wiki/Andy_Zeng "Andy Zeng"){.id-not}, Jonathan Tompson, [Igor Mordatch](https://scholar.google.com/citations?user=Vzr1RukAAAAJ "Igor Mordatch"){.id-not}, Yevgen Chebotar, Pierre Sermanet, Noah Brown, Tomas Jackson, Linda Luu, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, [Karol Hausman](https://en.wikipedia.org/wiki/Karol_Hausman "Karol Hausman"){.id-not}, Brian Ichter]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2207.05608%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2207.05608#google"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

28. `https://arxiv.org/abs/2207.04429`:
    [`“LM-Nav: Robotic Navigation With Large Pre-Trained Models of Language, Vision, and Action”`{=html}](https://arxiv.org/abs/2207.04429 "'LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action', Shah et al 2022"){#shah-et-al-2022
    .link-annotated},
    [Dhruv Shah, Blazej Osinski, Brian Ichter, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2207.04429.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2207.04429"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

29. `https://arxiv.org/abs/2206.14176`:
    [`“DayDreamer: World Models for Physical Robot Learning”`{=html}](https://arxiv.org/abs/2206.14176 "'DayDreamer: World Models for Physical Robot Learning', Wu et al 2022"){#wu-et-al-2022-06
    .link-annotated},
    [Philipp Wu, Alejandro Escontrela, Danijar Hafner[, Ken Goldberg, [Pieter Abbeel](https://en.wikipedia.org/wiki/Pieter_Abbeel "Pieter Abbeel"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.14176.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.14176"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

30. `https://arxiv.org/abs/2206.11795#openai`:
    [`“Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos”`{=html}](https://arxiv.org/abs/2206.11795#openai "'Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos', Baker et al 2022"){#baker-et-al-2022-2
    .link-annotated},
    [Bowen Baker, Ilge Akkaya, Peter Zhokhov[, Joost Huizinga, [Jie Tang](https://en.wikipedia.org/wiki/Jie_Tang "Jie Tang"){.id-not}, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, [Jeff Clune](http://jeffclune.com/ "Jeff Clune—Professor—Computer Science—University of British Columbia"){.link-annotated-partial
    .id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.11795%2523openai.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.11795#openai"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

31. `https://arxiv.org/abs/2206.11309#microsoft`:
    [`“GODEL: Large-Scale Pre-Training for Goal-Directed Dialog”`{=html}](https://arxiv.org/abs/2206.11309#microsoft "'GODEL: Large-Scale Pre-Training for Goal-Directed Dialog', Peng et al 2022"){#peng-et-al-2022-2
    .link-annotated},
    [Baolin Peng, Michel Galley, Pengcheng He[, Chris Brockett, Lars Liden, Elnaz Nouri, [Zhou Yu](https://en.wikipedia.org/wiki/Zhou_Yu "Zhou Yu"){.id-not}, Bill Dolan, [Jianfeng Gao](https://www.microsoft.com/en-us/research/people/jfgao/ "Jianfeng Gao at Microsoft Research"){.link-annotated-partial
    .id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.11309%2523microsoft.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.11309#microsoft"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

32. `https://arxiv.org/abs/2206.04114#google`:
    [`“Director: Deep Hierarchical Planning from Pixels”`{=html}](https://arxiv.org/abs/2206.04114#google "'Director: Deep Hierarchical Planning from Pixels', Hafner et al 2022"){#hafner-et-al-2022
    .link-annotated},
    [Danijar Hafner, Kuang-Huei Lee, Ian Fischer, [Pieter Abbeel](https://en.wikipedia.org/wiki/Pieter_Abbeel "Pieter Abbeel"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2206.04114%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2206.04114#google"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

33. `https://arxiv.org/abs/2204.05080#deepmind`:
    [`“Semantic Exploration from Language Abstractions and Pretrained Representations”`{=html}](https://arxiv.org/abs/2204.05080#deepmind "'Semantic Exploration from Language Abstractions and Pretrained Representations', Tam et al 2022"){#tam-et-al-2022
    .link-annotated},
    [Allison C. Tam, Neil C. Rabinowitz, Andrew K. Lampinen[, Nicholas A. Roy, Stephanie C. Y. Chan, D. J. Strouse, Jane X. Wang, Andrea Banino, [Felix Hill](https://fh295.github.io/ "Language Understanding Grounded in Perception and Action"){.link-annotated-partial
    .id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2204.05080%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2204.05080#deepmind"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

34. `https://arxiv.org/abs/2204.01691#google`:
    [`“Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances”`{=html}](https://arxiv.org/abs/2204.01691#google "'Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances', Ahn et al 2022"){#ahn-et-al-2022
    .link-annotated},
    [Michael Ahn, Anthony Brohan, Noah Brown[, Yevgen Chebotar, [Omar Cortes](https://en.wikipedia.org/wiki/Omar_Cortes "Omar Cortes"){.id-not}, Byron David, [Chelsea Finn](https://en.wikipedia.org/wiki/Chelsea_Finn "Chelsea Finn"){.id-not}, Keerthana Gopalakrishnan, [Karol Hausman](https://en.wikipedia.org/wiki/Karol_Hausman "Karol Hausman"){.id-not}, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, [Eric Jang](https://evjang.com/about/){.link-annotated-partial
    .id-not}, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J. Joshi, [Ryan Julian](https://en.wikipedia.org/wiki/Ryan_Julian "Ryan Julian"){.id-not}, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, [Sergey Levine](https://scholar.google.com/citations?user=8R35rCwAAAAJ "Sergey Levine"){.id-not}, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, [Fei Xia](https://en.wikipedia.org/wiki/Fei_Xia "Fei Xia"){.id-not}, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2204.01691%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2204.01691#google"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

35. `https://arxiv.org/abs/2106.13281#google`:
    [`“Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation”`{=html}](https://arxiv.org/abs/2106.13281#google "'Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation', Freeman et al 2021"){#freeman-et-al-2021
    .link-annotated},
    [C. Daniel Freeman, Erik Frey, Anton Raichuk[, Sertan Girgin, [Igor Mordatch](https://scholar.google.com/citations?user=Vzr1RukAAAAJ "Igor Mordatch"){.id-not}, Olivier Bachem]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F2106.13281%2523google.html "Directory-tag link-bibliography for link https://arxiv.org/abs/2106.13281#google"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

36. `https://waymo.com/blog/2021/03/replaying-real-life/`:
    [`“Replaying Real Life: How the Waymo Driver Avoids Fatal Human Crashes”`{=html}](https://waymo.com/blog/2021/03/replaying-real-life/ "'Replaying real life: how the Waymo Driver avoids fatal human crashes', Waymo 2021"){#waymo-2021
    .link-annotated}, [Waymo]{.author .cite-author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fwaymo.com%252Fblog%252F2021%252F03%252Freplaying-real-life%252F.html "Directory-tag link-bibliography for link https://waymo.com/blog/2021/03/replaying-real-life/"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

37. `https://research.google/blog/introducing-dreamer-scalable-reinforcement-learning-using-world-models/`:
    [`“Introducing Dreamer: Scalable Reinforcement Learning Using World Models”`{=html}](https://research.google/blog/introducing-dreamer-scalable-reinforcement-learning-using-world-models/ "'Introducing Dreamer: Scalable Reinforcement Learning Using World Models', Hafner 2020"){#hafner-2020
    .link-annotated}, [Danijar Hafner]{.author .cite-author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Fresearch.google%252Fblog%252Fintroducing-dreamer-scalable-reinforcement-learning-using-world-models%252F.html "Directory-tag link-bibliography for link https://research.google/blog/introducing-dreamer-scalable-reinforcement-learning-using-world-models/"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

38. `https://arxiv.org/abs/1901.03559#deepmind`:
    [`“DRC: An Investigation of Model-Free Planning”`{=html}](https://arxiv.org/abs/1901.03559#deepmind "'DRC: An investigation of model-free planning', Guez et al 2019"){#guez-et-al-2019
    .link-annotated},
    [[Arthur Guez](https://scholar.google.com/citations?user=iyD9aw8AAAAJ "Arthur Guez"){.id-not}, [Mehdi Mirza](https://scholar.google.com/citations?user=c646VbAAAAAJ "Mehdi Mirza"){.id-not}, Karol Gregor[, Rishabh Kabra, Sébastien Racanière, Théophane Weber, David Raposo, [Adam Santoro](https://scholar.google.com/citations?user=evIkDWoAAAAJ "Adam Santoro"){.id-not}, Laurent Orseau, [Tom Eccles](https://en.wikipedia.org/wiki/Tom_Eccles "Tom Eccles"){.id-not}, Greg Wayne, [David Silver](https://en.wikipedia.org/wiki/David_Silver_(computer_scientist) "David Silver (programmer)"){.id-not}, [Timothy Lillicrap](https://en.wikipedia.org/wiki/Timothy_Lillicrap "Timothy Lillicrap"){.id-not}]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1901.03559%2523deepmind.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1901.03559#deepmind"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

39. `2018-gudmundsson.pdf`:
    [`“Human-Like Playtesting With Deep Learning”`{=html}](/doc/reinforcement-learning/imitation-learning/2018-gudmundsson.pdf "'Human-Like Playtesting with Deep Learning', Gudmundsson et al 2018"){#gudmundsson-et-al-2018
    .link-annotated},
    [Stefan Freyr Gudmundsson, Philipp Eisen, Erik Poromaa[, Alex Nodet, Sami Purmonen, Bartlomiej Kozakowski, Richard Meurling, Lele Cao]{.collapse}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fimitation-learning%252F2018-gudmundsson.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/imitation-learning/2018-gudmundsson.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

40. `2018-everitt.pdf`:
    [`“The Alignment Problem for Bayesian History-Based Reinforcement Learners”`{=html}](/doc/reinforcement-learning/model/2018-everitt.pdf "'The Alignment Problem for Bayesian History-Based Reinforcement Learners', Everitt & Hutter 2018"){#everitt-hutter-2018
    .link-annotated},
    [[Tom Everitt](https://www.tomeveritt.se/ "Tom Everitt"){.link-annotated-partial
    .id-not}, [Marcus Hutter](https://en.wikipedia.org/wiki/Marcus_Hutter "Marcus Hutter"){.id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fmodel%252F2018-everitt.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/model/2018-everitt.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

41. `https://arxiv.org/abs/1705.07615`:
    [`“AIXIjs: A Software Demo for General Reinforcement Learning”`{=html}](https://arxiv.org/abs/1705.07615 "'AIXIjs: A Software Demo for General Reinforcement Learning', Aslanides 2017"){#aslanides-2017
    .link-annotated}, [John Aslanides]{.author .cite-author}

    [[link-bibliography](/metadata/annotation/link-bibliography/https%253A%252F%252Farxiv.org%252Fabs%252F1705.07615.html "Directory-tag link-bibliography for link https://arxiv.org/abs/1705.07615"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

42. `2010-silver.pdf`:
    [`“POMCP: Monte-Carlo Planning in Large POMDPs”`{=html}](/doc/reinforcement-learning/model/2010-silver.pdf "'POMCP: Monte-Carlo Planning in Large POMDPs', Silver & Veness 2010"){#silver-veness-2010
    .link-annotated},
    [[David Silver](https://en.wikipedia.org/wiki/David_Silver_(computer_scientist) "David Silver (programmer)"){.id-not}, [Joel Veness](https://scholar.google.co.uk/citations?user=_iYrAxEAAAAJ "Joel Veness"){.link-annotated-partial
    .id-not}]{.author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fmodel%252F2010-silver.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/model/2010-silver.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}

43. `2001-cazenave.pdf`:
    [`“Iterative Widening”`{=html}](/doc/reinforcement-learning/model/2001-cazenave.pdf "'Iterative widening', Cazenave 2001"){#cazenave-2001
    .link-annotated}, [Tristan Cazenave]{.author .cite-author}

    [[link-bibliography](/metadata/annotation/link-bibliography/%252Fdoc%252Freinforcement-learning%252Fmodel%252F2001-cazenave.pdf.html "Directory-tag link-bibliography for link /doc/reinforcement-learning/model/2001-cazenave.pdf"){.id-not
    .include-even-when-collapsed}]{.collapse
    .tag-index-link-bibliography-block}