“‘RL Scaling’ Tag”,2019-09-09 (AI scaling, meta-learning)⁠:

Bibliography for tag reinforcement-learning/scaling, most recent first: 12 related tags, 142 annotations, & 21 links (parent).

[page summary]

• See Also
• Gwern
  • “Research Ideas”, Gwern2017
  • “It Looks Like You’re Trying To Take Over The World”, Gwern2022
  • “The Scaling Hypothesis”, Gwern2020
  • “Why Tool AIs Want to Be Agent AIs”, Gwern2016
• Links
  • “Data Scaling Laws in Imitation Learning for Robotic Manipulation”, Lin et al 2024
  • “AI Alignment via Slow Substrates: Early Empirical Results With StarCraft II”, Leong2024
  • “MLE-Bench: Evaluating Machine Learning Agents on Machine Learning Engineering”, Chan et al 2024
  • “NAVIX: Scaling MiniGrid Environments With JAX”, Pignatelli et al 2024
  • “JEST: Data Curation via Joint Example Selection Further Accelerates Multimodal Learning”, Evans et al 2024
  • “AI Search: The Bitter-Er Lesson”, McLaughlin2024
  • “Foundational Challenges in Assuring Alignment and Safety of Large Language Models”, Anwar et al 2024
  • “Simple and Scalable Strategies to Continually Pre-Train Large Language Models”, Ibrahim et al 2024
  • “Robust Agents Learn Causal World Models”, Richens & Everitt2024
  • “Grandmaster-Level Chess Without Search”, Ruoss et al 2024
  • “Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training”, Hubinger et al 2024
  • “Vision-Language Models As a Source of Rewards”, Baumli et al 2023
  • “JaxMARL: Multi-Agent RL Environments in JAX”, Rutherford et al 2023
  • “Diversifying AI: Towards Creative Chess With AlphaZero (AZ_db)”, Zahavy et al 2023
  • “Deep RL at Scale: Sorting Waste in Office Buildings With a Fleet of Mobile Manipulators”, Herzog et al 2023
  • “Emergence of Belief-Like Representations through Reinforcement Learning”, Hennig et al 2023
  • “Scaling Laws for Single-Agent Reinforcement Learning”, Hilton et al 2023
  • “DreamerV3: Mastering Diverse Domains through World Models”, Hafner et al 2023
  • “Offline Q-Learning on Diverse Multi-Task Data Both Scales And Generalizes”, Kumar et al 2022
  • “VeLO: Training Versatile Learned Optimizers by Scaling Up”, Metz et al 2022
  • “Broken Neural Scaling Laws”, Caballero et al 2022
  • “Scaling Laws for Reward Model Overoptimization”, Gao et al 2022
  • “SAP: Bidirectional Language Models Are Also Few-Shot Learners”, Patel et al 2022
  • “g.pt: Learning to Learn With Generative Models of Neural Network Checkpoints”, Peebles et al 2022
  • “Human-Level Atari 200× Faster”, Kapturowski et al 2022
  • “Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, Ganguli et al 2022
  • “AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Model”, Soltan et al 2022
  • “TextWorldExpress: Simulating Text Games at One Million Steps Per Second”, Jansen & Côté2022
  • “Demis Hassabis: DeepMind—AI, Superintelligence & the Future of Humanity § Turing Test”, Hassabis & Fridman2022
  • “Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos”, Baker et al 2022
  • “Multi-Game Decision Transformers”, Lee et al 2022
  • “Task-Agnostic Continual Reinforcement Learning: In Praise of a Simple Baseline (3RL)”, Caccia et al 2022
  • “CT0: Fine-Tuned Language Models Are Continual Learners”, Scialom et al 2022
  • “Flexible Diffusion Modeling of Long Videos”, Harvey et al 2022
  • “Instruction Induction: From Few Examples to Natural Language Task Descriptions”, Honovich et al 2022
  • “Gato: A Generalist Agent”, Reed et al 2022
  • “Data Distributional Properties Drive Emergent Few-Shot Learning in Transformers”, Chan et al 2022
  • “Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale”, Ramrakhya et al 2022
  • “Do As I Can, Not As I Say (SayCan): Grounding Language in Robotic Affordances”, Ahn et al 2022
  • “Socratic Models: Composing Zero-Shot Multimodal Reasoning With Language”, Zeng et al 2022
  • “InstructGPT: Training Language Models to Follow Instructions With Human Feedback”, Ouyang et al 2022
  • “A Data-Driven Approach for Learning to Control Computers”, Humphreys et al 2022
  • “EvoJAX: Hardware-Accelerated Neuroevolution”, Tang et al 2022
  • “Accelerated Quality-Diversity for Robotics through Massive Parallelism”, Lim et al 2022
  • “Don’t Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning (ExORL)”, Yarats et al 2022
  • “Can Wikipedia Help Offline Reinforcement Learning?”, Reid et al 2022
  • “In Defense of the Unitary Scalarization for Deep Multi-Task Learning”, Kurin et al 2022
  • “The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models”, Pan et al 2022
  • “WebGPT: Browser-Assisted Question-Answering With Human Feedback”, Nakano et al 2021
  • “WebGPT: Improving the Factual Accuracy of Language Models through Web Browsing”, Hilton et al 2021
  • “Acquisition of Chess Knowledge in AlphaZero”, McGrath et al 2021
  • “AW-Opt: Learning Robotic Skills With Imitation and Reinforcement at Scale”, Lu et al 2021
  • “An Explanation of In-Context Learning As Implicit Bayesian Inference”, Xie et al 2021
  • “Procedural Generalization by Planning With Self-Supervised World Models”, Anand et al 2021
  • “MetaICL: Learning to Learn In Context”, Min et al 2021
  • “Collaborating With Humans without Human Data”, Strouse et al 2021
  • “T0: Multitask Prompted Training Enables Zero-Shot Task Generalization”, Sanh et al 2021
  • “Bridge Data: Boosting Generalization of Robotic Skills With Cross-Domain Datasets”, Ebert et al 2021
  • “Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning”, Rudin et al 2021
  • “Recursively Summarizing Books With Human Feedback”, Wu et al 2021
  • “FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei et al 2021
  • “Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation”, Nair et al 2021
  • “WarpDrive: Extremely Fast End-To-End Deep Multi-Agent Reinforcement Learning on a GPU”, Lan et al 2021
  • “Multi-Task Self-Training for Learning General Representations”, Ghiasi et al 2021
  • “Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning”, Makoviychuk et al 2021
  • “Open-Ended Learning Leads to Generally Capable Agents”, Team et al 2021
  • “Megaverse: Simulating Embodied Agents at One Million Experiences per Second”, Petrenko et al 2021
  • “Evaluating Large Language Models Trained on Code”, Chen et al 2021
  • “PES: Unbiased Gradient Estimation in Unrolled Computation Graphs With Persistent Evolution Strategies”, Vicol et al 2021
  • “Multimodal Few-Shot Learning With Frozen Language Models”, Tsimpoukelli et al 2021
  • “Brax—A Differentiable Physics Engine for Large Scale Rigid Body Simulation”, Freeman et al 2021
  • “PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World”, Zellers et al 2021
  • “From Motor Control to Team Play in Simulated Humanoid Football”, Liu et al 2021
  • “Reward Is Enough”, Silver et al 2021
  • “Podracer Architectures for Scalable Reinforcement Learning”, Hessel et al 2021
  • “MuZero Unplugged: Online and Offline Reinforcement Learning by Planning With a Learned Model”, Schrittwieser et al 2021
  • “Scaling Scaling Laws With Board Games”, Jones2021
  • “Large Batch Simulation for Deep Reinforcement Learning”, Shacklett et al 2021
  • “Stockfish and Lc0, Test at Different Number of Nodes”, Meloni2021
  • “Training Larger Networks for Deep Reinforcement Learning”, Ota et al 2021
  • “Investment vs. Reward in a Competitive Knapsack Problem”, Neumann & Gros2021
  • “NNUE: The Neural Network of the Stockfish Chess Engine”, Goucher2021
  • “Imitating Interactive Intelligence”, Abramson et al 2020
  • “Scaling down Deep Learning”, Greydanus2020
  • “Understanding RL Vision: With Diverse Environments, We Can Analyze, Diagnose and Edit Deep Reinforcement Learning Models Using Attribution”, Hilton et al 2020
  • “Meta-Trained Agents Implement Bayes-Optimal Agents”, Mikulik et al 2020
  • “Measuring Progress in Deep Reinforcement Learning Sample Efficiency”, Anonymous2020
  • “Learning to Summarize from Human Feedback”, Stiennon et al 2020
  • “Measuring Hardware Overhang”, hippke 2020
  • “Sample Factory: Egocentric 3D Control from Pixels at 100,000 FPS With Asynchronous Reinforcement Learning”, Petrenko et al 2020
  • “Real World Games Look Like Spinning Tops”, Czarnecki et al 2020
  • “Agent57: Outperforming the Human Atari Benchmark”, Puigdomènech et al 2020
  • “Deep Neuroethology of a Virtual Rodent”, Merel et al 2020
  • “Near-Perfect Point-Goal Navigation from 2.5 Billion Frames of Experience”, Wijmans & Kadian2020
  • “Procgen Benchmark: We’re Releasing Procgen Benchmark, 16 Simple-To-Use Procedurally-Generated Environments Which Provide a Direct Measure of How Quickly a Reinforcement Learning Agent Learns Generalizable Skills”, Cobbe et al 2019
  • “DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames”, Wijmans et al 2019
  • “Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning”, Vinyals et al 2019
  • “Solving Rubik’s Cube With a Robot Hand”, OpenAI et al 2019
  • “Fine-Tuning Language Models from Human Preferences”, Ziegler et al 2019
  • “Emergent Tool Use from Multi-Agent Interaction § Surprising Behavior”, Baker et al 2019
  • “Emergent Tool Use from Multi-Agent Interaction § Surprising Behavior”, Baker et al 2019
  • “Meta Reinforcement Learning”, Weng2019
  • “Human-Level Performance in 3D Multiplayer Games With Population-Based Reinforcement Learning”, Jaderberg et al 2019
  • “AI-GAs: AI-Generating Algorithms, an Alternate Paradigm for Producing General Artificial Intelligence”, Clune2019
  • “Meta-Learning of Sequential Strategies”, Ortega et al 2019
  • “Habitat: A Platform for Embodied AI Research”, Savva et al 2019
  • “The Bitter Lesson”, Sutton2019
  • “Benchmarking Classic and Learned Navigation in Complex 3D Environments”, Mishkin et al 2019
  • “Dota 2 With Large Scale Deep Reinforcement Learning: §4.3: Batch Size”, Berner2019 (page 13)
  • “An Empirical Model of Large-Batch Training”, McCandlish et al 2018
  • “How AI Training Scales”, McCandlish et al 2018
  • “Bayesian Layers: A Module for Neural Network Uncertainty”, Tran et al 2018
  • “Quantifying Generalization in Reinforcement Learning”, Cobbe et al 2018
  • “One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets With RL”, Paine et al 2018
  • “Robot Learning in Homes: Improving Generalization and Reducing Dataset Bias”, Gupta et al 2018
  • “Human-Level Performance in First-Person Multiplayer Games With Population-Based Deep Reinforcement Learning”, Jaderberg et al 2018
  • “QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation”, Kalashnikov et al 2018
  • “Playing Atari With Six Neurons”, Cuccu et al 2018
  • “AI and Compute”, Amodei et al 2018
  • “Accelerated Methods for Deep Reinforcement Learning”, Stooke & Abbeel2018
  • “One Big Net For Everything”, Schmidhuber2018
  • “Interactive Grounded Language Acquisition and Generalization in a 2D World”, Yu et al 2018
  • “Emergence of Locomotion Behaviors in Rich Environments”, Heess et al 2017
  • “Deep Reinforcement Learning from Human Preferences”, Christiano et al 2017
  • “Evolution Strategies As a Scalable Alternative to Reinforcement Learning”, Salimans et al 2017
  • “On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models”, Schmidhuber2015
  • “Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning”, Parisotto et al 2015
  • “Gorila: Massively Parallel Methods for Deep Reinforcement Learning”, Nair et al 2015
  • “Algorithmic Progress in Six Domains”, Grace2013
  • “Robot Predictions Evolution”, Moravec2004
  • “When Will Computer Hardware Match the Human Brain?”, Moravec1998
  • “Human Window on the World”, Michie1985
  • “Time for AI to Cross the Human Performance Range in Chess”
  • “Eric Jang”
  • “Trading Off Compute in Training and Inference”
  • “Trading Off Compute in Training and Inference § MCTS Scaling”
  • “Submission #6347: Chef Stef’s NES Arkanoid warpless in 11:11.18”
  • “[The Addictiveness & Adversarialness of Playing against LeelaQueenOdds]”, Järviniemi2024
  • “Training a CUDA TDS Ant Using C++ ARS Linear Policy: The Video Is Real-Time, After a Few Minutes (in the 30 Million Steps) the Training Curve Is Flat (I Trained Until a Billion Steps). Note That This Ant Is PD Control, and Not Identical to Either MuJoCo or PyBullet Ant, so the Training Curves Are Not Comparable Yet. Will Fix That.”
  • “Ilya Sutskever: Deep Learning | AI Podcast #94 With Lex Fridman”
  • “Target-Driven Visual Navigation in Indoor Scenes Using Deep Reinforcement Learning [Video]”
  • “If You Want to Solve a Hard Problem in Reinforcement Learning, You Just Scale. It’s Just Gonna Work Just like Supervised Learning. It’s the Same, the Same Story Exactly. It Was Kind of Hard to Believe That Supervised Learning Can Do All Those Things, but It’s Not Just Vision, It’s Everything and the Same Thing Seems to Hold for Reinforcement Learning Provided You Have a Lot of Experience.”
  • Wikipedia
• Miscellaneous
• Bibliography