Bibliography (47):

  1. https://deepmind.google/

  2. https://www.reddit.com/r/mlscaling/comments/vq6qh1/demis_hassabis_gato_is_our_most_general_agent_so/ienfekn/

  3. Decision Transformer: Reinforcement Learning via Sequence Modeling

  4. The Arcade Learning Environment: An Evaluation Platform for General Agents

  5. A domain-specific supercomputer for training deep neural networks

  6. ​ scaling-hypothesis#blessings-of-scale

    [Transclude the forward-link's context]

  7. The Bitter Lesson

  8. Multi-task Deep Reinforcement Learning with PopArt

  9. https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/

  10. https://www.lesswrong.com/posts/5onEtjNEhqcfX3LXG/a-generalist-agent-new-deepmind-publication

  11. https://news.ycombinator.com/item?id=31355657

  12. https://x.com/ohlennart/status/1524877652867309570

  13. PaLM: Scaling Language Modeling with Pathways

  14. Chinchilla: Training Compute-Optimal Large Language Models

  15. DeepMind Lab

  16. Imagination-Augmented Agents for Deep Reinforcement Learning

  17. BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning

  18. https://github.com/google-deepmind/dm_control

  19. Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning

  20. Procgen Benchmark: We’re releasing Procgen Benchmark, 16 simple-to-use procedurally-generated environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills

  21. Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes

  22. One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control

  23. Offline Learning from Demonstrations and Unlabeled Experience

  24. Scaling Language Models: Methods, Analysis & Insights from Training Gopher

  25. ALIGN: Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision

  26. Microsoft COCO: Common Objects in Context

  27. Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning

  28. Flamingo: a Visual Language Model for Few-Shot Learning

  29. OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge

  30. VQA: Visual Question Answering

  31. T0: Multitask Prompted Training Enables Zero-Shot Task Generalization

  32. FLAN: Finetuned Language Models Are Zero-Shot Learners

  33. Progressive Neural Networks

  34. Benchmarking End-to-End Behavioral Cloning on Video Games

  35. GPT-3: Language Models are Few-Shot Learners

  36. Scaling Laws for Autoregressive Generative Modeling

  37. Image GPT (iGPT): We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples

  38. How AI Training Scales

  39. WebGPT: Browser-assisted question-answering with human feedback

  40. Perceiver: General Perception with Iterative Attention

  41. Open-Ended Learning Leads to Generally Capable Agents

  42. From Motor Control to Team Play in Simulated Humanoid Football

  43. Scaling Laws for Transfer

  44. Scaling Laws for Neural Language Models

  45. Wikipedia Bibliography:

    1. Neural scaling lawβ€‰β€Š:

      https://en.wikipedia.org/wiki/Neural_scaling_law

    2. Reinforcement learning

    3. Loss function