“Meta Reinforcement Learning”, Lilian Weng2019-06-23 (RL exploration, meta-learning, RL scaling; similar)⁠:

[Review/discussion] Meta-RL is meta-learning on reinforcement learning tasks. After trained over a distribution of tasks, the agent is able to solve a new task by developing a new RL algorithm with its internal activity dynamics. This post starts with the origin of meta-RL and then dives into three key components of meta-RL…, a good meta-learning model is expected to generalize to new tasks or new environments that have never been encountered during training. The adaptation process, essentially a mini learning session, happens at test with limited exposure to the new configurations. Even without any explicit fine-tuning (no gradient backpropagation on trainable variables), the meta-learning model autonomously adjusts internal hidden states to learn.

• On the Origin of Meta-RL
  • Back in 2001
  • Proposal in 2016
• Define Meta-RL
  • Formulation
  • Main Differences from RL
  • Key Components
• Meta-Learning Algorithms for Meta-RL
  • Optimizing Model Weights for Meta-learning
  • Meta-learning Hyperparameters
  • Meta-learning the Loss Function
  • Meta-learning the Exploration Strategies
  • Episodic Control
• Training Task Acquisition
  • Task Generation by Domain Randomization
  • Evolutionary Algorithm on Environment Generation
  • Learning with Random Rewards
• References

Similar Links: