“Meta-Trained Agents Implement Bayes-Optimal Agents”, 2020-10-21 (; similar):
Memory-based meta-learning is a powerful technique to build agents that adapt fast to any task within a target distribution. A previous theoretical study has argued that this remarkable performance is because the meta-training protocol incentivizes agents to behave Bayes-optimally.
We empirically investigate this claim on a number of prediction and bandit tasks.
Inspired by ideas from theoretical computer science, we show that meta-learned and Bayes-optimal agents not only behave alike, but they even share a similar computational structure, in the sense that one agent system can simulate the other. Furthermore, we show that Bayes-optimal agents are fixed points of the meta-learning dynamics.
Our results suggest that memory-based meta-learning might serve as a general technique for numerically approximating Bayes-optimal agents—that is, even for task distributions for which we currently don’t possess tractable models.