“Learning Humanoid Locomotion With Transformers”, Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, Koushil Sreenath2023-03-06 (knowledge distillation, Decision Transformer, robotics)⁠:

We present a sim-to-real learning-based approach for real-world humanoid locomotion.

Our controller is a causal Transformer trained by autoregressive prediction of future actions from the history of observations and actions.

We hypothesize that the observation-action history contains useful information about the world that a powerful Transformer model can use to adapt its behavior in-context, without updating its weights. We do not use state estimation, dynamics models, trajectory optimization, reference trajectories, or pre-computed gait libraries.

Our controller is trained with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed to the real world in a zero-shot fashion.

We evaluate our approach in high-fidelity simulation and successfully deploy it to the real robot [a 45kg 1.6m tall ‘Digit’ humanoid robot by Agility Robotics] as well. To the best of our knowledge, this is the first demonstration of a fully learning-based method for real-world full-sized humanoid locomotion.