# GoalsEye: Learning High Speed Precision Table Tennis on a Physical Robot

**Authors:** Tianli Ding, Laura Graesser, Saminda Abeyruwan, David B. D'Ambrosio, Anish Shankar, Pierre Sermanet, Pannag R. Sanketi, Corey Lynch  
**arXiv:** [2210.03662](https://arxiv.org/abs/2210.03662)  
**Date:** 2022-10-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.03662v2/

## Key points

- Learning goal conditioned control in the real world is a challenging problem in robotics
- Reinforcement learning systems have potential but are often too costly for real-world deployment
- Imitation learning approaches require curated demonstration data and lack continuous improvement mechanisms
- Iterative imitation techniques can learn goal-directed control from undirected demonstration data and improve continuously via self-supervised goal reaching
- Results so far have been limited to simulated environments, but this study shows iterative imitation learning can scale to goal-directed behavior on a real robot in high-speed precision table tennis
- The approach offers a straightforward way to do continuous on-robot learning without complexities such as reward design or sim-to-real transfer, and is sample efficient enough to train on a physical robot in just a few hours
- The resulting policy can perform on par or better than amateur humans at the task of returning the ball to specific targets on the table, with an improvement of 3.4% for balls landed within 30cm and 3.6% for balls landed within 20cm over average human performance
- The study demonstrates that iterative imitation learning can continuously improve in the real world beyond an initial undirected bootstrap dataset, sidestepping the complexities of reinforcement learning (e.g., exploration, reward shaping and sim-to-real transfer) and excel at dynamic tasks requiring precision

## Summary

Learning goal conditioned control in the real world is a challenging open problem in robotics. Reinforcement learning systems have the potential to learn autonomously via trial-and-error, but in practice, the costs of manual reward design, ensuring safe exploration and hyperparameter tuning are often enough to preclude real-world deployment. Imitation learning approaches offer a simple way to learn control in the real world but typically require costly curated demonstration data and lack a mechanism for continuous improvement. Recently, iterative imitation techniques have been shown to learn goal-directed control from undirected demonstration data and improve continuously via self-supervised goal reaching. However, results so far have been limited to simulated environments. In this work, researchers present evidence that iterative imitation learning can scale to goal-directed behavior on a real robot in a dynamic setting: high-speed precision table tennis. The approach offers a straightforward way to do continuous on-robot learning without complexities such as reward design or sim-to-real transfer. It is also scalable - sample efficient enough to train on a physical robot in just a few hours. The study finds that the resulting policy can perform on par or better than amateur humans at the task of returning the ball to specific targets on the table. Despite not reaching advanced amateur human performance levels, GoalsEye obtains an improvement of 3.4% for balls landed within 30cm and 3.6% for balls landed within 20cm over average human performance. The experiments showcase the sample efficiency of this approach over reinforcement learning methods and highlight the benefits of iterative self-supervised improvement over pure imitation learning methods. The study demonstrates that iterative imitation learning can continuously improve in the real world beyond an initial undirected bootstrap dataset, sidestepping the complexities of reinforcement learning (e.g., exploration, reward shaping and sim-to-real transfer) and excel at dynamic tasks requiring precision. Overall, this research provides valuable insights into how machine-learning based systems can be trained efficiently for complex real world tasks with significant implications for autonomous systems that can learn and improve continuously in dynamic environments without requiring extensive human intervention.