# Gaze-Informed Multi-Objective Imitation Learning from Human Demonstrations

**Authors:** Ritwik Bera, Vinicius G. Goecks, Gregory M. Gremillion, Vernon J. Lawhern, John Valasek, Nicholas R. Waytowich  
**arXiv:** [2102.13008](https://arxiv.org/abs/2102.13008)  
**Date:** 2021-02-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2102.13008v1/

## Key points

- Teaching learning agents from human demonstrations via supervised learning is widely explored in human-robot interaction.
- The study introduces a novel approach that incorporates eye gaze information to enhance agent performance.
- The proposed imitation learning architecture aims to improve task completion rates and optimize path efficiency by leveraging insights into where the demonstrator directs their visual attention.
- A Gaze-Informed Multi-Objective Imitation Learning framework is introduced, which learns from human action demonstrations and eye tracking data concurrently.
- The approach is designed for tasks where human gaze information provides contextual cues for effective decision-making.
- Tested in a visual navigation scenario with an unmanned quadrotor, the model achieves significantly higher task completion rates and generates more efficient navigation paths compared to a baseline model.
- The model demonstrates an ability to predict human visual attention patterns, showcasing multimodal learning capabilities from additional human input modalities.
- Emphasizing the importance of integrating visual attention information into agent training processes encourages adoption of such approaches in training agents for visuomotor tasks.

## Summary

In the field of human-robot interaction, teaching learning agents from human demonstrations via supervised learning has been widely explored and applied across various domains such as self-driving cars and robot manipulation. This study introduces a novel approach that incorporates eye gaze information to enhance agent performance. By leveraging insights into where the demonstrator directs their visual attention, the proposed imitation learning architecture aims to improve task completion rates and optimize path efficiency. Specifically, this work introduces a Gaze-Informed Multi-Objective Imitation Learning framework that concurrently learns from human action demonstrations and eye tracking data. This approach is designed to tackle tasks where human gaze information plays a crucial role in providing contextual cues for effective decision-making. The methodology is put to the test in a visual navigation scenario, where an unmanned quadrotor is trained to locate and navigate towards a target vehicle within a realistic simulated environment. Comparative analysis against a baseline imitation learning model reveals that the proposed gaze-augmented architecture achieves significantly higher task completion rates while generating more efficient navigation paths. Moreover, the model demonstrates an ability to predict human visual attention patterns, showcasing its capacity for multimodal learning from additional human input modalities. By emphasizing the importance of integrating visual attention information into agent training processes, this research encourages the adoption of such approaches in training agents for visuomotor tasks. Authored by Ritwik Bera, Vinicius G. Goecks, Gregory M. Gremillion, Vernon J. Lawhern, John Valasek, and Nicholas R. Waytowich, this study titled "Gaze-Informed Multi-Objective Imitation Learning from Human Demonstrations" underscores the significance of incorporating eye gaze data in enhancing agent performance during human-robot interactions.