# Learning Robust Policies via Interpretable Hamilton-Jacobi Reachability-Guided Disturbances

**Authors:** Hanyang Hu, Xilun Zhang, Xubo Lyu, Mo Chen  
**arXiv:** [2409.19746](https://arxiv.org/abs/2409.19746)  
**Date:** 2024-09-29  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2409.19746v1/

## Key points

- Deep Reinforcement Learning (RL) successful in navigating complex dynamics
- Challenge: susceptibility to unknown disturbances and adversarial attacks
- Innovative framework integrates model-based control with adversarial RL training
- Utilization of Hamilton-Jacobi reachability-guided disturbances for training
- Aim: fortify system against unforeseen challenges, enhance robustness
- Evaluation across three tasks: reach-avoid game in simulation and real-world, quadrotor stabilization task in virtual setting
- Learned critic network aligns closely with ground-truth HJ value function
- Policy network performance comparable to other learning-based methods
- Promising avenue for enhancing resilience of deep RL systems in robotics

## Summary

In the field of robotics, Deep Reinforcement Learning (RL) has proven to be highly successful in navigating complex and diverse dynamics. However, a persistent challenge lies in its susceptibility to unknown disturbances and adversarial attacks. To address this issue, an innovative framework has been developed that integrates model-based control principles with adversarial RL training. This approach aims to enhance robustness without relying on external black-box adversaries. The key concept introduced is the utilization of Hamilton-Jacobi reachability-guided disturbances for adversarial RL training. By employing interpretable worst-case or near-worst-case disturbances as adversaries against the robust policy, the researchers aim to fortify the system against unforeseen challenges. This advancement holds significant potential for bolstering the adaptability and reliability of robotic systems operating in dynamic and uncertain environments. The effectiveness of this approach was evaluated across three distinct tasks: a reach-avoid game conducted in both simulation and real-world environments, as well as a dynamic quadrotor stabilization task simulated in a virtual setting. Through rigorous testing and analysis, it was confirmed that the learned critic network aligns closely with the ground-truth HJ value function. Furthermore, the policy network demonstrated performance levels comparable to other learning-based methods. Overall, this research showcases a promising avenue for enhancing the resilience of deep RL systems in robotics by leveraging interpretable disturbances guided by Hamilton-Jacobi reachability principles.