# Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving

**Authors:** Lin-Chi Wu, Zengjie Zhang, Sofie Haesaert, Zhiqiang Ma, Zhiyong Sun  
**arXiv:** [2306.03220](https://arxiv.org/abs/2306.03220)  
**Date:** 2023-06-05  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2306.03220v2/

## Key points

- The paper focuses on using reinforcement learning (RL) in motion planning for autonomous driving.
- Existing approaches struggle to determine a suitable reward function for RL agents that balances safe and risky behaviors.
- Risk-aware reward shaping is introduced to improve training and testing performance of RL agents by promoting exploration and discouraging risky actions.
- Safety specifications are incorporated into the reward function to guide agents towards safer driving behaviors, such as collision avoidance.
- Experimental studies show that risk-aware reward shaping, particularly when combined with proximal policy optimization (PPO), enhances the performance of RL agents in autonomous driving scenarios.

## Summary

The paper "Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving" delves into the use of reinforcement learning (RL) in motion planning for autonomous driving. The focus is on developing an optimal driving policy through interactions with the environment. However, determining a suitable reward function for RL agents poses a challenge as existing approaches prioritize safe driving states without considering risky behaviors. To address this limitation, the study introduces risk-aware reward shaping to enhance the training and testing performance of RL agents in autonomous driving scenarios. By incorporating safety specifications and reshaping reward terms to promote exploration and discourage risky driving actions, the proposed method aims to improve overall agent behavior. The research outlines essential principles for reward shaping in autonomous driving, emphasizing the importance of training RL agents to navigate tracks without collisions with obstacles. By encoding safety requirements such as collision avoidance into the reward function, the study guides agents towards safer driving behaviors. Experimental studies using OpenAI Gym demonstrate the advantages of risk-aware reward shaping for various RL agents. The results suggest that proximal policy optimization (PPO) is particularly effective when combined with this approach. Overall, the paper provides valuable insights into leveraging risk-aware reward shaping to enhance the performance and safety of RL agents in autonomous driving applications. By integrating awareness of potential risks into the training process, this method offers a promising avenue for improving autonomous vehicle behavior.