# Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability

**Authors:** Zhihao Li, Toshiyuki Motoyoshi, Kazuma Sasaki, Tetsuya Ogata, Shigeki Sugano  
**arXiv:** [1809.11100](https://arxiv.org/abs/1809.11100)  
**Date:** 2018-09-28  
**Summary page:** https://www.summarizepaper.com/en/paper/1809.11100v1/

## Key points

- Authors address two major problems with current end-to-end deep learning driving models: poor generalization ability and lack of accident explanation capability
- Leveraging knowledge from associated easy tasks can be beneficial in addressing difficult tasks
- Proposed model is trained stepwise with multi-task perception-related basic knowledge and driving knowledge
- Segmentation maps and depth maps are used as sources of "what & where" and "how far" knowledge
- Experimental results demonstrate the effectiveness of multi-task perception knowledge in improving generalization ability and accident explanation capability
- Average success rate of completing most difficult navigation tasks surpassed current benchmark method by 15% in trained weather conditions and 20% in untrained weathers
- Demonstration video showcasing their approach can be found at https://www.youtube.com/watch?v=N7ePnnZZwdE

## Summary

In the paper titled "Rethinking Self-driving: Multi-task Knowledge for Better Generalization and Accident Explanation Ability," authors Zhihao Li, Toshiyuki Motoyoshi, Kazuma Sasaki, Tetsuya Ogata, and Shigeki Sugano address two major problems with current end-to-end deep learning driving models. The first problem is the poor generalization ability of these models in unobserved driving environments when the diversity of the training driving dataset is limited. The second problem is the lack of accident explanation ability when these models fail to perform as expected. They believe that leveraging knowledge from associated easy tasks can be beneficial in addressing difficult tasks. The proposed model is trained stepwise with multi-task perception-related basic knowledge and driving knowledge. The authors specifically consider segmentation maps and depth maps as sources of "what & where" and "how far" knowledge for tackling easier driving-related perception problems before generating final control commands for more challenging driving tasks. Experimental results demonstrate the effectiveness of multi-task perception knowledge in improving generalization ability and accident explanation capability. Using their method, the average success rate of completing the most difficult navigation tasks in an untrained city during CoRL tests surpassed the current benchmark method by 15% in trained weather conditions and 20% in untrained weathers. A demonstration video showcasing their approach can be found at https://www.youtube.com/watch?v=N7ePnnZZwdE. Overall, this paper presents a novel approach to self-driving using multi-task knowledge that addresses key limitations of existing deep learning models. The results highlight improved generalization ability and accident explanation capability making it a promising contribution to autonomous vehicle research.