# Perceive With Confidence: Statistical Safety Assurances for Navigation with Learning-Based Perception

**Authors:** Anushri Dixit, Zhiting Mei, Meghan Booker, Mariko Storey-Matsutani, Allen Z. Ren, Anirudha Majumdar  
**arXiv:** [2403.08185](https://arxiv.org/abs/2403.08185)  
**Date:** 2024-03-13  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2403.08185v1/

## Key points

- Framework for quantifying uncertainty of pre-trained perception models
- Approach involves producing a calibrated perception system for statistical safety assurances in navigation tasks
- Evaluation through experiments in simulation and on hardware using a quadruped robot navigating indoor environments with unseen objects
- Significant improvements in empirical safety rates compared to baseline approaches
- Perceive with Confidence (PwC) framework provides statistical assurance for generalization and adaptation to changes in state distributions during deployment
- Challenges include the need for an a priori bound on environment dimensions and potential conservatism from bounding box inflation
- Future work may explore alternative occupancy representations beyond bounding boxes, such as voxel-wise occupancy confidences
- Constructing non-conformity score functions incorporating confidences from pre-trained models could reduce conservatism

## Summary

The authors present a framework for quantifying the uncertainty of pre-trained perception models to provide statistical safety assurances for navigation tasks. The approach involves producing a calibrated perception system that processes the outputs of a pre-trained model and ensures generalization and robustness when used with a planner. This framework, known as Perceive with Confidence (PwC), was evaluated through experiments in simulation and on hardware using a quadruped robot navigating through indoor environments with unseen objects. Results demonstrate significant improvements in empirical safety rates compared to baseline approaches. PwC provides statistical assurance for generalization to novel environmental factors and allows for adaptation to changes in state distributions during deployment. Challenges identified include the need for an a priori bound on the outermost dimensions of the environment for calibration purposes and potential conservatism introduced by bounding box inflation from conformal prediction. To address these challenges, future work may explore alternative occupancy representations beyond bounding boxes, such as voxel-wise occupancy confidences generated by networks for scene completion. Additionally, constructing non-conformity score functions that incorporate confidences from pre-trained models could potentially reduce conservatism in the perception system.