# Towards on-sky adaptive optics control using reinforcement learning

**Authors:** J. Nousiainen, C. Rajani, M. Kasper, T. Helin, S. Y. Haffert, C. Vérinaud, J. R. Males, K. Van Gorkom, L. M. Close, J. D. Long, A. D. Hedglen, O. Guyon, L. Schatz, M. Kautz, J. Lumbres, A. Rodack, J. M. Knight, K. Miller  
**arXiv:** [2205.07554](https://arxiv.org/abs/2205.07554)  
**Date:** 2022-05-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2205.07554v1/

## Key points

- Direct imaging of potentially habitable exoplanets is a crucial scientific objective for future high contrast imaging instruments on ground-based telescopes.
- eXtreme Adaptive Optics (XAO) systems with thousands of actuators are used to achieve this goal.
- Current XAO systems' control laws have limitations when imaging habitable exoplanets at small angular separations from their host stars.
- The study introduces a new method called PO4AO to improve adaptive optics correction.
- PO4AO learns a dynamics model and optimizes a control neural network known as a policy, building upon previous work in Reinforcement Learning for AO.
- The method is evaluated through numerical simulations and laboratory experiments using different telescope aperture cases.
- PO4AO significantly improves coronagraphic contrast in both simulations and experiments, achieving contrast improvements by factors of 3-5 within the control region of deformable mirrors (DM) and Pyramid wavefront sensors (WFS).
- PO4AO has fast training timescales of 5-10 seconds and low inference time (< ms), making it suitable for real-time control on large telescopes.
- The study presents an innovative approach to adaptive optics control using reinforcement learning techniques.
- By reducing residual flux in the coronagraphic point spread function, PO4AO holds promise for advancing direct imaging capabilities and understanding potentially habitable exoplanets.

## Summary

The direct imaging of potentially habitable exoplanets is a crucial scientific objective for the next generation of high contrast imaging instruments on ground-based extremely large telescopes. To achieve this goal, these instruments are equipped with eXtreme Adaptive Optics (XAO) systems that control thousands of actuators at high frame rates. However, the current XAO systems' control laws leave strong residuals when imaging habitable exoplanets located at small angular separations from their host stars. This study aims to address these limitations and improve the adaptive optics correction by introducing a new method called PO4AO. Building upon previous work in Reinforcement Learning for AO, PO4AO learns a dynamics model and optimizes a control neural network known as a policy. The method is evaluated through numerical simulations of XAO with Pyramid wavefront sensing for different telescope aperture cases, including 8-m and 40-m telescopes. Additionally, experiments are conducted in a laboratory environment using MagAO-X at the Steward laboratory. The results demonstrate that PO4AO significantly improves coronagraphic contrast in both numerical simulations and laboratory experiments. In simulation, the method achieves contrast improvements by factors of 3-5 within the control region of deformable mirrors (DM) and Pyramid wavefront sensors (WFS). Moreover, PO4AO exhibits fast training timescales of typically 5-10 seconds and has low inference time (< ms), making it suitable for real-time control even on extremely large telescopes. Overall, this study presents an innovative approach to adaptive optics control using reinforcement learning techniques. By effectively reducing residual flux in the coronagraphic point spread function, PO4AO holds great promise for advancing direct imaging capabilities and enhancing our understanding of potentially habitable exoplanets.