# DeepThermal: Combustion Optimization for Thermal Power Generating Units Using Offline Reinforcement Learning

**Authors:** Xianyuan Zhan, Haoran Xu, Yue Zhang, Xiangyu Zhu, Honglei Yin, Yu Zheng  
**arXiv:** [2102.11492](https://arxiv.org/abs/2102.11492)  
**Date:** 2021-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2102.11492v3/

## Key points

- DeepThermal is a data-driven AI system developed to optimize the combustion control strategy for thermal power generating units (TPGUs).
- The core of DeepThermal is MORE, a model-based offline reinforcement learning (RL) framework.
- MORE solves a complex constrained Markov decision process problem through purely offline training.
- DeepThermal utilizes historical operational data to learn a data-driven combustion process simulator.
- The RL agent of MORE is trained using real historical data and carefully filtered simulation data.
- DeepThermal has been successfully deployed in four large coal-fired thermal power plants in China.
- Real-world experiments have demonstrated that DeepThermal effectively improves the combustion efficiency of TPGUs.
- DeepThermal's performance has been compared with state-of-the-art algorithms on standard offline RL benchmarks, showcasing its superior capabilities.
- The authors of DeepThermal include Xianyuan Zhan, Haoran Xu, Yue Zhang, Xiangyu Zhu, Honglei Yin and Yu Zheng.
- Their research on DeepThermal was published in the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI2022).
- DeepThermal presents an innovative approach to optimizing combustion control strategies for TPGUs using offline reinforcement learning techniques.

## Summary

DeepThermal is a data-driven AI system developed to optimize the combustion control strategy for thermal power generating units (TPGUs). The combustion efficiency of TPGUs is a critical task in the energy industry, and DeepThermal aims to address this challenge. At the core of DeepThermal is MORE, a model-based offline reinforcement learning (RL) framework that leverages historical operational data of a TGPU. The MORE framework solves a complex constrained Markov decision process problem through purely offline training. It utilizes the offline dataset to learn a data-driven combustion process simulator. This simulator serves as the foundation for training the RL agent of MORE. The RL agent is trained using a combination of real historical data and carefully filtered and processed simulation data, employing a novel restrictive exploration scheme. DeepThermal has been successfully deployed in four large coal-fired thermal power plants in China. Real-world experiments have demonstrated that DeepThermal effectively improves the combustion efficiency of TPGUs. Furthermore, its performance has been compared with state-of-the-art algorithms on standard offline RL benchmarks, showcasing its superior capabilities. The authors of DeepThermal include Xianyuan Zhan, Haoran Xu, Yue Zhang, Xiangyu Zhu, Honglei Yin and Yu Zheng. Their research on DeepThermal was published in the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI2022). The paper provides valuable insights into optimizing combustion efficiency in TPGUs using offline reinforcement learning techniques. In summary, DeepThermal presents an innovative approach to optimizing combustion control strategies for TPGUs by leveraging historical operational data and employing advanced RL techniques through MORE which achieves significant improvements in combustion efficiency. Its successful deployment in real-world scenarios and superior performance compared to existing algorithms highlight its potential impact on the energy industry's challenges.