# A Model-Based Reinforcement Learning Approach for PID Design

**Authors:** Hozefa Jesawada, Amol Yerudkar, Carmen Del Vecchio, Navdeep Singh  
**arXiv:** [2206.03567](https://arxiv.org/abs/2206.03567)  
**Date:** 2022-06-07  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2206.03567v1/

## Key points

- Generalized method for designing PID controllers using model-based reinforcement learning
- Leveraging PILCO method and KLD to transform optimal policy into robust PID tuning parameters
- Utilizing KLD to design interpretable PID controllers for underactuated systems
- Robustness against disturbances and system parameter uncertainties
- Proposing a KLD-based framework for designing optimal PID tuning parameters for MIMO systems
- Utilizing PILCO structure to present main results on PID design
- Demonstrating effectiveness through simulation studies on a benchmark cart-pole system
- Integration with various model-based and model-free algorithms
- Designing robust PID controllers without prior knowledge of system dynamics, utilizing recorded system data for iterative optimization
- Improved performance and robustness compared to traditional methods in handling disturbances and uncertainties in real-world scenarios.

## Summary

The paper presents a generalized method for designing Proportional-Integral-Derivative (PID) controllers using a model-based reinforcement learning (RL) approach. PID controllers are widely used in industrial process control applications due to their simplicity, but fine-tuning their parameters can be challenging. The proposed method leverages the probabilistic inference for learning control (PILCO) method and Kullback-Leibler divergence (KLD) to transform the optimal policy obtained from PILCO into robust PID tuning parameters for underactuated mechanical systems. Unlike previous approaches, this paper utilizes KLD to design interpretable PID controllers for underactuated systems. The designed controllers are robust against disturbances and system parameter uncertainties. The main contributions of the paper include: 1. Proposing a KLD-based generalized framework for designing optimal PID tuning parameters for Multiple-Input-Multiple-Output (MIMO) systems; 2. Utilizing the PILCO structure to present the main results on PID design; 3. Demonstrating the effectiveness of the proposed method through simulation studies on a benchmark cart-pole system. The presented method is general and can be integrated with various model-based and model-free algorithms. It enables the design of robust PID controllers without prior knowledge of system dynamics, utilizing recorded system data for iterative optimization. Overall, this research provides a novel approach to designing PID controllers using RL techniques, offering improved performance and robustness compared to traditional methods. The simulation studies demonstrate the effectiveness of the devised PID controllers in handling disturbances and uncertainties in real-world scenarios.