# Optimization Theory Based Deep Reinforcement Learning for Resource Allocation in Ultra-Reliable Wireless Networked Control Systems

**Authors:** Hamida Qumber Ali, Amirhassan Babazadeh Darabi, Sinem Coleri  
**arXiv:** [2311.16895](https://arxiv.org/abs/2311.16895)  
**Date:** 2023-11-28  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2311.16895v1/

## Key points

- Wireless Networked Control Systems (WNCS) require a design that minimizes complexity and communication overhead while ensuring ultra-high reliability.
- A groundbreaking approach is introduced using optimization theory based deep reinforcement learning (DRL) for joint controller and communication system design.
- The framework aims to minimize power consumption, adhere to communication constraints, and maintain stability in the control system.
- Key decision variables include sampling period in the control system, blocklength, and packet error probability in the communication system.
- The methodology unfolds in two stages: optimization theory and DRL, where optimal conditions are derived and complex blocks are addressed using DRL techniques.
- Extensive simulations show that this approach outperforms traditional methods, achieving close-to-optimal performance with reduced complexity.
- The framework enables more efficient resource allocation in ultra-reliable wireless networked control systems, enhancing overall performance and reliability.

## Summary

In the realm of Wireless Networked Control Systems (WNCS), the intricate interplay between control and communication systems necessitates a design that minimizes complexity and communication overhead while ensuring ultra-high reliability. To address this challenge, a groundbreaking approach is introduced in the form of an optimization theory based deep reinforcement learning (DRL) framework for the joint design of controller and communication systems. The primary objective of this framework is to minimize power consumption while adhering to the schedulability and rate constraints of the communication system within the finite blocklength regime, as well as maintaining stability in the control system. Key decision variables in this framework include the sampling period in the control system, along with blocklength and packet error probability in the communication system. The proposed methodology unfolds in two distinct stages: optimization theory and DRL. In the optimization theory stage, optimal conditions are derived to establish mathematical relationships among decision variables, enabling the problem to be deconstructed into manageable building blocks. Subsequently, in the DRL stage, complex yet unsolvable blocks are replaced by deep reinforcement learning techniques. Extensive simulations have demonstrated that this optimization theory based DRL approach surpasses both traditional optimization theory methods and pure DRL approaches. It achieves close-to-optimal performance levels with significantly reduced complexity. This innovative framework paves the way for more efficient resource allocation in ultra-reliable wireless networked control systems, offering a promising avenue for enhancing overall system performance and reliability.