# Towards autonomous system: flexible modular production system enhanced with large language model agents

**Authors:** Yuchen Xia, Manthan Shenoy, Nasser Jazdi, Michael Weyrich  
**arXiv:** [2304.14721](https://arxiv.org/abs/2304.14721)  
**Date:** 2023-04-28  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2304.14721v4/

## Key points

- Integration of large language models (LLMs), digital twins, and industrial automation systems for intelligent planning and control of production processes in modular facilities
- Retrofitting automation system to create executable control interfaces with fine-granular functionalities and coarse-granular skills
- Development of digital twin system to register interfaces and provide descriptive information about the production system
- LLM-agents designed to interpret descriptive information and control physical system through service interfaces, enabling autonomous planning and control of flexible production processes
- Challenges include ensuring deterministic outputs in non-deterministic environments, addressing data dilemmas for testing, and computational complexity of LLM models for local deployment

## Summary

, , , , 

This paper presents a novel framework that integrates large language models (LLMs), digital twins, and industrial automation systems to enable intelligent planning and control of production processes in modular facilities. The retrofitting of the automation system allows for the creation of executable control interfaces with fine-granular functionalities and coarse-granular skills. Low-level functionalities are executed by automation components, while high-level skills are performed by automation modules. A digital twin system is developed to register these interfaces and provide additional descriptive information about the production system. Based on the retrofitted automation system and digital twins, LLM-agents are designed to interpret descriptive information and control the physical system through service interfaces. These intelligent agents operate on different levels within the automation system, enabling autonomous planning and control of flexible production processes. When given a task instruction as input, the LLM-agents orchestrate a sequence of atomic functionalities and skills to accomplish the task. The research demonstrates how the implemented prototype can handle un-predefined tasks, plan production processes, and execute operations efficiently. By integrating LLMs into industrial automation systems within smart factories, there is potential for more agile, flexible, and adaptive production processes. However, there are critical insights regarding challenges such as ensuring deterministic outputs in non-deterministic environments and addressing data dilemmas for comprehensive testing and evaluation. The computational complexity of LLM models poses challenges for local deployment in real-world industrial settings due to their size and computational requirements. Fine-tuning smaller models with domain-specific data may be a solution, but larger models trained on diverse knowledge domains could offer stronger general intelligence for specialized tasks. In conclusion, further collaborative efforts are needed to develop benchmarks for evaluating LLM agents in industrial automation tasks. The paper also highlights the importance of defining clear roles and objectives for LLM agents to align with specific tasks effectively. Additionally, leveraging information from digital twins can enhance descriptive capabilities for improved task execution efficiency in industrial automation systems.