Résultats de recherche pour « LLM-based Multi-Agent Reinforcement Learning »

Fleets of physical components, deployed artificial-intelligence (AI) models, and same-role AI agents degrade over time and require replacement or retraining. Each intervention has a cost, while each batch adds shared setup, validation, or release costs. Evidence is buried in work orders, transcripts, and deployment logs. Existing decompositions have unmeasured optimality gaps, reinforcement learning provides no certificate, and both require a new solve per fleet. Language-model studies emphasize extraction accuracy rather than maintenance costs from reading errors. We introduce a large-language-model (LLM)-observed multi-agent maintenance framework with an amortized reinforcement-learning policy. A zero-shot LLM converts unit records into structured, auditable events describing actions, replacement or retraining status, confidence, and abstention. A tracker maps events to fleet health counts, while an exact count-based model generates optimal actions to train a reusable policy that transfers zero-shot to new fleets. We prove the state reduction and optimal renewal ordering, and certify each solved action against the Bellman equation. Across nine degradation datasets, the policy adds 1.00% cost. On logs, the reader matches supervised baselines with scarce labels. In 42,000 agent episodes, coordinated retraining saves 11.9% in graphicsprocessing-unit (GPU) seconds, while a 7-billion-parameter reader cuts text-induced maintenance regret from 56.0% to 31.7% with zero false resets.

Multi-agent systems (MAS) have gained popularity due to their effectiveness in diverse applications. Among these, decentralized approaches, which rely on inter-agent communication, have emerged as a promising alternative to centralized MAS coordination methods. However, existing communication protocols often learn abstract representations that are difficult for humans to interpret. Also, the limited wireless bandwidth in real-world applications imposes a significant challenge on efficient communication among agents. To address these issues, in this paper, we consider a multi-robot navigation scenario and propose a goal-oriented human-interpretable feature communication protocol, enabling robots to transmit only the most semantically important features. Building on this protocol, we first introduce a Multi-Agent Proximal Policy Optimization (MAPPO)-based Bandwidth-Adaptive Dual-Policy (MAPPO-BDP) framework, which employs a dual-policy architecture to jointly optimize feature selection and navigation under bandwidth constraints. However, as MAPPO-BDP requires retraining for different bandwidth constraints, we further propose a Large Language Model (LLM)-based Bandwidth-Adaptive Dual-Policy (LLM-BDP) framework. By leveraging LLM-based multi-agents, LLM-BDP can adapt to varying bandwidth constraints by simply modifying the input prompt, thereby eliminating the need for retraining. Experimental results show that both MAPPO-BDP and LLM-BDP frameworks successfully complete navigation tasks within a limited number of steps by effectively communicating the most important features under bandwidth constraints. We demonstrate that LLM-BDP achieves better reasoning adaptability by utilizing more advanced LLMs and engineering example prompts. Notably, with human-interpretable feature communication protocol, robots trained with the MAPPO-BDP framework can seamlessly collaborate with robots from the LLM-BDP framework to complete tasks, even without co-training, demonstrating the practical applicability of the proposed communication protocol in new scenarios.

Multi-agent systems (MAS) have gained popularity due to their effectiveness in diverse applications. Among these, decentralized approaches, which rely on inter-agent communication, have emerged as a promising alternative to centralized MAS coordination methods. However, existing communication protocols often learn abstract representations that are difficult for humans to interpret. Also, the limited wireless bandwidth in real-world applications imposes a significant challenge on efficient communication among agents. To address these issues, in this paper, we consider a multi-robot navigation scenario and propose a goal-oriented human-interpretable feature communication protocol, enabling robots to transmit only the most semantically important features. Building on this protocol, we first introduce a Multi-Agent Proximal Policy Optimization (MAPPO)-based Bandwidth-Adaptive Dual-Policy (MAPPO-BDP) framework, which employs a dual-policy architecture to jointly optimize feature selection and navigation under bandwidth constraints. However, as MAPPO-BDP requires retraining for different bandwidth constraints, we further propose a Large Language Model (LLM)-based Bandwidth-Adaptive Dual-Policy (LLM-BDP) framework. By leveraging LLM-based multi-agents, LLM-BDP can adapt to varying bandwidth constraints by simply modifying the input prompt, thereby eliminating the need for retraining. Experimental results show that both MAPPO-BDP and LLM-BDP frameworks successfully complete navigation tasks within a limited number of steps by effectively communicating the most important features under bandwidth constraints. We demonstrate that LLM-BDP achieves better reasoning adaptability by utilizing more advanced LLMs and engineering example prompts. Notably, with human-interpretable feature communication protocol, robots trained with the MAPPO-BDP framework can seamlessly collaborate with robots from the LLM-BDP framework to complete tasks, even without co-training, demonstrating the practical applicability of the proposed communication protocol in new scenarios.

Cross-organization cyber defense must reconcile collaborative learning with privacy and adversarial robustness — yet standard federated learning ships full gradient tensors, leaking sensitive posture and inviting Byzantine manipulation. We present FedMARL-LTI, a federated multi-agent reinforcement learning framework whose architecture answers both pressures with a single decision: organizations share neither raw data nor model weights, only differentially-private 768-dimensional semantic threat embeddings. The contribution is fourfold. (1) Semantic Abstraction (SA) channel: per organization, each round, the local gradient is summarized by an LLM, projected to a 768-dim embedding, L2-clipped, and Gaussian-noised before any numeric quantity leaves the host. The bottleneck reduces the per-element noise scale from O(√(d_model )) to O(√m) with m=768≪dmodel≈3×105. (2) Formal privacy analysis: the SA+DP cascade satisfies (ε,δ)-DP and bounds per-round mutual-information leakage by min{Ttoklog2V, m⁄2 log2 (1+C2/(mσ2))} (Theorem 1), with Rényi composition over T federation rounds (Theorem 2). (3) Byzantine-resilient ClippedClustering aggregator combining L2 clipping with cosine-similarity clustering. (4) Hierarchical MARL policy with threat-profile-aware LLM-IRR reward shaping, wired end-to-end and disclosed honestly (the LLM call is currently stubbed with a deterministic projection for reproducibility). We evaluate on CybORG CAGE-4 with n=5 organizations, 30 federation rounds × 5 episodes × 100 steps per round. The SA channel adds statistical-zero utility cost vs. no-privacy baseline: SA-only Δreward = -0.66 (t=+0.31, NS), dual SA + Weight-DP Δreward = +1.90 (t=-0.71, NS), all N=5 seeds, all |t|< 1.3. A controlled signal/noise probe confirms a 19.58× improvement of SA over Weight-DP at fixed DP budget — matching the predicted √(d/m)≈19.8. Under Byzantine sign_flip at 30% (N=15), ClippedClustering is directionally strongest (F1=0.025 vs FedAvg 0.020, Krum 0.016) but the edge is not statistically significant (CC vs Krum t=+1.59, p=0.15, d=+0.58; the earlier N=5 “3.4×” gap was small-sample optimism, §5.2); its decisive Byzantine win is the harsher random_noise attack, where FedAvg diverges to NaN and Krum collapses to 0.002 while ClippedClustering survives at 0.020 (§5.7, Cohen’s d=+3.77). The cooperative-PPO family (MAPPO, IPPO) outperforms value/actor-critic (QMIX, MADDPG) by ≈20 reward units, p< 0.001. All host-level F1 values stay below 0.05 at the 15K-step training horizon used here; the relative claims of the paper (privacy zero-cost, ClippedClustering’s decisive Byzantine win on the harshest attacks per §5.7, cooperative-PPO dominance) are unaffected by this scope. A 200K-step long-horizon replication (§6.3 L1) lifts F1 above the 15K plateau (to ≈0.044, N=5) — confirming that horizon, not the privacy/Byzantine machinery, gates absolute accuracy — but a finer 60-checkpoint run shows the climb is volatile and non-monotonic and does not reach deployment-grade, an honest stability-not-compute limitation. We release all 141 raw run JSON outputs (Phases 1–3, the L4 backend comparison, and the algorithm/aggregator baselines), the figures, and analysis scripts for replication.

Abstract Traditional neoclassical labor market models, predicated on perfect competition, complete information, and full rationality, fail to capture real-world complexities—including information asymmetry, bounded rationality, and dynamic skill mismatches—resulting in inadequate explanations of structural unemployment and inefficient job matching. To address this, the study develops an integrated simulation framework combining Large Language Models (LLMs), Deep Reinforcement Learning (DRL), and agent-based modeling: LLMs process job descriptions/resumes for multi-modal skill representation and belief updating; DRL models game-theoretic interactions between firms (optimizing hiring/salary) and job seekers (optimizing job search/negotiation) under bounded rationality; additional modules include policy-macroeconomy coupling (for taxes/subsidies/minimum wage) and skill evolution networks (for skill obsolescence/emergence). Experiments, calibrated to U.S. BLS/JOLTS data, yield concrete results: (1) A 10% hiring subsidy reduced unemployment to 4.5% and matching efficiency to 80%, outperforming other policies; (2) A 5% corporate tax increase raised unemployment to 8% and lowered vacancy rates to 3.5%; (3) Low-skill automation shocks pushed unemployment to 9% due to persistent skill mismatches. The framework accurately replicates dynamic labor market behaviors, outperforming traditional models in capturing policy heterogeneity and real-world inefficiencies.

<p>There is growing importance in complex engineering systems to operate autonomously, especially given potential malfunctions that may occur in the system, such as sensors, actuators, components, communication networks, and controllers. Fault detection, isolation, and reconstruction (FDIR) are crucial for autonomous systems. There is significant demand to evolve efficient intelligent systems to detect faults, isolate fault locations and autonomously reconstruct any component of a complex dynamical system. Hence, it is essential to detect, isolate, and reconstruct the faults efficiently and on time when the systems are in operation. This dissertation presents a novel methodology for developing smart autonomous multiagent systems (SAMAS). The SAMAS comprises several agents; some are homogeneous while others are heterogeneous. The proposed method involves developing a multi-agent systems (MAS) model and designing a decentralized smart control system. The MAS model contains homogeneous and heterogeneous agents, communicates among agents through an undirected connected graph, external disturbances, goals, and constraints. The sensor, actuator, communication, and controller faults are also modeled in the MAS model. A decentralized smart control system is designed to create a SAMAS model in the presence of uncertainty in each agents’ dynamics and faults located in the sensors, actuators, communication networks, and controllers. The proposed control method is based on non-linear filtering techniques, deep reinforcement learning, and robust control techniques. A Chebyshev neural network (CNN) is incorporated to learn the uncertain nonlinear functions in the agent dynamics of MAS. Additionally, robust control term using the hyperbolic tangent function is applied to counteract the neural network approximation errors. Meanwhile, a novel algorithm has been proposed which is employed to estimate the uncertain states of the agent dynamics and to train the internal parameters of the neural network given a set of prior measurements. Moreover, an adaptive threshold method has been proposed to detect any kinds of faults present in the system followed by a likelihood-based isolation method to locate each faulty agent. The novel algorithm is known as a reinforced unscented Kalman filter (RUKF). The primary purpose of the RUKF is to detect and isolate the faults, and to adapt the process and measurement noise covariance matrices to reconstruct the faults. To assess the performance of the proposed methodology, we developed a SAMAS consisting of six heterogeneous uncertain agent dynamics. The SAMAS model and the proposed control methodology are numerically simulated using MATLAB. A Monte-Carlo (MC) simulation was carried out to assess the performance of the proposed control methodology in the presence of uncertain agent dynamics and with the sensor, actuator, communication, and controller faults. The fault isolation results are summarized in confusion matrices for each faulty case. The stability of the RUKF, which ran in conjunction with a robust control method, has been proven using the Lyapunov stability approach. Extensive simulations were conducted to evaluate the performance of the proposed method. In this study, the proposed method showed superior performance to the standard unscented Kalman filter (UKF) and adaptive UKF (AUKF). The proposed fault isolation scheme isolated the faulty agents with over 96.68% success rate at the system level. Hence, the results of the numerical simulations show the feasibility of the proposed approach. The proposed RUKF is also computationally less expensive than the standard UKF and AUKF. The proposed approach can be considered a promising tool to evaluate fault detection, isolation, and reconstruction in complex engineering systems. Furthermore, the proposed approach can be extended to other deep space complex systems. The proposed approach can also be used for MAS problems for the financial sector, such as stock market prediction, economic time series, and multi-arm bandit problems.</p>

<p>There is growing importance in complex engineering systems to operate autonomously, especially given potential malfunctions that may occur in the system, such as sensors, actuators, components, communication networks, and controllers. Fault detection, isolation, and reconstruction (FDIR) are crucial for autonomous systems. There is significant demand to evolve efficient intelligent systems to detect faults, isolate fault locations and autonomously reconstruct any component of a complex dynamical system. Hence, it is essential to detect, isolate, and reconstruct the faults efficiently and on time when the systems are in operation. This dissertation presents a novel methodology for developing smart autonomous multiagent systems (SAMAS). The SAMAS comprises several agents; some are homogeneous while others are heterogeneous. The proposed method involves developing a multi-agent systems (MAS) model and designing a decentralized smart control system. The MAS model contains homogeneous and heterogeneous agents, communicates among agents through an undirected connected graph, external disturbances, goals, and constraints. The sensor, actuator, communication, and controller faults are also modeled in the MAS model. A decentralized smart control system is designed to create a SAMAS model in the presence of uncertainty in each agents’ dynamics and faults located in the sensors, actuators, communication networks, and controllers. The proposed control method is based on non-linear filtering techniques, deep reinforcement learning, and robust control techniques. A Chebyshev neural network (CNN) is incorporated to learn the uncertain nonlinear functions in the agent dynamics of MAS. Additionally, robust control term using the hyperbolic tangent function is applied to counteract the neural network approximation errors. Meanwhile, a novel algorithm has been proposed which is employed to estimate the uncertain states of the agent dynamics and to train the internal parameters of the neural network given a set of prior measurements. Moreover, an adaptive threshold method has been proposed to detect any kinds of faults present in the system followed by a likelihood-based isolation method to locate each faulty agent. The novel algorithm is known as a reinforced unscented Kalman filter (RUKF). The primary purpose of the RUKF is to detect and isolate the faults, and to adapt the process and measurement noise covariance matrices to reconstruct the faults. To assess the performance of the proposed methodology, we developed a SAMAS consisting of six heterogeneous uncertain agent dynamics. The SAMAS model and the proposed control methodology are numerically simulated using MATLAB. A Monte-Carlo (MC) simulation was carried out to assess the performance of the proposed control methodology in the presence of uncertain agent dynamics and with the sensor, actuator, communication, and controller faults. The fault isolation results are summarized in confusion matrices for each faulty case. The stability of the RUKF, which ran in conjunction with a robust control method, has been proven using the Lyapunov stability approach. Extensive simulations were conducted to evaluate the performance of the proposed method. In this study, the proposed method showed superior performance to the standard unscented Kalman filter (UKF) and adaptive UKF (AUKF). The proposed fault isolation scheme isolated the faulty agents with over 96.68% success rate at the system level. Hence, the results of the numerical simulations show the feasibility of the proposed approach. The proposed RUKF is also computationally less expensive than the standard UKF and AUKF. The proposed approach can be considered a promising tool to evaluate fault detection, isolation, and reconstruction in complex engineering systems. Furthermore, the proposed approach can be extended to other deep space complex systems. The proposed approach can also be used for MAS problems for the financial sector, such as stock market prediction, economic time series, and multi-arm bandit problems.</p>