Search Results for « Multi-Agent Reinforcement Learning »

Effective management of multi-intersection traffic signal control (MTSC) is vital for intelligent transportation systems. Multi-agent reinforcement learning (MARL) has shown promise in achieving MTSC. However, existing MARL-based MTSC algorithms have primarily focused on capturing the spatial relationship between multi-intersection traffic signals but have overlooking the importance of the temporally stable traffic pattern. This pattern refers to the fixed positions and relatively stable traffic flow between intersections over short periods in real-world MTSC scenarios, which indicates that the learned spatial relationships between traffic signals should co- evolve over time. To this end, we propose a novel algorithm called Coevolutionary Multi-Agent Reinforcement Learning (Co- evoMARL). CoevoMARL employs a graph neural network to capture the complex spatial interaction network among traffic signals. Furthermore, we propose a relationship-driven progres- sive LSTM (RDP-LSTM) that dynamically evolves the learned spatial interaction network over time by leveraging insights from the temporally stable traffic pattern. To accelerate convergence, we also propose the mutual information reward optimization (MIRO) technique, which strengthens the correlation between policy learning and high-performance samples by using a mutual information-based intrinsic reward. Experimental results on both synthetic and real-world datasets demonstrate the superiority of CoevoMARL over existing MTSC algorithms, providing valuable insights into incorporating the temporally stable traffic pattern.

<p>Effective management of multi-intersection traffic signal control (MTSC) is vital for intelligent transportation systems. Multi-agent reinforcement learning (MARL) has shown promise in achieving MTSC. However, existing MARL-based MTSC algorithms have primarily focused on capturing the spatial relationship between multi-intersection traffic signals but have overlooking the importance of the temporally stable traffic pattern. This pattern refers to the fixed positions and relatively stable traffic flow between intersections over short periods in real-world MTSC scenarios, which indicates that the learned spatial relationships between traffic signals should co- evolve over time. To this end, we propose a novel algorithm called Coevolutionary Multi-Agent Reinforcement Learning (Co- evoMARL). CoevoMARL employs a graph neural network to capture the complex spatial interaction network among traffic signals. Furthermore, we propose a relationship-driven progres- sive LSTM (RDP-LSTM) that dynamically evolves the learned spatial interaction network over time by leveraging insights from the temporally stable traffic pattern. To accelerate convergence, we also propose the mutual information reward optimization (MIRO) technique, which strengthens the correlation between policy learning and high-performance samples by using a mutual information-based intrinsic reward. Experimental results on both synthetic and real-world datasets demonstrate the superiority of CoevoMARL over existing MTSC algorithms, providing valuable insights into incorporating the temporally stable traffic pattern. </p>

This research explores the application of Multi-Agent Reinforcement Learning (MARL) to a cooperative spacecraft docking problem with three chaser spacecraft and a lightly tumbling target. The study extends Proximal Policy Optimisation (PPO) based single-agent reinforcement learning (RL) docking to multi-agent spacecraft using Multi-Agent Proximal Policy Optimisation (MAPPO), targeting a simple 3-DOF planar setup. The primary goal is to develop a safe, decentralised docking policy that accounts for low-thrust constraints, fuel efficiency, and inter-agent communication limitations. Agents are only given access to the noisy measurement of the nearest agent to mimic large constellations. The policy is trained with a reward function that penalises position, velocity, fuel consumption, and angular errors while encouraging successful docking and collision avoidance. Experimental validation is conducted via Monte Carlo simulations. Results demonstrate the feasibility of applying MARL to spacecraft docking tasks, achieving a 99.1% docking success rate in simulation. The research highlights the potential of reinforcement learning approaches for future distributed multi-agent space missions. However, further work is needed to address robustness concerns and optimise the policy for more complex scenarios and a large number of agents.