# Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning

**Authors:** Tan-Hanh Pham, Godwyll Aikins, Tri Truong, Kim-Doang Nguyen  
**arXiv:** [2409.14435](https://arxiv.org/abs/2409.14435)  
**Date:** 2024-09-22  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2409.14435v1/

## Key points

- Robotic manipulators are crucial in industries for complex and repetitive tasks
- Hardware failures can disrupt task execution
- Study focuses on enabling robotic manipulator to operate effectively despite joint malfunctions
- Approach involves developing a reinforcement learning (RL) framework for adaptive compensation
- Experiment conducted on Franka robot with 7 degrees of freedom (DOFs)
- Problem formulated as a partially observable Markov decision process (POMDP)
- Scenarios explored include permanently broken joints and intermittent functionality
- RL algorithm successfully completes tasks with an average success rate of 93.6%
- Reinforcement learning enhances resilience and reliability of robotic systems
- Research aims to advance fault-tolerant control in robotics using RL techniques

## Summary

Robotic manipulators are essential in various industries for performing complex and repetitive tasks. However, they are susceptible to unexpected hardware failures, which can disrupt task execution. In this study, we tackle the challenge of enabling a robotic manipulator to continue operating effectively despite joint malfunctions. Our approach involves developing a reinforcement learning (RL) framework that can adaptively compensate for non-functional joints during task execution. The experimental platform used in this research is the Franka robot with 7 degrees of freedom (DOFs). We formulate the problem as a partially observable Markov decision process (POMDP), where the robot undergoes training under different joint failure conditions and is tested in both familiar and unfamiliar scenarios. We explore scenarios where a joint is permanently broken as well as situations where it functions intermittently. To evaluate the effectiveness of our approach, we compare it with traditional inverse kinematics-based control methods. The results demonstrate that the RL algorithm enables the robot to successfully complete tasks even when faced with joint failures, achieving an impressive average success rate of 93.6%. This showcases the robustness and adaptability of our proposed framework. Furthermore, our findings emphasize the potential of reinforcement learning to enhance the resilience and reliability of robotic systems, making them better equipped to handle unpredictable environments. By publishing all related codes and models online, we aim to contribute to advancing fault-tolerant control in robotics and providing more robust solutions for increasingly complex robotic systems. Moving forward, our research aims to further explore fault-tolerant control strategies in robotics by leveraging reinforcement learning techniques. The paper is structured into sections that review related work on fault-tolerant control strategies and reinforcement learning in robotic systems, present details of our methodology including problem formulation and simulation setup, illustrate the limitations of traditional methods when faced with joint failures, discuss experimental results from various fault scenarios showcasing the effectiveness of our approach, and conclude with insights on future directions for research in this field.