Results of the summarizing process for the arXiv paper: 2303.01646v1

The deployment of autonomous robots in complex environments presents challenges such as platform degradation, environmental uncertainties, and deviations from validated operation conditions. These factors can make it difficult for human partners to understand the capabilities and limitations of robots. To address this issue, self-assessment of robot competency in dynamic and uncertain environments is crucial for successful human-robot teaming. In this study presented at the HRI 2023 workshop on Variable Autonomy for Human-Robot Teaming, an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm was evaluated for autonomous agents to dynamically assess task confidence during execution. The ET-GOA algorithm uses a fast online statistical test of the agent's observations and its model predictions to determine when competency assessment is necessary. The agent then selects a goal based on equation 2 and navigates directly to that goal until it either reaches the goal or triggers a re-assessment and chooses a new goal. Experimental results using ET-GOA to generate competency reports during a simulated delivery task showed significant effects of each reporting condition on deliveries in both static and dynamic environments. In the static environment, there were significant increases in deliveries with ET-GOA compared to random and with GOA compared to random. There was no difference between GOA and ET-GOA in the static environment. In contrast, in the dynamic environment, there were significant differences between all reporting conditions on deliveries. Overall, this study demonstrates that self-assessing agents using algorithms such as ET-GOA can improve their performance in delivering tasks by dynamically assessing their own competency levels during execution. Future research directions include exploring other types of statistical tests for competency assessment and investigating how these assessments can be used to improve human-robot teaming further.

• - Deployment of autonomous robots in complex environments presents challenges such as platform degradation, environmental uncertainties, and deviations from validated operation conditions.
• - These factors can make it difficult for human partners to understand the capabilities and limitations of robots.
• - Self-assessment of robot competency in dynamic and uncertain environments is crucial for successful human-robot teaming.
• - An Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm was evaluated for autonomous agents to dynamically assess task confidence during execution.
• - The ET-GOA algorithm uses a fast online statistical test of the agent's observations and its model predictions to determine when competency assessment is necessary.
• - Experimental results using ET-GOA to generate competency reports during a simulated delivery task showed significant effects on deliveries in both static and dynamic environments.
• - In the static environment, there were significant increases in deliveries with ET-GOA compared to random and with GOA compared to random. There was no difference between GOA and ET-GOA in the static environment.
• - In the dynamic environment, there were significant differences between all reporting conditions on deliveries.
• - Self-assessing agents using algorithms such as ET-GOA can improve their performance in delivering tasks by dynamically assessing their own competency levels during execution.

1. Robots that can work on their own in complicated places face difficulties like wear and tear, unpredictable surroundings, and not following the usual rules. 2. These things can make it hard for people to know what the robots can do and what they can't do. 3. It's important for robots to be able to tell how good they are at a job when things are always changing. 4. A special computer program called ET-GOA was tested to help robots figure out how well they were doing during a task. 5. The ET-GOA program uses math to quickly check if the robot is doing what it should be doing. Definitions- Autonomous: Something that works by itself without needing someone else to control it - Dynamic: Always changing or moving - Algorithm: A set of instructions given to a computer or machine to follow - Competency: How good someone or something is at doing a certain job or task

Self-Assessment of Autonomous Agents for Human-Robot Teaming

The deployment of autonomous robots in complex environments presents a number of challenges, including platform degradation, environmental uncertainties, and deviations from validated operation conditions. These factors can make it difficult for human partners to understand the capabilities and limitations of robots. To address this issue, self-assessment of robot competency in dynamic and uncertain environments is crucial for successful human-robot teaming. In this study presented at the HRI 2023 workshop on Variable Autonomy for Human-Robot Teaming, an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm was evaluated for autonomous agents to dynamically assess task confidence during execution. The ET-GOA algorithm uses a fast online statistical test of the agent's observations and its model predictions to determine when competency assessment is necessary. The agent then selects a goal based on equation 2 and navigates directly to that goal until it either reaches the goal or triggers a re-assessment and chooses a new goal.

Experimental Results

Experimental results using ET-GOA to generate competency reports during a simulated delivery task showed significant effects of each reporting condition on deliveries in both static and dynamic environments. In the static environment, there were significant increases in deliveries with ET-GOA compared to random and with GOA compared to random. There was no difference between GOA and ET-GOA in the static environment. In contrast, in the dynamic environment, there were significant differences between all reporting conditions on deliveries.

Conclusion

Overall, this study demonstrates that self-assessing agents using algorithms such as ET-GOA can improve their performance in delivering tasks by dynamically assessing their own competency levels during execution. Future research directions include exploring other types of statistical tests for competency assessment and investigating how these assessments can be used to improve human-robot teaming further.