# Are AlphaZero-like Agents Robust to Adversarial Perturbations?

**Authors:** Li-Cheng Lan, Huan Zhang, Ti-Rong Wu, Meng-Yu Tsai, I-Chen Wu, Cho-Jui Hsieh  
**arXiv:** [2211.03769](https://arxiv.org/abs/2211.03769)  
**Date:** 2022-11-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.03769v1/

## Key points

- AlphaZero has demonstrated that neural-network-based Go AIs can surpass human performance by a large margin
- Researchers have raised concerns about whether these agents are robust to adversarial perturbations
- Li-Cheng Lan and colleagues investigate the existence of adversarial states in Go AIs that may lead them to play surprisingly wrong actions
- Adversarial state is one that leads to an undoubtedly inferior action that is obvious even for Go beginners
- The authors develop the first adversarial attack on Go AIs which can efficiently search for adversarial states by strategically reducing the search space
- Both Policy-Value neural network (PV-NN) and Monte Carlo tree search (MCTS) can be misled by adding one or two meaningless stones
- 90% of examples indeed lead the AI agent to play an obviously inferior action when evaluated with amateur human Go players
- This study highlights potential vulnerabilities in current AI systems' robustness when faced with adversarial perturbations in complex games like Go.

## Summary

In recent years, the success of AlphaZero (AZ) has demonstrated that neural-network-based Go AIs can surpass human performance by a large margin. However, researchers have raised concerns about whether these agents are robust to adversarial perturbations. In this paper titled "Are AlphaZero-like Agents Robust to Adversarial Perturbations? ", Li-Cheng Lan and colleagues investigate the existence of adversarial states in Go AIs that may lead them to play surprisingly wrong actions. They extend the concept of adversarial examples to the game of Go by generating perturbed states that are semantically equivalent to the original state by adding meaningless moves to the game. An adversarial state is one that leads to an undoubtedly inferior action that is obvious even for Go beginners. The authors acknowledge that searching for such states is challenging due to the large, discrete, and non-differentiable search space in Go. To tackle this challenge, they develop the first adversarial attack on Go AIs which can efficiently search for adversarial states by strategically reducing the search space. This method can also be extended to other board games such as NoGo. Experimentally, they show that both Policy-Value neural network (PV-NN) and Monte Carlo tree search (MCTS) can be misled by adding one or two meaningless stones. For example, on 58% of AlphaGo Zero self-play games their method can make KataGo agent with 50 simulations of MCTS plays a losing action by adding two meaningless stones. The authors further evaluated their algorithm's found adversarial examples with amateur human Go players and found that 90% of examples indeed lead the AI agent to play an obviously inferior action. Overall, this study highlights potential vulnerabilities in current AI systems' robustness when faced with adversarial perturbations in complex games like Go. The authors provide their code at \url{https://PaperCode.cc/GoAttack} for further exploration and development. The findings of this study have implications for developing more robust AI systems in complex games and other domains.