# The Evaluation of Rating Systems in Team-based Battle Royale Games

**Authors:** Arman Dehpanah, Muheeb Faizan Ghori, Jonathan Gemmell, Bamshad Mobasher  
**arXiv:** [2105.14069](https://arxiv.org/abs/2105.14069)  
**Date:** 2021-05-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2105.14069v2/

## Key points

- Online competitive games use rating systems to match players with similar skills
- Less attention has been paid to evaluating the performance of these rating systems
- The paper "The Evaluation of Rating Systems in Team-based Battle Royale Games" explores the utility of several metrics for evaluating three popular rating systems on a real-world dataset of over 25,000 team battle royale matches
- Results suggest considerable differences in the evaluation patterns among the metrics studied
- Normalized discounted cumulative gain (NDCG) demonstrated more reliable performance and flexibility compared to other metrics studied
- Evaluating rating systems is crucial for ensuring fair gameplay experiences for all players
- The authors' findings can inform future research aimed at developing better evaluation methods for rating systems in online competitive games to ensure fairness for all participants.

## Summary

The popularity of online competitive games has led to the use of rating systems to match players with similar skills, creating a fair and exciting experience. However, while there has been a significant amount of research on improving these systems, less attention has been paid to how their performance is evaluated. In their paper titled "The Evaluation of Rating Systems in Team-based Battle Royale Games," Arman Dehpanah, Muheeb Faizan Ghori, Jonathan Gemmell, and Bamshad Mobasher explore the utility of several metrics for evaluating three popular rating systems on a real-world dataset of over 25,000 team battle royale matches. Their results suggest considerable differences in the evaluation patterns among the metrics studied. Some metrics were highly impacted by the inclusion of new players while many could not capture the real differences between certain groups of players. Among all metrics studied, normalized discounted cumulative gain (NDCG) demonstrated more reliable performance and flexibility. It alleviated most of the challenges faced by other metrics while adding freedom to adjust the focus of evaluations on different groups of players. This study highlights that evaluating rating systems is crucial for ensuring fair gameplay experiences for all players. The authors' findings can inform future research aimed at developing better evaluation methods for rating systems in online competitive games to ensure fairness for all participants.