# CatBoost: unbiased boosting with categorical features

**Authors:** Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin  
**arXiv:** [1706.09516](https://arxiv.org/abs/1706.09516)  
**Date:** 2017-06-28  
**Summary page:** https://www.summarizepaper.com/en/paper/1706.09516v5/

## Key points

- CatBoost is a new gradient boosting toolkit that outperforms other publicly available boosting implementations in terms of quality on various datasets.
- The paper introduces two critical algorithmic advances in CatBoost: ordered boosting and an innovative algorithm for processing categorical features.
- Ordered boosting addresses a prediction shift caused by target leakage present in existing gradient boosting algorithms, improving prediction accuracy.
- CatBoost's algorithm efficiently and accurately handles categorical features, which pose challenges in traditional gradient boosting algorithms.
- The paper provides a detailed analysis of the prediction shift problem caused by target leakage and explains how the proposed algorithms effectively solve it.
- Empirical results demonstrate that CatBoost achieves excellent performance across different datasets.
- CatBoost surpasses existing implementations and establishes itself as a powerful tool for accurate predictions on diverse datasets.

## Summary

This paper titled "CatBoost: unbiased boosting with categorical features" presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. The authors demonstrate that the combination of these techniques leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. The paper introduces two critical algorithmic advances in CatBoost. The first is the implementation of ordered boosting, which is a permutation-driven alternative to the classic algorithm. This approach addresses a prediction shift caused by target leakage present in existing gradient boosting algorithms. By using ordered boosting, CatBoost effectively mitigates this problem and improves prediction accuracy. The second innovation in CatBoost is an innovative algorithm for processing categorical features. Categorical features pose challenges in traditional gradient boosting algorithms, but CatBoost's algorithm handles them efficiently and accurately. This technique further contributes to the superior performance of CatBoost compared to other implementations. The paper provides a detailed analysis of the prediction shift problem caused by target leakage and explains how the proposed algorithms effectively solve it. Through empirical results, the authors demonstrate that CatBoost achieves excellent performance across various datasets. Overall, this paper highlights the significant contributions of ordered boosting and innovative categorical feature processing in improving gradient boosting performance. The findings showcase how CatBoost surpasses existing implementations and establishes itself as a powerful tool for accurate predictions on diverse datasets.