# Evaluation of Confidence-based Ensembling in Deep Learning Image Classification

**Authors:** Rafael Rosales, Peter Popov, Michael Paulitsch  
**arXiv:** [2303.03185](https://arxiv.org/abs/2303.03185)  
**Date:** 2023-03-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.03185v1/

## Key points

- Ensembling is a widely recognized technique for enhancing machine learning model performance
- Conf-Ensemble focuses on utilizing model confidence rather than errors to address challenging edge cases in classification tasks
- Conf-Ensemble outperformed traditional boosting in binary classification scenarios with limited feature space
- Conf-Ensemble was evaluated in a complex image classification task using the ImageNet dataset
- An enhancement to Conf-Ensemble was proposed to increase the number of samples fed into successive ensemble members
- A three-member Conf-Ensemble incorporating this improvement demonstrated improved accuracy compared to a single model
- Challenges exist in leveraging big data and achieving significant benefits through specialization on complex input samples within multi-label classification tasks
- Studies have explored prediction diversity within ensembles and newer deep learning models like transformers and those generated through neural architecture search (NAS)
- A novel diversity metric based on attribution has been proposed to provide insights into why models make specific predictions
- Further research is needed to overcome limitations and fully harness the potential of Conf-Ensemble across diverse classification scenarios

## Summary

Ensembling is a widely recognized technique for enhancing the performance of machine learning models. A recent adaptation known as Conf-Ensemble has emerged, focusing on utilizing model confidence rather than errors to address challenging edge cases in classification tasks. This method involves creating successive model experts for samples that pose difficulty, even if they were not necessarily misclassified by the previous model. Initially shown to outperform traditional boosting in binary classification scenarios with a limited feature space, Conf-Ensemble was further evaluated in a more complex image classification task using the ImageNet dataset, which consists of 224x224x3 features across 1000 classes. The study aimed to assess the applicability of Conf-Ensemble in safety-critical applications requiring AI-based perception. To address this challenge, an enhancement to Conf-Ensemble was proposed to increase the number of samples fed into successive ensemble members. A three-member Conf-Ensemble incorporating this improvement demonstrated improved accuracy compared to a single model, although the magnitude of improvement was not substantial. These findings shed light on the complexities involved in leveraging big data and highlight the challenges associated with achieving significant benefits through specialization on complex input samples within multi-label classification tasks. Additionally, related studies have explored the impact of prediction diversity within ensembles and evaluated newer deep learning models like transformers and those generated through neural architecture search (NAS). A novel diversity metric based on attribution has been proposed to provide insights into why models make specific predictions. Overall, while Conf-Ensemble shows promise in enhancing ensemble accuracy through specialized expertise, further research is needed to overcome limitations and fully harness its potential across diverse classification scenarios.