# DiSMEC - Distributed Sparse Machines for Extreme Multi-label Classification

**Authors:** Rohit Babbar, Bernhard Shoelkopf  
**arXiv:** [1609.02521](https://arxiv.org/abs/1609.02521)  
**Date:** 2016-09-08  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1609.02521v1/

## Key points

- DiSMEC is a framework for extreme multi-label classification with supervised learning and large-scale datasets.
- The datasets exhibit fit to power-law distribution where most labels have very few positive instances in the data distribution.
- Most state-of-the-art approaches use low-dimensional linear subspace to capture correlation among labels, but this can be violated in the presence of power-law distributed extremely large and diverse label spaces.
- Unlike other methods, DiSMEC does not make any low rank assumptions on the label matrix and instead uses one versus rest linear classifiers coupled with explicit capacity control to control model size.
- DiSMEC can learn classifiers for datasets consisting hundreds of thousands labels within few hours using double layer parallelization.
- The explicit capacity control mechanism filters out spurious parameters which keep the model compact in size without losing prediction accuracy.
- Empirical evaluation on publicly available real world datasets consisting up to 670,000 labels showed that DiSMEC significantly boosted prediction accuracies compared to SLECC and FastXML, with an absolute improvement of 10% and 15%, respectively.
- Overall, DiSMEC presents a promising solution for extreme multi-label classification that does not rely on low rank assumptions and provides explicit capacity control while maintaining high prediction accuracy.

## Summary

DiSMEC is a large-scale distributed framework for extreme multi-label classification which involves supervised learning with hundreds of thousands or even millions of labels. The datasets in this type of classification exhibit fit to power-law distribution where a large fraction of labels have very few positive instances in the data distribution. Most state-of-the-art approaches attempt to capture correlation among labels by embedding the label matrix to a low-dimensional linear subspace. However, this approach can be easily violated in the presence of power-law distributed extremely large and diverse label spaces. Unlike most state-of-the-art methods, DiSMEC does not make any low rank assumptions on the label matrix and instead uses one versus rest linear classifiers coupled with explicit capacity control to control model size. Using double layer parallelization, DiSMEC can learn classifiers for datasets consisting hundreds of thousands labels within few hours. The explicit capacity control mechanism filters out spurious parameters which keep the model compact in size without losing prediction accuracy. The authors conducted extensive empirical evaluation on publicly available real world datasets consisting up to 670,000 labels and compared DiSMEC with recent state of the art approaches such as SLEEC and FastXML. On some of the datasets, DiSMEC significantly boosted prediction accuracies - 10% better compared to SLECC and 15% better compared to FastXML, in absolute terms. Overall, DiSMEC presents a promising solution for extreme multi-label classification that does not rely on low rank assumptions and provides explicit capacity control while maintaining high prediction accuracy.