# Improved Text Classification via Test-Time Augmentation

**Authors:** Helen Lu, Divya Shanmugam, Harini Suresh, John Guttag  
**arXiv:** [2206.13607](https://arxiv.org/abs/2206.13607)  
**Date:** 2022-06-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.13607v1/

## Key points

- Test-time augmentation (TTA) is a technique used in image classification to improve model performance without additional training.
- TTA has seen limited adoption in natural language processing (NLP) due to the difficulty of identifying label-preserving transformations.
- The authors present augmentation policies that yield significant accuracy improvements with language models using TTA.
- Augmentation policy design, such as the number of samples generated from a single non-deterministic augmentation, has a considerable impact on the benefit of TTA.
- The authors apply an augmentation policy containing M transforms to generate M transformed inputs from a text input t.
- A single prediction is generated by applying a simple average to the M + 1 logit predictions.
- The study evaluates the performance of their method across the WILDS CivilComments dataset which consists of 448,000 comments made on Wikipedia talk pages labeled for toxicity and identity-based hate speech detection tasks.
- Experiments show that test-time augmentation can deliver consistent improvements over current state-of-the-art approaches across binary classification tasks and datasets.
- Certain combinations of augmentations yield better results than others.
- This study demonstrates how test-time augmentation can be applied effectively to improve text classification models' performance without additional training.

## Summary

Test-time augmentation (TTA) is a well-established technique in image classification that involves aggregating predictions across transformed examples of test inputs to improve model performance without additional training. However, TTA has seen limited adoption in natural language processing (NLP) due to the difficulty of identifying label-preserving transformations. In this paper titled "Improved Text Classification via Test-Time Augmentation," the authors present augmentation policies that yield significant accuracy improvements with language models using TTA. The study shows that augmentation policy design, such as the number of samples generated from a single non-deterministic augmentation, has a considerable impact on the benefit of TTA. The authors apply an augmentation policy containing M transforms to generate M transformed inputs from a text input t. All M + 1 inputs, including the original and transformed ones, are then passed into a pre-trained classifier f to generate (M + 1) RC vectors containing class logit predictions. A single prediction is generated by applying a simple average to the M + 1 logit predictions. The authors choose averaging because it is the simplest version of TTA and suits their goal of understanding the baseline value of TTA in NLP. The study evaluates the performance of their method across a dataset and model architecture laid out in detail in subsequent sections. They use the WILDS CivilComments dataset which consists of 448,000 comments made on Wikipedia talk pages labeled for toxicity and identity-based hate speech detection tasks. Experiments show that test-time augmentation can deliver consistent improvements over current state-of-the-art approaches across binary classification tasks and datasets. The authors also find that certain combinations of augmentations such as Insertion+Swap+Synonym (PPDB)+Synonym (WordNet) or Deletion+Insertion+Swap+Synonym (PPDB) yield better results than others. Overall, this study demonstrates how test-time augmentation can be applied effectively to improve text classification models' performance without additional training. The authors' augmentation policies and findings provide valuable insights for future research in this area.