# Linear Adversarial Concept Erasure

**Authors:** Shauli Ravfogel, Michael Twiton, Yoav Goldberg, Ryan Cotterell  
**arXiv:** [2201.12091](https://arxiv.org/abs/2201.12091)  
**Date:** 2022-01-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.12091v1/

## Key points

- Authors address lack of control over pre-trained representations in neural models trained on textual data
- Proposed method identifies and erases linear subspace corresponding to a specific concept
- Method aims to mitigate bias effectively by preventing recovery of the erased concept by linear predictors
- Problem framed as constrained, linear minimax game with closed-form solution for certain objectives
- Introduce R-LACE, a convex relaxation technique that performs well in other scenarios
- Experiments focused on binary gender removal show significant reduction in bias based on evaluation metrics
- Method proves highly expressive in mitigating bias within deep nonlinear classifiers while being tractable and interpretable
- Offers promising implications for improving fairness and reducing undesired influences in neural models trained on textual data

## Summary

In their paper titled "Linear Adversarial Concept Erasure," authors Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan Cotterell address the issue of lack of control over pre-trained representations in modern neural models trained on textual data. These representations are crucial for real-world applications but can perpetuate biases and unwanted concepts. To tackle this problem, the authors propose a novel approach to identify and erase a linear subspace corresponding to a specific concept. Their method aims to mitigate bias effectively by preventing linear predictors from recovering this concept. The authors frame the problem as a constrained, linear minimax game and introduce a closed-form solution for certain objectives. They also present R-LACE, a convex relaxation technique that performs well in other scenarios. Through experiments focused on binary gender removal, the proposed method successfully identifies a low-dimensional subspace whose removal significantly reduces bias based on intrinsic and extrinsic evaluation metrics. Despite its linear nature, the method proves to be highly expressive in mitigating bias within deep nonlinear classifiers while maintaining tractability and interpretability. This innovative approach offers promising implications for improving fairness and reducing undesired influences in neural models trained on textual data.