A Data-Centric Approach for Improving Adversarial Training Through the Lens of Out-of-Distribution Detection

Authors: Mohammad Azizmalayeri, Arman Zarei, Alireza Isavand, Mohammad Taghi Manzuri, Mohammad Hossein Rohban

Accepted to CSICC 2023
License: CC BY 4.0

Results of the summarizing process for the arXiv paper: 2301.10454v1

  • Comprehensive Summary
  • Key points
  • Layman's Summary
  • Blog article
In this study, the researchers address the issue of adversarial perturbations in machine learning models. While these models have achieved super-human performance in various real-world applications, they are still vulnerable to imperceptible adversarial perturbations. Adversarial training has been identified as an effective solution to this problem, where the model is trained using adversarially perturbed samples instead of original ones. To improve adversarial training, several methods have been developed over recent years, such as data augmentation or modifying training attacks. However, in this work, the researchers propose a new data-centric approach to tackle this problem. They first demonstrate that existing model-based methods can be equivalent to applying smaller perturbation or optimization weights to hard training examples. Building on this finding, the researchers suggest detecting and removing these hard samples directly from the training procedure instead of using complicated algorithms to mitigate their effects. For detection, they utilize maximum softmax probability as an effective method for out-of-distribution detection since hard samples can be considered as out-of-distribution samples for the entire data distribution. The effectiveness of this approach is demonstrated through experiments conducted on SVHN and CIFAR-10 datasets. The results show that by removing hard samples from the training process, adversarial training can be significantly improved without adding excessive computational cost. Overall, this study presents a novel perspective on improving adversarial training by focusing on data-centric approaches and effectively detecting and removing hard samples during the training procedure. The proposed method shows promising results in enhancing model robustness against adversarial perturbations.
AI-generated keywords: Adversarial Perturbations Machine Learning Models Adversarial Training Data-centric Approach Maximum Softmax Probability

AI-generated Key Points

  • Researchers address the issue of adversarial perturbations in machine learning models
  • Adversarial training is identified as an effective solution, where models are trained using adversarially perturbed samples
  • Existing model-based methods can be equivalent to applying smaller perturbation or optimization weights to hard training examples
  • Researchers propose a data-centric approach by detecting and removing hard samples directly from the training procedure
  • Maximum softmax probability is used for out-of-distribution detection to identify hard samples
  • Experiments conducted on SVHN and CIFAR-10 datasets show significant improvement in adversarial training by removing hard samples
  • The proposed method enhances model robustness against adversarial perturbations.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Abstract: Current machine learning models achieve super-human performance in many real-world applications. Still, they are susceptible against imperceptible adversarial perturbations. The most effective solution for this problem is adversarial training that trains the model with adversarially perturbed samples instead of original ones. Various methods have been developed over recent years to improve adversarial training such as data augmentation or modifying training attacks. In this work, we examine the same problem from a new data-centric perspective. For this purpose, we first demonstrate that the existing model-based methods can be equivalent to applying smaller perturbation or optimization weights to the hard training examples. By using this finding, we propose detecting and removing these hard samples directly from the training procedure rather than applying complicated algorithms to mitigate their effects. For detection, we use maximum softmax probability as an effective method in out-of-distribution detection since we can consider the hard samples as the out-of-distribution samples for the whole data distribution. Our results on SVHN and CIFAR-10 datasets show the effectiveness of this method in improving the adversarial training without adding too much computational cost.

Submitted to arXiv on 25 Jan. 2023

Tools for this paper

Explain it differently 3 free rewrites left today
Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Markdown (Notion, Obsidian) · Premium
Citation alerts Premium
Journal club Slides to present it Premium
For journalists Press kit and hype check Premium

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.

AI assistant instructions?

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.