# Removing Backdoor-Based Watermarks in Neural Networks with Limited Data

**Authors:** Xuankai Liu, Fengting Li, Bihan Wen, Qi Li  
**arXiv:** [2008.00407](https://arxiv.org/abs/2008.00407)  
**Date:** 2020-08-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2008.00407v1/

## Key points

- Deep neural networks are popular and successful in various fields
- Training deep models requires large amounts of data and computational resources
- There is a growing demand for trading trained deep models, which can be lucrative
- Current trading schemes pose risks related to copyright infringement and trustworthiness issues
- Watermarking techniques have been proposed to protect the intellectual property of deep models
- Backdoor-based watermarking is the most commonly used approach
- The robustness of watermarking methods has not been thoroughly evaluated under realistic conditions with limited availability of data and no knowledge about specific watermarking patterns
- The authors introduce a novel framework called WILD for removing backdoor-based watermarks from deep neural networks using only a small portion of training data
- The output model obtained after removing the watermarks performs just as well as models trained from scratch without any injected watermarks
- WILD leverages a novel data augmentation method and distribution alignment between normal and perturbed data in the feature space to ensure generalization across all trigger contents
- Experimental results demonstrate that WILD effectively removes watermarks without compromising performance when access to training data is limited

## Summary

Deep neural networks have become increasingly popular and successful in various fields. However, training these deep models requires large amounts of data and computational resources. As a result, there is a growing demand for trading trained deep models, which can be highly lucrative. Unfortunately, the current trading schemes pose risks related to copyright infringement and trustworthiness issues. For example, a sold model can be illegally resold to others without proper authorization, leading to significant profits for unauthorized individuals. To address this problem, researchers have proposed various watermarking techniques to protect the intellectual property of deep models. Among these techniques, backdoor-based watermarking is the most commonly used approach. However, the robustness of these watermarking methods has not been thoroughly evaluated under realistic conditions where there is limited availability of in-distribution data and no knowledge about the specific watermarking patterns. In this paper, the authors benchmark the robustness of watermarking techniques and introduce a novel framework called WILD (Watermark removal with Limited Data) for removing backdoor-based watermarks from deep neural networks using only a small portion of training data. The output model obtained after removing the watermarks performs just as well as models trained from scratch without any injected watermarks. The proposed WILD framework leverages a novel data augmentation method that mimics the behavior of watermark triggers. Additionally, it combines distribution alignment between normal and perturbed (e.g., occluded) data in the feature space to ensure generalization across all typical types of trigger contents. Experimental results demonstrate that WILD effectively removes watermarks without compromising the performance of deep models on their original tasks when access to training data is limited. This research contributes to addressing the challenges associated with protecting model intellectual property while ensuring trustworthiness in trading deep neural networks.