# Dataset Distillation with Neural Characteristic Function: A Minmax Perspective

**Authors:** Shaobo Wang, Yicun Yang, Zhiyuan Liu, Chenghao Sun, Xuming Hu, Conghui He, Linfeng Zhang  
**arXiv:** [2502.20653](https://arxiv.org/abs/2502.20653)  
**Date:** 2025-02-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.20653v1/

## Key points

- Dataset distillation is a popular method in deep learning for reducing data requirements.
- Existing distance metrics in distribution matching may not accurately capture distributional differences, leading to unreliable measures of discrepancy.
- A new approach reframes dataset distillation as a minmax optimization problem and introduces Neural Characteristic Function Discrepancy (NCFD) as a comprehensive metric for measuring distributional variances.
- NCFD leverages the Characteristic Function (CF) to encapsulate complete distributional information and utilizes a neural network to optimize the sampling strategy for the CF's frequency arguments.
- The proposed method, Neural Characteristic Function Matching (\mymethod{}), aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data to achieve a balance between realism and diversity in synthetic samples.
- Experimental results show significant performance improvements over state-of-the-art methods on datasets with varying resolutions, with a notable 20.5% accuracy boost observed on ImageSquawk dataset.
- The method reduces GPU memory usage by over 300 times and achieves processing speeds that are 20 times faster compared to existing techniques.
- The research marks a milestone achievement by accomplishing lossless compression of CIFAR-100 using only 2.3 GB of memory on a single NVIDIA 2080 Ti GPU, which has not been reported before in literature.

## Summary

, , , , 

In the field of deep learning, dataset distillation has become a popular method for reducing data requirements. Among the various techniques used, distribution matching-based approaches have gained attention for their efficient computational performance and strong results. However, existing distance metrics employed in distribution matching often fall short in accurately capturing distributional differences, leading to unreliable measures of discrepancy. To address this issue, a new approach is proposed in this paper that reframes dataset distillation as a minmax optimization problem. The authors introduce Neural Characteristic Function Discrepancy (NCFD), a comprehensive metric grounded in theory for measuring distributional variances. NCFD leverages the Characteristic Function (CF) to encapsulate complete distributional information and utilizes a neural network to optimize the sampling strategy for the CF's frequency arguments. This optimization process aims to maximize the discrepancy between distributions to enhance distance estimation accuracy. Furthermore, the authors minimize the distinction between real and synthetic data under this optimized NCFD measure. Their novel method, named Neural Characteristic Function Matching (\mymethod{}), aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data. This alignment helps achieve a balance between realism and diversity in synthetic samples. Experimental results demonstrate significant performance improvements over state-of-the-art methods on datasets with varying resolutions. Notably, there is a remarkable 20.5% accuracy boost observed on ImageSquawk dataset. Additionally, the proposed method reduces GPU memory usage by over 300 times and achieves processing speeds that are 20 times faster compared to existing techniques. Remarkably, this work marks a milestone achievement as it accomplishes lossless compression of CIFAR-100 using only 2.3 GB of memory on a single NVIDIA 2080 Ti GPU – an accomplishment not reported before in literature. Accepted by CVPR 2025 with 11 pages and 7 figures, this research showcases promising advancements in dataset distillation through Neural Characteristic Function Matching methodology developed by Shaobo Wang et al., which will be presented at Conference on Computer Vision and Pattern Recognition in 2025.