# On the Efficacy of Differentially Private Few-shot Image Classification

**Authors:** Marlon Tobaben, Aliaksandra Shysheya, John Bronskill, Andrew Paverd, Shruti Tople, Santiago Zanella-Beguelin, Richard E Turner, Antti Honkela  
**arXiv:** [2302.01190](https://arxiv.org/abs/2302.01190)  
**Date:** 2023-02-02  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2302.01190v2/

## Key points

- The paper discusses training differentially private (DP) models for image classification
- DP models aim to achieve accuracy comparable to non-private models while preserving privacy
- DP models are typically pretrained on public datasets and fine-tuned on private downstream datasets
- Few-shot DP is important for applications like personalization and federated learning
- The authors conducted experiments to understand the effectiveness of few-shot DP under various conditions
- Increasing shots per class is necessary to achieve DP accuracy on par with non-private models as privacy level increases
- Learning parameter-efficient FiLM adapters under DP is competitive or superior to other approaches
- DP federated learning systems achieved state-of-the-art performance on the FLAIR benchmark
- The research provides insights into implementing few-shot DP image classification effectively in privacy-sensitive scenarios.

## Summary

The paper titled "On the Efficacy of Differentially Private Few-shot Image Classification" discusses the recent progress in training differentially private (DP) models for image classification. These DP models aim to achieve accuracy comparable to non-private models while preserving privacy. Typically, these models are pretrained on large public datasets and then fine-tuned on private downstream datasets that have similar distributions. However, in certain applications such as personalization and federated learning, it is crucial for DP models to perform well in the few-shot setting, where obtaining large amounts of labeled data is challenging. Additionally, these models should be effective across a wide variety of domains for specialist settings. To understand the conditions under which few-shot DP can be effective, the authors conducted an exhaustive set of experiments. They analyzed how the accuracy and vulnerability to attacks of few-shot DP image classification models are affected by various factors including the number of shots per class, privacy level, model architecture, downstream dataset, and subset of learnable parameters. The results showed that achieving DP accuracy on par with non-private models requires increasing the shots per class as the privacy level increases. For example, at ε=1 (a common measure of privacy level), the shots per class may need to be increased by 20-35 times. The study also found that learning parameter-efficient FiLM adapters under DP is competitive with or even superior to learning just the final classifier layer or all network parameters. Furthermore, the authors evaluated DP federated learning systems and achieved state-of-the-art performance on the challenging FLAIR benchmark. Overall, this research provides insights into how few-shot DP image classification can be effectively implemented and highlights important considerations for achieving accurate and secure classification in privacy-sensitive scenarios.