# Chasing Better Deep Image Priors between Over- and Under-parameterization

**Authors:** Qiming Wu, Xiaohan Chen, Yifan Jiang, Zhangyang Wang  
**arXiv:** [2410.24187](https://arxiv.org/abs/2410.24187)  
**Date:** 2024-10-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.24187v1/

## Key points

- Deep Neural Networks (DNNs) are recognized for serving as over-parameterized deep image priors (DIP) in deep learning.
- Compact, under-parameterized image priors like deep decoders show impressive performance in image restoration tasks with a compromise in accuracy.
- The concept of "lottery image prior" (LIP) is proposed to find intermediate parameterized image priors that balance performance, efficiency, and transferability.
- LIP leverages sparsity within DNNs to identify sparse subnetworks within over-parameterized DIPs that match the original DNN's performance in various image inverse problems.
- Identified LIP subnetworks outperform deep decoders while maintaining comparable model sizes and demonstrate high transferability across different images and restoration tasks.
- LIP concept extends to compressive sensing image reconstruction using a pre-trained GAN generator as the prior, confirming its validity in diverse settings.
- This research demonstrates the relevance of the lottery ticket hypothesis (LTH) in inverse problems or image priors, offering new possibilities for optimizing deep learning models for enhanced efficiency and performance in image restoration applications.

## Summary

In the realm of deep learning, <kw> Deep Neural Networks (DNNs) </kw> have been widely recognized for their ability to serve as over-parameterized deep image priors (DIP), effectively regularizing various image inverse problems. On the other end of the spectrum, researchers have also introduced remarkably compact, under-parameterized image priors such as deep decoders, which demonstrate impressive performance in image restoration tasks despite a slight compromise in accuracy. This dichotomy has prompted a critical question: is there an optimal solution that lies somewhere between these two extremes? Can we identify "intermediate" parameterized image priors that strike a balance between performance, efficiency, and maintaining strong transferability? Drawing inspiration from the <kw> lottery ticket hypothesis (LTH) </kw>, a novel concept known as "lottery image prior" (LIP) is proposed and investigated. The LIP leverages the inherent sparsity within DNNs and posits that within an over-parameterized DNN-based image prior, there exists a sparse subnetwork that can be isolated and trained to match the performance of the original DNN when applied as a prior in various image inverse problems. The results of this study validate the effectiveness of LIPs, showcasing the successful identification of sparse subnetworks within over-parameterized DIPs across significant sparsity ranges. These identified LIP subnetworks exhibit superior performance compared to deep decoders while maintaining comparable model sizes. They not only preserve the efficacy of their over-parameterized counterparts but also demonstrate high transferability across different images and types of restoration tasks. Furthermore, the concept of LIP is extended to compressive sensing image reconstruction by utilizing a pre-trained GAN generator as the prior, further confirming its validity in diverse settings. This groundbreaking research marks the first demonstration of the relevance of LTH in the context of inverse problems or image priors. Published in Transactions on Machine Learning Research (08/2023), this study sheds light on new possibilities for optimizing deep learning models for enhanced efficiency and performance in image restoration applications.