# SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder

**Authors:** Dihan Zheng, Yihang Zou, Xiaowen Zhang, Chenglong Bao  
**arXiv:** [2403.17502](https://arxiv.org/abs/2403.17502)  
**Date:** 2024-03-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.17502v1/

## Key points

- Data bottleneck is a significant challenge in learning-based image restoration methods
- SeNM-VAE (Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder) is a novel approach to address this challenge
- SeNM-VAE leverages both paired and unpaired datasets to create realistic degraded data for training
- The method models the conditional distribution of degraded and clean images through a specially designed graphical model
- It operates within the variational inference framework with an objective function to handle both paired and unpaired data sets effectively
- SeNM-VAE can generate high-quality synthetic degraded images for tasks like image denoising and super-resolution
- It outperforms existing noise modeling methods that rely solely on unpaired data in producing synthetic degraded images
- SeNM-VAE demonstrates remarkable efficacy in downstream image restoration tasks even with limited paired data, showcasing superior performance on benchmark datasets like SIDD when provided with more paired data.

## Summary

In the realm of learning-based image restoration methods, the data bottleneck has emerged as a significant challenge. To combat this issue, researchers have explored the generation of synthesized training data using paired or unpaired samples. In response to this challenge, a novel approach known as SeNM-VAE (Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder) has been proposed. This method leverages both paired and unpaired datasets to create realistic degraded data for training purposes. The core concept behind SeNM-VAE involves modeling the conditional distribution of degraded and clean images through a specially designed graphical model. Operating within the variational inference framework, an objective function is developed to effectively handle both paired and unpaired data sets. By employing this method, paired training samples are generated for tasks such as real-world image denoising and super-resolution. One notable strength of the SeNM-VAE approach lies in its ability to produce high-quality synthetic degraded images when compared to other existing noise modeling methods that rely solely on unpaired data. Furthermore, the performance of SeNM-VAE extends beyond just generating synthetic data; it also demonstrates remarkable efficacy in downstream image restoration tasks even when faced with limited paired data. Moreover, when provided with more paired data for training, SeNM-VAE showcases superior performance on benchmark datasets like SIDD (Standard Image Denoising Dataset). The collaborative efforts of authors Dihan Zheng, Yihang Zou, Xiaowen Zhang, and Chenglong Bao have culminated in a cutting-edge methodology that not only addresses the data bottleneck in image restoration but also sets a new standard for semi-supervised noise modeling techniques.