# Hypernetworks for Continual Semi-Supervised Learning

**Authors:** Dhanajit Brahma, Vinay Kumar Verma, Piyush Rai  
**arXiv:** [2110.01856](https://arxiv.org/abs/2110.01856)  
**Date:** 2021-10-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2110.01856v1/

## Key points

- Continual learning involves learning from sequentially arriving data with changing task distributions over time
- The authors propose a framework called Meta-Consolidation for Continual Semi-Supervised Learning (MCSSL)
- MCSSL utilizes a hypernetwork to learn the meta-distribution that generates the weights of a semi-supervised auxiliary classifier generative adversarial network (Semi-ACGAN) as the base network
- Knowledge of sequential tasks is consolidated in the hypernetwork while the base network learns the semi-supervised learning task
- A new benchmark dataset called Semi-Split CIFAR-10 is introduced for continual semi-supervised learning, obtained by modifying Split CIFAR-10 where tasks with labeled and unlabeled data arrive sequentially
- Experiments are conducted comparing MCSSL's performance with existing continual learning approaches on the Semi-Split CIFAR-10 benchmark dataset
- Model performance is assessed using average accuracy and average forgetting, including variations in the number of labeled data used during training
- Results show that MCSSL outperforms existing approaches on this dataset, demonstrating promising results
- This paper presents a novel framework and benchmark dataset for addressing semi-supervised continual learning problems.

## Summary

The paper discusses the problem of continual learning which involves learning from sequentially arriving data with changing task distributions over time. To address this issue, the authors propose a framework called Meta-Consolidation for Continual Semi-Supervised Learning (MCSSL). This framework utilizes a hypernetwork to learn the meta-distribution that generates the weights of a semi-supervised auxiliary classifier generative adversarial network (Semi-ACGAN) as the base network. The knowledge of sequential tasks is consolidated in the hypernetwork while the base network learns the semi-supervised learning task. Additionally, they introduce a new benchmark dataset called Semi-Split CIFAR-10 for continual semi-supervised learning. This dataset is obtained by modifying Split CIFAR-10 where tasks with labeled and unlabeled data arrive sequentially. To evaluate their approach, experiments are conducted comparing its performance with several existing continual learning approaches on the Semi-Split CIFAR-10 benchmark dataset. They measure average accuracy and average forgetting to assess model performance including variations in number of labeled data used during training. Results show that MCSSL model demonstrates promising results and outperforms existing approaches on this dataset. Overall, this paper presents a novel framework and benchmark dataset for addressing semi-supervised continual learning problems with promising results.