# Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentation

**Authors:** Lei Zhou, Huidong Liu, Joseph Bae, Junjun He, Dimitris Samaras, Prateek Prasanna  
**arXiv:** [2203.05573](https://arxiv.org/abs/2203.05573)  
**Date:** 2022-03-10  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2203.05573v2/

## Key points

- Masked Autoencoder (MAE) is an effective pre-training method for Vision Transformers (ViT) in natural image analysis
- MAE enables ViT encoder to aggregate contextual information and infer masked image regions, crucial in medical image domain
- Self-pretraining approach using MAE for medical image analysis tasks due to lack of ImageNet-scale medical image dataset
- MAE self-pretraining significantly enhances medical image tasks such as chest X-ray disease classification, abdominal CT multi-organ segmentation, and MRI brain tumor segmentation
- ViT with MAE self-pretraining outperforms state-of-the-art CNN-based models utilizing ImageNet pre-training and other self-supervised pre-training methods like MoCo and LSAE
- MAE self-pretraining shows substantial improvements in abdomen multi-organ segmentation compared to UNETR baseline model
- Superior performance indicated by average Dice Similarity Coefficient (DSC) scores with increasing training data sizes, highlighting effectiveness of the proposed approach in enhancing segmentation accuracy

## Summary

Masked Autoencoder (MAE) has proven to be an effective pre-training method for Vision Transformers (ViT) in natural image analysis. By reconstructing full images from partially masked inputs, MAE enables the ViT encoder to aggregate contextual information and infer masked image regions. This is particularly crucial in the medical image domain where anatomical structures are interconnected. Due to the lack of an ImageNet-scale medical image dataset for pre-training, researchers have turned to a self-pretraining approach using MAE for medical image analysis tasks. This method pre-trains a ViT on the training set of the target data, making it beneficial for scenarios where acquiring pre-training data is challenging. Experimental results demonstrate that MAE self-pretraining significantly enhances various medical image tasks such as chest X-ray disease classification, abdominal CT multi-organ segmentation, and MRI brain tumor segmentation. The study conducted by Lei Zhou et al. shows promising outcomes in improving performance across these tasks. Comparisons with state-of-the-art CNN-based models utilizing ImageNet pre-training and self-supervised pre-training methods like MoCo and LSAE reveal that ViT with MAE self-pretraining outperforms them all. Specifically focusing on abdomen multi-organ segmentation results presented in Table 1 show substantial improvements achieved through MAE self-pretraining compared to the UNETR baseline model. The average Dice Similarity Coefficient (DSC) scores indicate superior performance with increasing training data sizes and highlight the effectiveness of the proposed approach in enhancing segmentation accuracy. In conclusion, this research demonstrates that MAE self-pretraining with Vision Transformers holds great potential for advancing medical image analysis tasks by leveraging context aggregation abilities essential for understanding complex anatomical structures. The findings underscore the importance of tailored pre-training strategies in domains where large-scale datasets are limited and showcase significant performance gains across diverse medical imaging applications.