# Assemble Foundation Models for Automatic Code Summarization

**Authors:** Jian Gu, Pasquale Salza, Harald C. Gall  
**arXiv:** [2201.05222](https://arxiv.org/abs/2201.05222)  
**Date:** 2022-01-13  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2201.05222v2/

## Key points

- Authors propose a flexible and robust approach for automatic code summarization using neural models
- Code summarization reduces the need for manual writing in software development
- Reusing foundation models pretrained on massive data is more effective than customized models
- Authors assemble various foundation models into a single neural model called AdaMo
- Gaussian noise is used to optimize the latent representation of AdaMo
- Two adaptive schemes for knowledge transfer are proposed: continuous pretraining and intermediate finetuning
- Intermediate stage tasks are designed for general sequence-to-sequence learning
- AdaMo is evaluated against a benchmark dataset and outperforms state-of-the-art models in generating accurate code summaries
- The proposed approach improves software development processes by reducing manual writing efforts and enhancing code understanding through concise summaries.

## Summary

In their paper titled "Assemble Foundation Models for Automatic Code Summarization," authors Jian Gu, Pasquale Salza, and Harald C. Gall propose a flexible and robust approach for automatic code summarization using neural models. They highlight the benefits of code summarization in daily software development as it reduces the need for manual writing. The authors note that artificial intelligence is currently undergoing a paradigm shift, with foundation models pretrained on massive data and finetuned to downstream tasks surpassing specially customized models. Inspired by this trend, they suggest reusing foundation models instead of starting from scratch. To implement their approach, the authors assemble various foundation models like CodeBERT and GPT-2 into a single neural model called AdaMo. They also introduce Gaussian noise as a simulation of contextual information to optimize the latent representation. Additionally, they propose two adaptive schemes for knowledge transfer: continuous pretraining and intermediate finetuning. These schemes are complemented by the design of intermediate stage tasks for general sequence-to-sequence learning. The authors evaluate AdaMo against a benchmark dataset for code summarization and compare its performance with state-of-the-art models. Their findings demonstrate the effectiveness of AdaMo in generating accurate code summaries. Overall, this research contributes to advancing automatic code summarization techniques by leveraging existing foundation models and incorporating adaptive schemes for knowledge transfer. The proposed approach shows promise in improving software development processes by reducing manual writing efforts and enhancing code understanding through concise summaries.