# Nova$^+$: Generative Language Models for Binaries

**Authors:** Nan Jiang, Chengxiao Wang, Kevin Liu, Xiangzhe Xu, Lin Tan, Xiangyu Zhang  
**arXiv:** [2311.13721](https://arxiv.org/abs/2311.13721)  
**Date:** 2023-11-22  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2311.13721v1/

## Key points

- Challenges faced by large language models (LLMs) in modeling and learning binary code:
- Handling hex-decimal values
- Complex global dependencies
- Compiler optimization levels
- Nova pre-trained on binary corpora
- Initially pre-trained using standard language modeling tasks
- Demonstrates superior performance on BCSD, BCT, and BCR compared to GPT-3.5 and other techniques
- Nova$^+ developed as an enhancement to Nova
- Incorporates two new pre-training tasks: optimization generation and optimization level prediction
- Nova outperforms existing methods on five benchmarks across three downstream tasks
- Nova$^+$ outperforms all other methods across all three downstream tasks on five benchmarks
- Specialized training for LLMs when dealing with binary code is crucial
- Innovative approaches like Nova$^+ significantly improve LLM capabilities in various binary-related tasks

## Summary

In their paper titled "Nova$^+$: Generative Language Models for Binaries," authors Nan Jiang, Chengxiao Wang, Kevin Liu, Xiangzhe Xu, Lin Tan, and Xiangyu Zhang discuss the challenges faced by large language models (LLMs) in modeling and learning binary code. Existing LLMs have shown effectiveness in tasks like code generation and program repair but are primarily focused on source code. The main obstacles for LLMs when it comes to binary code include handling hex-decimal values, complex global dependencies, and compiler optimization levels. To address these challenges and bring the benefits of LLMs to the binary domain, the authors introduce Nova and Nova$^+$ - LLMs pre-trained on binary corpora. Nova is initially pre-trained using standard language modeling tasks and demonstrates superior performance on five benchmarks across three downstream tasks: binary code similarity detection (BCSD), binary code translation (BCT), and binary code recovery (BCR) compared to GPT-3.5 and other existing techniques. Furthermore, Nova$^+$ is developed as an enhancement to Nova by incorporating two new pre-training tasks: optimization generation and optimization level prediction. These tasks are specifically designed to improve the model's understanding of binary optimization techniques and align equivalent binaries. The results show that Nova$^+$ outperforms all other methods across all three downstream tasks on five benchmarks, highlighting the significance of these new pre-training strategies in enhancing LLM performance for binary-related applications. Overall, the study underscores the importance of specialized training for LLMs when dealing with binary code and showcases how innovative approaches like Nova$^+ can significantly improve their capabilities in various binary-related tasks.