# BiLLM: Pushing the Limit of Post-Training Quantization for LLMs

**Authors:** Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, Xiaojuan Qi  
**arXiv:** [2402.04291](https://arxiv.org/abs/2402.04291)  
**Date:** 2024-02-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.04291v1/

## Key points

- Authors introduce 1-bit post-training quantization scheme for pretrained large language models (LLMs)
- BiLLM addresses challenge of reducing memory and computational resources while maintaining high performance
- Technique identifies and selects salient weights based on weight distribution of LLMs
- Minimizes compression loss through binary residual approximation strategy
- Optimal splitting search accurately groups and binarizes non-salient weights with bell-shaped distribution
- Achieves high-accuracy inference with only 1.08-bit weights across various LLM families and evaluation metrics
- Outperforms state-of-the-art quantization methods for LLMs significantly
- Completes binarization process of LLM with 7 billion weights in just 0.5 hours on a single GPU

## Summary

In their paper titled "BiLLM: Pushing the Limit of Post-Training Quantization for LLMs," authors Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi introduce a groundbreaking 1-bit post-training quantization scheme tailored specifically for pretrained large language models (LLMs). This novel technique addresses the challenge of reducing memory and computational resources required by LLMs while maintaining high performance. <br>
The authors identify binarization as a powerful compression technology that can drastically reduce model weights to just 1 bit. However, existing quantization techniques have struggled to maintain LLM performance under ultra-low bit-widths. In response to this issue, the authors present BiLLM which first identifies and selects salient weights based on the weight distribution of LLMs. They then minimize compression loss through an effective binary residual approximation strategy. Additionally,<br>
an optimal splitting search is proposed to accurately group and binarize non-salient weights with a bell-shaped distribution.<br>
The results of their study demonstrate that BiLLM achieves high-accuracy inference with only 1.08-bit weights across various LLM families and evaluation metrics. For instance,<br>
it achieved a perplexity of 8.41 on LLaMA2-70B.<br>
Furthermore,<br>
BiLLM outperforms state-of-the-art quantization methods for LLMs by significant margins.<br>
Notably,<br>
the efficiency of BiLLM is highlighted by its ability to complete the binarization process of an LLM with 7 billion weights in just 0.5 hours on a single GPU.<br> 
Overall,<br> 
BiLLM represents a significant advancement in post-training quantization techniques for LLMs, showcasing impressive performance while simultaneously reducing computational demands and memory requirements effectively.