# SaulLM-7B: A pioneering Large Language Model for Law

**Authors:** Pierre Colombo, Telmo Pessoa Pires, Malik Boudiaf, Dominic Culver, Rui Melo, Caio Corro, Andre F. T. Martins, Fabrizio Esposito, Vera Lúcia Raposo, Sofia Morgado, Michael Desa  
**arXiv:** [2403.03883](https://arxiv.org/abs/2403.03883)  
**Date:** 2024-03-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.03883v1/

## Key points

- Introduction of SaulLM-7B, a large language model tailored for the legal domain
- Built on Mistral 7B architecture with 7 billion parameters and trained on a vast English legal corpus
- Novel instructional fine-tuning method using legal datasets to enhance performance in legal tasks
- Release of SaulLM-7B and SaulLM-7B-Instruct under the MIT License to encourage adoption and innovation
- Focus on extending legal capabilities of language models by selecting Mistral 7B model known for high performance
- Two-step process employed to enhance Mistral's abilities in handling legal text effectively

## Summary

In this paper, titled "SaulLM-7B: A pioneering Large Language Model for Law," Pierre Colombo and a team of researchers introduce SaulLM-7B, a large language model (LLM) specifically designed for the legal domain. With an impressive 7 billion parameters, SaulLM-7B stands out as the first LLM tailored for legal text comprehension and generation. It is built on the Mistral 7B architecture and trained on an extensive English legal corpus containing over 30 billion tokens. This showcases cutting-edge proficiency in understanding and processing legal documents. The researchers also present a novel instructional fine-tuning method that utilizes legal datasets to further enhance SaulLM-7B's performance in legal tasks. To encourage widespread adoption and foster innovation within the legal domain and beyond, SaulLM-7B and its instructional variant, SaulLM-7B-Instruct, along with their evaluation code, are released under the MIT License. This open licensing approach aims to facilitate collaborative development and integration into various commercial and research initiatives. Moreover, the study delves into extending the legal capabilities of language models by selecting the Mistral 7B model with its 7 billion parameters known for achieving high performance across benchmarks and tasks. The methodology employed involves a two-step process aimed at enhancing Mistral's abilities in handling legal text effectively. Overall, this work contributes significantly to advancing language models' capabilities in comprehending and generating legal text while promoting collaboration and innovation through open licensing practices.