# BloombergGPT: A Large Language Model for Finance

**Authors:** Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, Gideon Mann  
**arXiv:** [2303.17564](https://arxiv.org/abs/2303.17564)  
**Date:** 2023-03-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.17564v1/

## Key points

- Financial technology heavily relies on natural language processing (NLP)
- Large Language Models (LLMs) have been effective in various applications
- No LLM specialized for the financial domain has been reported in literature
- A team of researchers present BloombergGPT: a 50 billion parameter language model trained on an extensive range of financial data
- The team constructed a massive dataset consisting of 363 billion tokens based on Bloomberg's vast data sources augmented with 345 billion tokens from general-purpose datasets
- This mixed dataset training led to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks
- The team validated BloombergGPT using standard LLM benchmarks and open financial benchmarks as well as a suite of internal benchmarks designed to accurately reflect their intended usage
- The authors explain their modeling choices, training process, and evaluation methodology in detail
- They plan to release training logs (Chronicles) detailing their experience in training BloombergGPT.

## Summary

The field of financial technology relies heavily on natural language processing (NLP) for tasks such as sentiment analysis, named entity recognition, and question answering. While Large Language Models (LLMs) have proven effective in various applications, no LLM specialized for the financial domain has been reported in literature. In response to this gap, a team of researchers including Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg and Gideon Mann present BloombergGPT: a 50 billion parameter language model trained on an extensive range of financial data. The team constructed a massive dataset consisting of 363 billion tokens based on Bloomberg's vast data sources augmented with 345 billion tokens from general-purpose datasets. This mixed dataset training led to a model that outperforms existing models on financial tasks by significant margins without sacrificing performance on general LLM benchmarks. The team validated BloombergGPT using standard LLM benchmarks and open financial benchmarks as well as a suite of internal benchmarks designed to accurately reflect their intended usage. Additionally, the authors explain their modeling choices, training process and evaluation methodology in detail. As the next step in their research journey, they plan to release training logs (Chronicles) detailing their experience in training BloombergGPT. Overall, this work presents an essential contribution to the field of NLP in finance and provides a valuable resource for future research endeavors in this area.