# Large Language Models Are Zero-Shot Time Series Forecasters

**Authors:** Nate Gruver, Marc Finzi, Shikai Qiu, Andrew Gordon Wilson  
**arXiv:** [2310.07820](https://arxiv.org/abs/2310.07820)  
**Date:** 2023-10-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.07820v1/

## Key points

- Large language models (LLMs) like GPT-3 and LLaMA-2 can be used for time series forecasting
- LLMs encode time series data as a string of numerical digits for next-token prediction in text
- LLMs can extrapolate time series data with zero-shot learning, performing comparably to purpose-built models
- Effective tokenization and conversion of discrete distributions enable flexible densities over continuous values
- LLMs naturally represent multimodal distributions, aligning well with repeated seasonal trends in time series datasets
- LLMs handle missing data without imputation through non-numerical text representation
- LLMs accommodate textual side information and provide explanations through question answering
- GPT-4 may perform worse than GPT-3 due to tokenization approach for numbers and poor uncertainty calibration
- Model design choices should be carefully considered for optimal performance

## Summary

In their paper titled "Large Language Models Are Zero-Shot Time Series Forecasters," authors Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew Gordon Wilson explore the potential of large language models (LLMs) such as GPT-3 and LLaMA-2 for time series forecasting. They propose a novel approach that encodes time series data as a string of numerical digits, allowing them to frame time series forecasting as next-token prediction in text. The authors find that LLMs can surprisingly extrapolate time series data with zero-shot learning, achieving performance levels comparable to or even surpassing purpose-built time series models trained specifically for downstream tasks. To enable this impressive performance, the authors introduce procedures for effectively tokenizing time series data and converting discrete distributions over tokens into highly flexible densities over continuous values. The success of LLMs in forecasting time series is attributed to their ability to naturally represent multimodal distributions. Additionally, these models exhibit biases towards simplicity and repetition which align well with the salient features often observed in many time series datasets such as repeated seasonal trends. Furthermore, the authors demonstrate how LLMs can handle missing data without requiring imputation through non-numerical text representation. They also show that these models can accommodate textual side information and provide explanations by answering questions related to their predictions. While increasing model size generally improves performance on time series forecasting tasks, the authors highlight an interesting finding: GPT-4 may perform worse than GPT-3 due to its tokenization approach for numbers and poor uncertainty calibration. These issues are likely a result of alignment interventions like Reinforcement Learning from Human Feedback (RLHF). Overall, this study showcases the potential of LLMs as zero-shot forecasters for time series data. The proposed approach not only achieves impressive performance but also offers flexibility in handling various aspects of time series analysis such as missing data and textual side information. However, careful consideration should be given to model design choices, as demonstrated by the potential drawbacks of GPT-4 compared to its predecessor.