# Experimenting with ChatGPT for Spreadsheet Formula Generation: Evidence of Risk in AI Generated Spreadsheets

**Authors:** Simon Thorne  
**arXiv:** [2309.00095](https://arxiv.org/abs/2309.00095)  
**Date:** 2023-08-31  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2309.00095v1/

## Key points

- Large Language Models (LLM) can interpret plain English sentences and generate complex computer programs in various modern languages.
- LLM tools make computer programming accessible to individuals regardless of their background or expertise.
- A study by Simon Thorne focused on ChatGPT's ability to generate valid spreadsheet formulae and computational outputs.
- ChatGPT demonstrated the capability to generate correct spreadsheet formulae with sound reasoning, deduction, and inference under certain circumstances.
- Challenges arose for ChatGPT when faced with limited information or overly complex problems, leading to diminished accuracy and capacity to reason effectively.
- Instances of producing false statements and "hallucinations" hindered the process of creating accurate spreadsheet formulae.
- Thorne's research highlights both the potential and limitations of using large language models like ChatGPT for tasks involving computational outputs.
- Caution is advised when relying on these models for critical tasks due to their susceptibility to inaccuracies under certain conditions.
- Further exploration into enhancing the robustness and reliability of such models is essential for maximizing their utility in practical applications.

## Summary

Large Language Models (LLM) have advanced to a level of sophistication where they can interpret plain English sentences and generate complex computer programs in various modern languages such as Python, Java Script, C++, and even spreadsheets. These tools offer powerful and accurate capabilities, making computer programming accessible to individuals regardless of their background or expertise. In a recent study by Simon Thorne titled "Experimenting with ChatGPT for Spreadsheet Formula Generation: Evidence of Risk in AI Generated Spreadsheets," the focus was on exploring the potential of ChatGPT in generating valid spreadsheet formulae and computational outputs. The experiments aimed to assess ChatGPT's ability to deduce, infer, and problem-solve answers within the context of creating spreadsheet formulae. The findings revealed that under certain circumstances, ChatGPT demonstrated the capability to generate correct spreadsheet formulae supported by sound reasoning, deduction, and inference. However, challenges arose when faced with limited information or overly complex problems. In these scenarios, the accuracy of ChatGPT diminished along with its capacity to reason effectively. This led to instances of producing false statements and "hallucinations," ultimately hindering the process of creating accurate spreadsheet formulae. Thorne's research sheds light on both the potential and limitations of utilizing large language models like ChatGPT for tasks involving computational outputs. While these tools show promise in simplifying programming processes for a wider audience, caution must be exercised when relying on them for critical tasks due to their susceptibility to inaccuracies under certain conditions. Further exploration into enhancing the robustness and reliability of such models is essential for maximizing their utility in practical applications.