# Mission: Impossible Language Models

**Authors:** Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, Christopher Potts  
**arXiv:** [2401.06416](https://arxiv.org/abs/2401.06416)  
**Date:** 2024-01-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.06416v1/

## Key points

- Study titled "Mission: Impossible Language Models" by Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts
- Lack of empirical evidence supporting claims that large language models (LLMs) can learn impossible languages
- Developed synthetic impossible languages with varying complexity to test GPT-2 small models
- GPT-2 struggled to grasp impossible languages compared to English as a control
- Challenges the idea that LLMs can effortlessly handle both possible and impossible linguistic structures
- Highlights limitations of current LLMs in learning complex linguistic patterns
- Emphasizes the need for further exploration into how different LLM architectures perform with various types of impossible languages

## Summary

In their study titled "Mission: Impossible Language Models," Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts delve into the debate surrounding large language models (LLMs) and their ability to learn both possible and impossible languages. The researchers address the lack of empirical evidence supporting claims that LLMs possess this capability by developing a series of synthetic impossible languages with varying levels of complexity. These constructed languages span a continuum of impossibility and were used to assess the capacity of GPT-2 small models to learn them at different stages of training. The key finding was that GPT-2 struggled to grasp these impossible languages compared to English as a control, challenging the notion that LLMs can effortlessly tackle both possible and impossible linguistic structures. This study not only sheds light on the limitations of current LLMs in learning complex linguistic patterns but also highlights the need for further exploration into how different LLM architectures perform when faced with various types of impossible languages. By pushing the boundaries of cognitive and typological investigations through experimentation with synthetic linguistic constructs, this research opens up new avenues for understanding the capabilities and constraints of language models in processing intricate linguistic phenomena.