# Structured Prompting: Scaling In-Context Learning to 1,000 Examples

**Authors:** Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, Furu Wei  
**arXiv:** [2212.06713](https://arxiv.org/abs/2212.06713)  
**Date:** 2022-12-13  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2212.06713v1/

## Key points

- Authors explore potential of large language models for zero- and few-shot performance without parameter updates
- Conventional in-context learning methods limited by length constraints for effective absorption of supervision from many examples
- Introduction of structured prompting as a novel approach to address length limitations and extend capabilities beyond few shots
- Structured prompting breaks length limit, enables scaling in-context learning to thousands of examples by encoding demonstration examples separately with position embeddings
- Adoption of structured prompting method allows scaling number of exemplars with linear complexity instead of quadratic complexity concerning length
- Experimental results demonstrate effectiveness in enhancing end-task performance and reducing evaluation variance compared to traditional in-context learning methods as number of demonstration examples increases

## Summary

In their paper titled "Structured Prompting: Scaling In-Context Learning to 1,000 Examples," authors Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei explore the potential of large language models in achieving impressive zero- and few-shot performance without the need for parameter updates. They highlight a common limitation in conventional in-context learning methods - the restriction imposed by length constraints that hinder effective absorption of supervision from a large number of examples. To address this challenge and extend the capabilities beyond few shots, the authors introduce structured prompting as a novel approach. This technique breaks the length limit and enables scaling in-context learning to thousands of examples. The key innovation lies in encoding demonstration examples separately with carefully designed position embeddings. These examples are then jointly attended by the test example through a rescaled attention mechanism. By adopting this structured prompting method, the authors demonstrate that it is possible to scale the number of exemplars with linear complexity instead of quadratic complexity concerning length. Experimental results across various tasks showcase the effectiveness of their approach in enhancing end-task performance and reducing evaluation variance compared to traditional in-context learning methods as the number of demonstration examples increases. The research findings provide valuable insights into advancing in-context learning capabilities for large language models. Interested readers can access the code for structured prompting at https://aka.ms/structured-prompting. Overall, this study sheds light on how structured prompting can significantly improve in-context learning by overcoming length constraints and scaling up to leverage supervision from a larger pool of examples effectively.