# Studying Large Language Model Generalization with Influence Functions

**Authors:** Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilė Lukošiūtė, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, Samuel R. Bowman  
**arXiv:** [2308.03296](https://arxiv.org/abs/2308.03296)  
**Date:** 2023-08-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.03296v1/

## Key points

- Researchers used influence functions to understand generalization properties of large language models (LLMs)
- Influence functions help identify training examples with significant impact on model behavior
- EK-FAC approximation method demonstrated similar accuracy to traditional estimators while reducing computation time for IHVP calculations
- Algorithmic techniques like TF-IDF filtering and query batching were explored to streamline gradient computations for candidate training sequences
- Study investigated various aspects of LLM generalization patterns, including sparsity of influence patterns, cross-lingual generalization, and role-playing behavior
- One surprising limitation identified: influences decay close to zero when key phrase order is flipped

## Summary

In the study titled "Studying Large Language Model Generalization with Influence Functions," researchers Roger Grosse, Juhan Bae, Cem Anil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger,
Kamilė Lukošiūtė,Karina Nguyen,Nicholas Joseph,Sam McCandlish,Jared Kaplan,and Samuel R. Bowman delve into the use of influence functions to gain insight into the generalization properties of large language models (LLMs). The research focuses on understanding and mitigating risks associated with machine learning models by identifying which training examples have the most significant impact on a model's behavior. The researchers introduce influence functions as a tool to determine how a model's parameters and outputs would change if a specific sequence were added to the training set. Through experiments using EK-FAC approximation method,the team demonstrates that this approach achieves similar accuracy to traditional influence function estimators while significantly reducing computation time for IHVP calculations. They also explore algorithmic techniques such as TF-IDF filtering and query batching to streamline the process of computing gradients for candidate training sequences. The study investigates various aspects of LLM generalization patterns using influence functions including sparsity of influence patterns, abstraction levels with scale increase in math and programming abilities cross-lingual generalization and role-playing behavior. Despite observing sophisticated forms of generalization in LLMs through these analyses, one surprising limitation is identified: influences decay close to zero when key phrase order is flipped. Overall,this research showcases how influence functions serve as a powerful tool for studying and understanding the generalization properties of large language models like never before.With 119 pages containing 47 figures and 22 tables in their comprehensive analysis published on arXiv.org under ID 2308.03296v1., these findings provide valuable insights for advancing our knowledge of machine learning model behaviors.