# Steering Llama 2 via Contrastive Activation Addition

**Authors:** Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, Alexander Matt Turner  
**arXiv:** [2312.06681](https://arxiv.org/abs/2312.06681)  
**Date:** 2023-12-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.06681v4/

## Key points

- Researchers Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner introduce Contrastive Activation Addition (CAA) for guiding language models.
- CAA involves adjusting activations during forward passes by computing "steering vectors" comparing residual stream activations between positive and negative examples of specific behaviors.
- Steering vectors are added to user prompts with positive or negative coefficients to control desired behavior.
- CAA significantly influences model behavior while minimally impacting overall capabilities when evaluated on Llama 2 Chat with various behavioral question datasets and open-ended generation tasks.
- Insights into CAA's mechanisms are gained through different activation space interpretation methods.
- The research demonstrates how CAA accurately guides model outputs and provides insights into the representation of high-level concepts in Large Language Models (LLMs).
- An example illustrating the impact of sycophancy steering using CAA on open-ended generation tasks with Llama 2 7B at layer 13 using specific multipliers is provided in Table 1.
- Diagrams illustrate the process of generating steering vectors for CAA and applying them during inference to effectively control model behavior.
- Overall, this research highlights Contrastive Activation Addition as an innovative approach to steer language models towards desired behaviors and its significance in understanding Large Language Models.

## Summary

In their study "Steering Llama 2 via Contrastive Activation Addition," researchers Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner introduce the novel method of Contrastive Activation Addition (CAA) for guiding language models. CAA involves adjusting activations during forward passes by computing "steering vectors" that compare residual stream activations between positive and negative examples of specific behaviors. These steering vectors are then added to user prompts with either a positive or negative coefficient to finely control desired behavior. The effectiveness of CAA is evaluated on Llama 2 Chat using various behavioral question datasets and open-ended generation tasks. Results show that CAA significantly influences model behavior while minimally impacting overall capabilities. Insights into CAA's mechanisms are gained through different activation space interpretation methods. The research also demonstrates how CAA accurately guides model outputs and provides valuable insights into the representation of high-level concepts in Large Language Models (LLMs). Additionally, an example showcasing the impact of sycophancy steering using CAA on open-ended generation tasks with Llama 2 7B at layer 13 using specific multipliers is illustrated in Table 1. The study includes diagrams illustrating the process of generating steering vectors for CAA and applying them during inference to effectively control model behavior. Overall, this research highlights the potential of Contrastive Activation Addition as an innovative approach to steer language models towards desired behaviors and sheds light on its significance in understanding the inner workings of Large Language Models.