# Guess the Instruction! Making Language Models Stronger Zero-Shot Learners

**Authors:** Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, Minjoon Seo  
**arXiv:** [2210.02969](https://arxiv.org/abs/2210.02969)  
**Date:** 2022-10-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.02969v1/

## Key points

- Authors propose Flipped Learning for meta-training language models (LMs) to improve zero-shot task generalization performance
- Meta-training involves fine-tuning LM on various downstream tasks
- Flipped Learning trains LM to generate task instruction given input instance and label
- Flipped selects label option most likely to generate task instruction during inference
- Evaluated Flipped on 14 tasks from BIG-bench benchmark
- Outperforms larger models T0-11B and 3-shot GPT-3 on average by 1.8% and 3.1% respectively
- Shows significant improvements on unseen labels, surpassing T0-11B by up to +20% average F1 score
- Code available at https://github.com/seonghyeonye/Flipped-Learning for further exploration and replication of findings
- Flipped Learning is a promising method for enhancing zero-shot task generalization in LMs

## Summary

In the paper "Guess the Instruction! Making Language Models Stronger Zero-Shot Learners," authors Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo propose a new method called Flipped Learning for meta-training language models (LMs) to improve their zero-shot task generalization performance. Meta-training involves fine-tuning an LM on various downstream tasks by maximizing the likelihood of the target label given the task instruction and input instance. While this approach has shown improvement in zero-shot task generalization, meta-trained LMs still struggle with challenging tasks that contain novel labels unseen during meta-training. Flipped Learning offers an alternative approach by training the LM to generate the task instruction given the input instance and label. During inference, the LM trained with Flipped Learning, referred to as Flipped, selects the label option that is most likely to generate the task instruction. The authors evaluate Flipped on 14 tasks from the BIG-bench benchmark. The results demonstrate that even though Flipped is only 3B-sized, it outperforms larger models such as T0-11B (4 times larger) and a 3-shot GPT-3 (175B) model (60 times larger) on average by 1.8% and 3.1%, respectively. Notably, Flipped shows significant improvements on unseen labels, surpassing T0-11B by up to +20% average F1 score. This indicates that Flipped's strong task generalization stems from its improved ability to generalize to novel labels. The authors have made their code available at https://github.com/seonghyeonye/Flipped-Learning for further exploration and replication of their findings. Overall, this paper introduces Flipped Learning as a promising method for enhancing zero-shot task generalization in language models. The experimental results highlight its effectiveness in handling challenging tasks with novel labels; showcasing its potential for improving LM performance in real world applications.