# Meta-Learning: A Survey

**Authors:** Joaquin Vanschoren  
**arXiv:** [1810.03548](https://arxiv.org/abs/1810.03548)  
**Date:** 2018-10-08  
**Summary page:** https://www.summarizepaper.com/en/paper/1810.03548v1/

## Key points

- Meta-learning, or learning to learn, focuses on observing the performance of different machine learning approaches on various tasks
- Analyzing this experience enables more efficient and quick learning of new tasks compared to traditional methods
- Meta-learning speeds up and enhances the design of machine learning pipelines and neural architectures
- Hand-engineered algorithms can be replaced with data-driven approaches through meta-learning
- Meta-learning is based on the idea that humans build upon previously learned skills and experiences when acquiring new skills
- Collecting and utilizing meta-data from prior learning tasks and models is a challenge in meta-learning
- Meta-data includes algorithm configurations, hyperparameter settings, pipeline compositions, network architectures, model evaluations, learned model parameters, and measurable properties of the task itself (meta-features)
- Knowledge extracted from prior meta-data guides the search for optimal models for new tasks
- The effectiveness of leveraging prior experience depends on the similarity between previous tasks and the new task at hand
- Real-world tasks often provide ample opportunities for learning from past experiences
- Different types of meta-learning techniques are categorized based on the type of meta-data they leverage
- Techniques range from those solely relying on model evaluations to more task-specific approaches
- The chapter serves as a comprehensive survey of meta-learning techniques and highlights its potential impact on improving machine learning design and decision making processes.

## Summary

Meta-learning, also known as learning to learn, is a field of study that focuses on systematically observing the performance of different machine learning approaches on various tasks. By analyzing this experience, or meta-data, meta-learning enables the ability to learn new tasks more efficiently and quickly than traditional methods. This not only speeds up and enhances the design of machine learning pipelines and neural architectures but also allows for the replacement of hand-engineered algorithms with novel data-driven approaches. In this chapter, the author provides an overview of the state-of-the-art in meta-learning. The concept of meta-learning is rooted in the idea that when humans learn new skills, they build upon previously learned skills and experiences. Similarly, when building machine learning models, prior experience with related tasks can be leveraged to make informed decisions. The challenge in meta-learning lies in collecting and utilizing meta-data from prior learning tasks and models. This includes information such as algorithm configurations, hyperparameter settings, pipeline compositions or network architectures, model evaluations (e.g., accuracy and training time), learned model parameters (e.g., weights of a neural net), and measurable properties of the task itself (known as meta-features). By extracting knowledge from this prior meta-data, it becomes possible to guide the search for optimal models for new tasks. The term "meta-learning" encompasses various approaches based on prior experience with other tasks. The effectiveness of leveraging prior experience depends on the similarity between previous tasks and the new task at hand. While there are limitations to leveraging prior experience in cases where tasks are unrelated or involve random noise, real-world tasks often provide ample opportunities for learning from past experiences. Throughout the chapter, different types of meta-learning techniques are categorized based on the type of meta-data they leverage. These techniques range from those that solely rely on model evaluations to more task specific approaches. Overall, this chapter serves as a comprehensive survey of meta-learning techniques and highlights its potential impact on improving machine learning design and decision making processes.