# Multi-task, multi-label and multi-domain learning with residual convolutional networks for emotion recognition

**Authors:** Gerard Pons, David Masip  
**arXiv:** [1802.06664](https://arxiv.org/abs/1802.06664)  
**Date:** 2018-02-19  
**Summary page:** https://www.summarizepaper.com/en/paper/1802.06664v1/

## Key points

- Automated emotion recognition in the wild from facial images is challenging due to changes in pose, orientation, and point of view.
- Deep Learning has made significant breakthroughs in this area, but current approaches are still affected by these challenges.
- The paper proposes a multi-task learning approach that utilizes a common feature representation shared with other related tasks.
- Emotion recognition can benefit from jointly learning a model with a detector of facial Action Units (collective muscle movements).
- A multi-task learning loss function is applied to tackle the problem of learning multiple tasks with heterogeneously labeled data.
- The proposed method is validated using two datasets acquired in non-controlled environments.
- An application to predict compound facial emotion expressions is presented, showing promise in real-world scenarios.
- This research contributes to advancing automated emotion recognition from facial images and improving deep learning algorithms for this task.

## Summary

Automated emotion recognition in the wild from facial images is a challenging problem due to strong changes in pose, orientation, and point of view. Recent advances in Deep Learning have made significant breakthroughs in this area; however, current approaches are still severely affected by these challenges. To address these issues, this paper proposes a multi-task learning approach that utilizes a common feature representation shared with other related tasks. Specifically, the authors demonstrate that emotion recognition can benefit from jointly learning a model with a detector of facial Action Units (collective muscle movements). A multi-task learning loss function is applied to tackle the problem of learning multiple tasks with heterogeneously labeled data and improve upon previous multi-task approaches. The effectiveness of the proposed method is validated using two datasets acquired in non-controlled environments. An application to predict compound facial emotion expressions is also presented which shows promise in overcoming variations encountered in real-world scenarios. This research contributes to advancing automated emotion recognition from facial images and provides insights into improving deep learning algorithms for this task.