FAU, Facial Expressions, Valence and Arousal: A Multi-task Solution

Abstract: In the paper, we aim to train a unified model that performs three tasks: Facial Action Units (FAU) prediction, seven basic facial expressions prediction, as well as valence and arousal prediction. The main challenge of this task is the lack of fully-annotated dataset. Most of existing datasets only contain one or two types of labels. To tackle this challenge, we propose an algorithm for the multitask model to learn from partial labels. The algorithm has two steps: first, we train a teacher model to perform all three tasks, where each instance is trained by the ground truth label of its corresponding task. Second, we refer to the outputs of the teacher model as the soft labels. We use the soft labels and the ground truths to train the student model. We find that the student model outperforms the teacher model on all the tasks, possibly due to the exposure to the full set of labels. Finally, we use ensemble modeling to boost the performance further on the three tasks.