Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Using Self-Supervised Auxiliary Tasks to Improve Fine-Grained Facial Representation
(Submitted on 13 May 2021 (v1), last revised 8 Aug 2022 (this version, v3))
Abstract: In this paper, at first, the impact of ImageNet pre-training on fine-grained Facial Emotion Recognition (FER) is investigated which shows that when enough augmentations on images are applied, training from scratch provides better result than fine-tuning on ImageNet pre-training. Next, we propose a method to improve fine-grained and in-the-wild FER, called Hybrid Multi-Task Learning (HMTL). HMTL uses Self-Supervised Learning (SSL) as an auxiliary task during classical Supervised Learning (SL) in the form of Multi-Task Learning (MTL). Leveraging SSL during training can gain additional information from images for the primary fine-grained SL task. We investigate how proposed HMTL can be used in the FER domain by designing two customized version of common pre-text task techniques, puzzling and in-painting. We achieve state-of-the-art results on the AffectNet benchmark via two types of HMTL, without utilizing pre-training on additional data. Experimental results on the common SSL pre-training and proposed HMTL demonstrate the difference and superiority of our work. However, HMTL is not only limited to FER domain. Experiments on two types of fine-grained facial tasks, i.e., head pose estimation and gender recognition, reveals the potential of using HMTL to improve fine-grained facial representation.
Submission history
From: Mahdi Pourmirzaei [view email][v1] Thu, 13 May 2021 16:56:36 GMT (1005kb)
[v2] Thu, 19 Aug 2021 18:10:15 GMT (953kb)
[v3] Mon, 8 Aug 2022 17:48:09 GMT (1048kb)
Link back to: arXiv, form interface, contact.