References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: MTGLS: Multi-Task Gaze Estimation with Limited Supervision
(Submitted on 23 Oct 2021 (v1), last revised 13 Dec 2021 (this version, v2))
Abstract: Robust gaze estimation is a challenging task, even for deep CNNs, due to the non-availability of large-scale labeled data. Moreover, gaze annotation is a time-consuming process and requires specialized hardware setups. We propose MTGLS: a Multi-Task Gaze estimation framework with Limited Supervision, which leverages abundantly available non-annotated facial image data. MTGLS distills knowledge from off-the-shelf facial image analysis models, and learns strong feature representations of human eyes, guided by three complementary auxiliary signals: (a) the line of sight of the pupil (i.e. pseudo-gaze) defined by the localized facial landmarks, (b) the head-pose given by Euler angles, and (c) the orientation of the eye patch (left/right eye). To overcome inherent noise in the supervisory signals, MTGLS further incorporates a noise distribution modelling approach. Our experimental results show that MTGLS learns highly generalized representations which consistently perform well on a range of datasets. Our proposed framework outperforms the unsupervised state-of-the-art on CAVE (by 6.43%) and even supervised state-of-the-art methods on Gaze360 (by 6.59%) datasets.
Submission history
From: Shreya Ghosh [view email][v1] Sat, 23 Oct 2021 00:20:23 GMT (5691kb,D)
[v2] Mon, 13 Dec 2021 13:07:14 GMT (286kb,D)
Link back to: arXiv, form interface, contact.