References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Learn to cycle: Time-consistent feature discovery for action recognition
(Submitted on 15 Jun 2020 (v1), last revised 23 Jun 2020 (this version, v2))
Abstract: Generalizing over temporal variations is a prerequisite for effective action recognition in videos. Despite significant advances in deep neural networks, it remains a challenge to focus on short-term discriminative motions in relation to the overall performance of an action. We address this challenge by allowing some flexibility in discovering relevant spatio-temporal features. We introduce Squeeze and Recursion Temporal Gates (SRTG), an approach that favors inputs with similar activations with potential temporal variations. We implement this idea with a novel CNN block that uses an LSTM to encapsulate feature dynamics, in conjunction with a temporal gate that is responsible for evaluating the consistency of the discovered dynamics and the modeled features. We show consistent improvement when using SRTG blocks, with only a minimal increase in the number of GFLOPs. On Kinetics-700, we perform on par with current state-of-the-art models, and outperform these on HACS, Moments in Time, UCF-101 and HMDB-51.
Submission history
From: Alexandros Stergiou MSc [view email][v1] Mon, 15 Jun 2020 09:36:28 GMT (6793kb,D)
[v2] Tue, 23 Jun 2020 14:06:36 GMT (6511kb,D)
Link back to: arXiv, form interface, contact.