We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Abstract: Deep neural networks based purely on attention have been successful across several domains, relying on minimal architectural priors from the designer. In Human Action Recognition (HAR), attention mechanisms have been primarily adopted on top of standard convolutional or recurrent layers, improving the overall generalization capability. In this work, we introduce Action Transformer (AcT), a simple, fully self-attentional architecture that consistently outperforms more elaborated networks that mix convolutional, recurrent and attentive layers. In order to limit computational and energy requests, building on previous human action recognition research, the proposed approach exploits 2D pose representations over small temporal windows, providing a low latency solution for accurate and effective real-time performance. Moreover, we open-source MPOSE2021, a new large-scale dataset, as an attempt to build a formal training and evaluation benchmark for real-time, short-time HAR. The proposed methodology was extensively tested on MPOSE2021 and compared to several state-of-the-art architectures, proving the effectiveness of the AcT model and laying the foundations for future work on HAR.
Comments: Published by Pattern Recognition, Elsevier
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Journal reference: Pattern Recognition, Volume 124, April 2022, 108487
DOI: 10.1016/j.patcog.2021.108487
Cite as: arXiv:2107.00606 [cs.CV]
  (or arXiv:2107.00606v6 [cs.CV] for this version)

Submission history

From: Francesco Salvetti [view email]
[v1] Thu, 1 Jul 2021 16:53:16 GMT (3501kb,D)
[v2] Fri, 2 Jul 2021 09:33:48 GMT (3516kb,D)
[v3] Tue, 6 Jul 2021 09:11:17 GMT (3628kb,D)
[v4] Thu, 23 Dec 2021 10:55:12 GMT (3823kb,D)
[v5] Mon, 27 Dec 2021 15:25:18 GMT (3823kb,D)
[v6] Mon, 10 Jan 2022 08:42:16 GMT (3822kb,D)

Link back to: arXiv, form interface, contact.