References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification
(Submitted on 8 Dec 2021 (v1), last revised 16 Oct 2022 (this version, v3))
Abstract: We propose MASTAF, a Model-Agnostic Spatio-Temporal Attention Fusion network for few-shot video classification. MASTAF takes input from a general video spatial and temporal representation,e.g., using 2D CNN, 3D CNN, and Video Transformer. Then, to make the most of such representations, we use self- and cross-attention models to highlight the critical spatio-temporal region to increase the inter-class variations and decrease the intra-class variations. Last, MASTAF applies a lightweight fusion network and a nearest neighbor classifier to classify each query video. We demonstrate that MASTAF improves the state-of-the-art performance on three few-shot video classification benchmarks(UCF101, HMDB51, and Something-Something-V2), e.g., by up to 91.6%, 69.5%, and 60.7% for five-way one-shot video classification, respectively.
Submission history
From: Xin Liu [view email][v1] Wed, 8 Dec 2021 20:41:40 GMT (1024kb,D)
[v2] Mon, 7 Mar 2022 20:37:37 GMT (1458kb,D)
[v3] Sun, 16 Oct 2022 20:01:25 GMT (761kb,D)
Link back to: arXiv, form interface, contact.