References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: High-order Tensor Pooling with Attention for Action Recognition
(Submitted on 11 Oct 2021 (v1), last revised 16 Dec 2023 (this version, v4))
Abstract: We aim at capturing high-order statistics of feature vectors formed by a neural network, and propose end-to-end second- and higher-order pooling to form a tensor descriptor. Tensor descriptors require a robust similarity measure due to low numbers of aggregated vectors and the burstiness phenomenon, when a given feature appears more/less frequently than statistically expected. The Heat Diffusion Process (HDP) on a graph Laplacian is closely related to the Eigenvalue Power Normalization (EPN) of the covariance/autocorrelation matrix, whose inverse forms a loopy graph Laplacian. We show that the HDP and the EPN play the same role, i.e., to boost or dampen the magnitude of the eigenspectrum thus preventing the burstiness. We equip higher-order tensors with EPN which acts as a spectral detector of higher-order occurrences to prevent burstiness. We also prove that for a tensor of order r built from d dimensional feature descriptors, such a detector gives the likelihood if at least one higher-order occurrence is 'projected' into one of binom(d,r) subspaces represented by the tensor; thus forming a tensor power normalization metric endowed with binom(d,r) such 'detectors'. For experimental contributions, we apply several second- and higher-order pooling variants to action recognition, provide previously not presented comparisons of such pooling variants, and show state-of-the-art results on HMDB-51, YUP++ and MPII Cooking Activities.
Submission history
From: Lei Wang [view email][v1] Mon, 11 Oct 2021 12:32:56 GMT (406kb,D)
[v2] Thu, 20 Jul 2023 14:29:07 GMT (1381kb,D)
[v3] Mon, 16 Oct 2023 05:05:06 GMT (1321kb,D)
[v4] Sat, 16 Dec 2023 04:39:08 GMT (1318kb,D)
Link back to: arXiv, form interface, contact.