We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Learning Using Privileged Information for Zero-Shot Action Recognition

Abstract: Zero-Shot Action Recognition (ZSAR) aims to recognize video actions that have never been seen during training. Most existing methods assume a shared semantic space between seen and unseen actions and intend to directly learn a mapping from a visual space to the semantic space. This approach has been challenged by the semantic gap between the visual space and semantic space. This paper presents a novel method that uses object semantics as privileged information to narrow the semantic gap and, hence, effectively, assist the learning. In particular, a simple hallucination network is proposed to implicitly extract object semantics during testing without explicitly extracting objects and a cross-attention module is developed to augment visual feature with the object semantics. Experiments on the Olympic Sports, HMDB51 and UCF101 datasets have shown that the proposed method outperforms the state-of-the-art methods by a large margin.
Subjects: Computer Vision and Pattern Recognition (cs.CV)
MSC classes: 68T45(Primary) 68T07(Secondary)
ACM classes: I.4.8
Cite as: arXiv:2206.08632 [cs.CV]
  (or arXiv:2206.08632v2 [cs.CV] for this version)

Submission history

From: Zhiyi Gao [view email]
[v1] Fri, 17 Jun 2022 08:46:09 GMT (1849kb,D)
[v2] Wed, 22 Jun 2022 09:53:07 GMT (1849kb,D)

Link back to: arXiv, form interface, contact.