We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: VN-Transformer: Rotation-Equivariant Attention for Vector Neurons

Abstract: Rotation equivariance is a desirable property in many practical applications such as motion forecasting and 3D perception, where it can offer benefits like sample efficiency, better generalization, and robustness to input perturbations. Vector Neurons (VN) is a recently developed framework offering a simple yet effective approach for deriving rotation-equivariant analogs of standard machine learning operations by extending one-dimensional scalar neurons to three-dimensional "vector neurons." We introduce a novel "VN-Transformer" architecture to address several shortcomings of the current VN models. Our contributions are: $(i)$ we derive a rotation-equivariant attention mechanism which eliminates the need for the heavy feature preprocessing required by the original Vector Neurons models; $(ii)$ we extend the VN framework to support non-spatial attributes, expanding the applicability of these models to real-world datasets; $(iii)$ we derive a rotation-equivariant mechanism for multi-scale reduction of point-cloud resolution, greatly speeding up inference and training; $(iv)$ we show that small tradeoffs in equivariance ($\epsilon$-approximate equivariance) can be used to obtain large improvements in numerical stability and training robustness on accelerated hardware, and we bound the propagation of equivariance violations in our models. Finally, we apply our VN-Transformer to 3D shape classification and motion forecasting with compelling results.
Comments: Published in Transactions on Machine Learning Research (TMLR), 2023; Previous version appeared in Workshop on Machine Learning for Autonomous Driving, Conference on Neural Information Processing Systems (NeurIPS), 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Robotics (cs.RO)
Cite as: arXiv:2206.04176 [cs.CV]
  (or arXiv:2206.04176v3 [cs.CV] for this version)

Submission history

From: Serge Assaad [view email]
[v1] Wed, 8 Jun 2022 21:48:47 GMT (2718kb,D)
[v2] Fri, 21 Oct 2022 14:23:31 GMT (2723kb,D)
[v3] Tue, 24 Jan 2023 20:27:42 GMT (2731kb,D)

Link back to: arXiv, form interface, contact.