Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Attention-Free Keyword Spotting
(Submitted on 14 Oct 2021 (v1), last revised 10 Apr 2022 (this version, v3))
Abstract: Till now, attention-based models have been used with great success in the keyword spotting problem domain. However, in light of recent advances in deep learning, the question arises whether self-attention is truly irreplaceable for recognizing speech keywords. We thus explore the usage of gated MLPs --previously shown to be alternatives to transformers in vision tasks-- for the keyword spotting task. We provide a family of highly efficient MLP-based models for keyword spotting, with less than 0.5 million parameters. We show that our approach achieves competitive performance on Google Speech Commands V2-12 and V2-35 benchmarks with much fewer parameters than self-attention-based methods.
Submission history
From: Ahmad Omar Ahsan Ahsan [view email][v1] Thu, 14 Oct 2021 22:23:53 GMT (199kb,D)
[v2] Mon, 18 Oct 2021 11:21:52 GMT (199kb,D)
[v3] Sun, 10 Apr 2022 13:50:49 GMT (296kb,D)
Link back to: arXiv, form interface, contact.