References & Citations
Computer Science > Computation and Language
Title: Bidirectional Representations for Low Resource Spoken Language Understanding
(Submitted on 24 Nov 2022 (v1), last revised 14 Oct 2023 (this version, v2))
Abstract: Most spoken language understanding systems use a pipeline approach composed of an automatic speech recognition interface and a natural language understanding module. This approach forces hard decisions when converting continuous inputs into discrete language symbols. Instead, we propose a representation model to encode speech in rich bidirectional encodings that can be used for downstream tasks such as intent prediction. The approach uses a masked language modelling objective to learn the representations, and thus benefits from both the left and right contexts. We show that the performance of the resulting encodings before fine-tuning is better than comparable models on multiple datasets, and that fine-tuning the top layers of the representation model improves the current state of the art on the Fluent Speech Command dataset, also in a low-data regime, when a limited amount of labelled data is used for training. Furthermore, we propose class attention as a spoken language understanding module, efficient both in terms of speed and number of parameters. Class attention can be used to visually explain the predictions of our model, which goes a long way in understanding how the model makes predictions. We perform experiments in English and in Dutch.
Submission history
From: Quentin Meeus [view email][v1] Thu, 24 Nov 2022 17:05:16 GMT (949kb,D)
[v2] Sat, 14 Oct 2023 13:18:26 GMT (2891kb,D)
Link back to: arXiv, form interface, contact.