We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Towards Joint Intent Detection and Slot Filling via Higher-order Attention

Abstract: Intent detection (ID) and Slot filling (SF) are two major tasks in spoken language understanding (SLU). Recently, attention mechanism has been shown to be effective in jointly optimizing these two tasks in an interactive manner. However, latest attention-based works concentrated only on the first-order attention design, while ignoring the exploration of higher-order attention mechanisms. In this paper, we propose a BiLinear attention block, which leverages bilinear pooling to simultaneously exploit both the contextual and channel-wise bilinear attention distributions to capture the second-order interactions between the input intent or slot features. Higher and even infinity order interactions are built by stacking numerous blocks and assigning Exponential Linear Unit (ELU) to blocks. Before the decoding stage, we introduce the Dynamic Feature Fusion Layer to implicitly fuse intent and slot information in a more effective way. Technically, instead of simply concatenating intent and slot features, we first compute two correlation matrices to weight on two features. Furthermore, we present Higher-order Attention Network for the SLU tasks. Experiments on two benchmark datasets show that our approach yields improvements compared with the state-of-the-art approach. We also provide discussion to demonstrate the effectiveness of the proposed approach.
Comments: The authors withdraw the manuscript and will update it in the original arXiv address (arXiv:2108.11916)
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2109.08890 [cs.CL]
  (or arXiv:2109.08890v2 [cs.CL] for this version)

Submission history

From: Zhiqi Huang [view email]
[v1] Sat, 18 Sep 2021 09:50:23 GMT (6085kb,D)
[v2] Wed, 22 Sep 2021 15:26:56 GMT (0kb,I)

Link back to: arXiv, form interface, contact.