Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Attention, please! A Critical Review of Neural Attention Models in Natural Language Processing
(Submitted on 4 Feb 2019 (this version), latest version 11 Oct 2021 (v4))
Abstract: Attention is an increasingly popular mechanism used in a wide range of neural architectures. Because of the fast-paced advances in this domain, a systematic overview of attention is still missing. In this article, we define a unified model for attention architectures for natural language processing, with a focus on architectures designed to work with vector representation of the textual data. We discuss the dimensions along which proposals differ, the possible uses of attention, and chart the major research activities and open challenges in the area.
Submission history
From: Andrea Galassi [view email][v1] Mon, 4 Feb 2019 17:14:13 GMT (6738kb,D)
[v2] Thu, 28 May 2020 16:09:50 GMT (6511kb,D)
[v3] Fri, 11 Sep 2020 10:52:29 GMT (2731kb)
[v4] Mon, 11 Oct 2021 10:54:55 GMT (2573kb)
Link back to: arXiv, form interface, contact.