We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CL

Change to browse by:

cs

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax

Authors: Jean-Philippe Bernardy (University of Gothenburg), Shalom Lappin (University of Gothenburg, Queen Mary University of London, and King's College London)
Abstract: We show that both an LSTM and a unitary-evolution recurrent neural network (URN) can achieve encouraging accuracy on two types of syntactic patterns: context-free long distance agreement, and mildly context-sensitive cross serial dependencies. This work extends recent experiments on deeply nested context-free long distance dependencies, with similar results. URNs differ from LSTMs in that they avoid non-linear activation functions, and they apply matrix multiplication to word embeddings encoded as unitary matrices. This permits them to retain all information in the processing of an input string over arbitrary distances. It also causes them to satisfy strict compositionality. URNs constitute a significant advance in the search for explainable models in deep learning applied to NLP.
Comments: In Proceedings E2ECOMPVEC, arXiv:2208.05313
Subjects: Computation and Language (cs.CL)
Journal reference: EPTCS 366, 2022, pp. 9-22
DOI: 10.4204/EPTCS.366.4
Cite as: arXiv:2208.05719 [cs.CL]
  (or arXiv:2208.05719v1 [cs.CL] for this version)

Submission history

From: EPTCS [view email]
[v1] Thu, 11 Aug 2022 09:30:49 GMT (62kb)

Link back to: arXiv, form interface, contact.