Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Automatic Rule Extraction from Long Short Term Memory Networks
(Submitted on 8 Feb 2017 (v1), last revised 24 Feb 2017 (this version, v2))
Abstract: Although deep learning models have proven effective at solving problems in natural language processing, the mechanism by which they come to their conclusions is often unclear. As a result, these models are generally treated as black boxes, yielding no insight of the underlying learned patterns. In this paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new approach for tracking the importance of a given input to the LSTM for a given output. By identifying consistently important patterns of words, we are able to distill state of the art LSTMs on sentiment analysis and question answering into a set of representative phrases. This representation is then quantitatively validated by using the extracted phrases to construct a simple, rule-based classifier which approximates the output of the LSTM.
Submission history
From: William Murdoch [view email][v1] Wed, 8 Feb 2017 17:46:37 GMT (21kb)
[v2] Fri, 24 Feb 2017 22:20:25 GMT (21kb)
Link back to: arXiv, form interface, contact.