Current browse context:
cs.CL
Change to browse by:
References & Citations
Computer Science > Computation and Language
Title: Recurrent Hierarchical Topic-Guided RNN for Language Generation
(Submitted on 21 Dec 2019 (v1), last revised 27 Jun 2020 (this version, v2))
Abstract: To simultaneously capture syntax and global semantics from a text corpus, we propose a new larger-context recurrent neural network (RNN) based language model, which extracts recurrent hierarchical semantic structure via a dynamic deep topic model to guide natural language generation. Moving beyond a conventional RNN-based language model that ignores long-range word dependencies and sentence order, the proposed model captures not only intra-sentence word dependencies, but also temporal transitions between sentences and inter-sentence topic dependencies. For inference, we develop a hybrid of stochastic-gradient Markov chain Monte Carlo and recurrent autoencoding variational Bayes. Experimental results on a variety of real-world text corpora demonstrate that the proposed model not only outperforms larger-context RNN-based language models, but also learns interpretable recurrent multilayer topics and generates diverse sentences and paragraphs that are syntactically correct and semantically coherent.
Submission history
From: Mingyuan Zhou [view email][v1] Sat, 21 Dec 2019 21:11:35 GMT (5992kb,D)
[v2] Sat, 27 Jun 2020 22:22:58 GMT (5937kb,D)
Link back to: arXiv, form interface, contact.