Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Review Networks for Caption Generation
(Submitted on 25 May 2016 (v1), last revised 27 Oct 2016 (this version, v4))
Abstract: We propose a novel extension of the encoder-decoder framework, called a review network. The review network is generic and can enhance any existing encoder- decoder model: in this paper, we consider RNN decoders with both CNN and RNN encoders. The review network performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a thought vector after each review step; the thought vectors are used as the input of the attention mechanism in the decoder. We show that conventional encoder-decoders are a special case of our framework. Empirically, we show that our framework improves over state-of- the-art encoder-decoder systems on the tasks of image captioning and source code captioning.
Submission history
From: Zhilin Yang [view email][v1] Wed, 25 May 2016 14:49:58 GMT (1874kb,D)
[v2] Thu, 26 May 2016 00:47:21 GMT (1874kb,D)
[v3] Tue, 7 Jun 2016 01:39:35 GMT (1875kb,D)
[v4] Thu, 27 Oct 2016 17:50:27 GMT (1877kb,D)
Link back to: arXiv, form interface, contact.