We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Encode, Review, and Decode: Reviewer Module for Caption Generation

Abstract: We propose a novel module, the reviewer module, to improve the encoder-decoder learning framework. The reviewer module is generic, and can be plugged into an existing encoder-decoder model. The reviewer module performs a number of review steps with attention mechanism on the encoder hidden states, and outputs a fact vector after each review step; the fact vectors are used as the input of the attention mechanism in the decoder. We show that the conventional encoder-decoders are a special case of our framework. Empirically, we show that our framework can improve over state-of-the-art encoder-decoder systems on the tasks of image captioning and source code captioning.
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:1605.07912 [cs.LG]
  (or arXiv:1605.07912v3 [cs.LG] for this version)

Submission history

From: Zhilin Yang [view email]
[v1] Wed, 25 May 2016 14:49:58 GMT (1874kb,D)
[v2] Thu, 26 May 2016 00:47:21 GMT (1874kb,D)
[v3] Tue, 7 Jun 2016 01:39:35 GMT (1875kb,D)
[v4] Thu, 27 Oct 2016 17:50:27 GMT (1877kb,D)

Link back to: arXiv, form interface, contact.