We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:


References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Does Vision Accelerate Hierarchical Generalization of Neural Language Learners?

Abstract: Neural language models (LMs) are arguably less data-efficient than humans -- why does this gap occur? In this study, we hypothesize that this gap stems from the learners' accessibility to modalities other than text, specifically, vision. We conducted two complementary experiments (using noisy, realistic data and a simplified, artificial one) toward the advantage of vision in the syntactic generalization of LMs. Our results showed that vision accelerated a proper linguistic generalization in the simplified, artificial setting, but LMs struggled with the noisy, realistic setting. These mixed results indicate several possibilities, e.g., vision can potentially boost language acquisition, but learners' additional visual/linguistic prior knowledge should be needed to robustly make use of raw images for efficient language acquisition.
Comments: Work in progress
Subjects: Computation and Language (cs.CL)
Cite as: arXiv:2302.00667 [cs.CL]
  (or arXiv:2302.00667v1 [cs.CL] for this version)

Submission history

From: Tatsuki Kuribayashi [view email]
[v1] Wed, 1 Feb 2023 18:53:42 GMT (1564kb,D)

Link back to: arXiv, form interface, contact.