We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computation and Language

Title: Multimodal Analogical Reasoning over Knowledge Graphs

Abstract: Analogical reasoning is fundamental to human cognition and holds an important place in various fields. However, previous studies mainly focus on single-modal analogical reasoning and ignore taking advantage of structure knowledge. Notably, the research in cognitive psychology has demonstrated that information from multimodal sources always brings more powerful cognitive transfer than single modality sources. To this end, we introduce the new task of multimodal analogical reasoning over knowledge graphs, which requires multimodal reasoning ability with the help of background knowledge. Specifically, we construct a Multimodal Analogical Reasoning dataSet (MARS) and a multimodal knowledge graph MarKG. We evaluate with multimodal knowledge graph embedding and pre-trained Transformer baselines, illustrating the potential challenges of the proposed task. We further propose a novel model-agnostic Multimodal analogical reasoning framework with Transformer (MarT) motivated by the structure mapping theory, which can obtain better performance. Code and datasets are available in this https URL
Comments: Accepted by ICLR 2023. The project website is this https URL
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM)
Cite as: arXiv:2210.00312 [cs.CL]
  (or arXiv:2210.00312v4 [cs.CL] for this version)

Submission history

From: Ningyu Zhang [view email]
[v1] Sat, 1 Oct 2022 16:24:15 GMT (10656kb,D)
[v2] Tue, 29 Nov 2022 10:40:00 GMT (10723kb,D)
[v3] Wed, 25 Jan 2023 05:26:39 GMT (10723kb,D)
[v4] Wed, 1 Mar 2023 02:51:12 GMT (10724kb,D)

Link back to: arXiv, form interface, contact.