We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.CV

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Computer Vision and Pattern Recognition

Title: Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning

Authors: Zhuowan Li (1), Xingrui Wang (2), Elias Stengel-Eskin (1), Adam Kortylewski (3 and 4), Wufei Ma (1), Benjamin Van Durme (1), Alan Yuille (1) ((1) Johns Hopkins University, (2) University of Southern California, (3) Max Planck Institute for Informatics, (4) University of Freiburg)
Abstract: Visual Question Answering (VQA) models often perform poorly on out-of-distribution data and struggle on domain generalization. Due to the multi-modal nature of this task, multiple factors of variation are intertwined, making generalization difficult to analyze. This motivates us to introduce a virtual benchmark, Super-CLEVR, where different factors in VQA domain shifts can be isolated in order that their effects can be studied independently. Four factors are considered: visual complexity, question redundancy, concept distribution and concept compositionality. With controllably generated data, Super-CLEVR enables us to test VQA methods in situations where the test data differs from the training data along each of these axes. We study four existing methods, including two neural symbolic methods NSCL and NSVQA, and two non-symbolic methods FiLM and mDETR; and our proposed method, probabilistic NSVQA (P-NSVQA), which extends NSVQA with uncertainty reasoning. P-NSVQA outperforms other methods on three of the four domain shift factors. Our results suggest that disentangling reasoning and perception, combined with probabilistic uncertainty, form a strong VQA model that is more robust to domain shifts. The dataset and code are released at this https URL
Comments: The dataset and code are released at this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
Cite as: arXiv:2212.00259 [cs.CV]
  (or arXiv:2212.00259v1 [cs.CV] for this version)

Submission history

From: Zhuowan Li [view email]
[v1] Thu, 1 Dec 2022 03:53:24 GMT (4595kb,D)

Link back to: arXiv, form interface, contact.