We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Artificial Intelligence

Title: Optimality and limitations of audio-visual integration for cognitive systems

Abstract: Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimises the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artefacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artefacts. Finally, we suggest avenues of research towards solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.
Comments: 20 pages, 6 figures, 1 table 16/06/2020: Updated version includes expanded discussion and addition of new references. Also updated author affiliation information. This version has been accepted for publication with Frontiers
Subjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Neurons and Cognition (q-bio.NC)
Cite as: arXiv:1912.00581 [cs.AI]
  (or arXiv:1912.00581v3 [cs.AI] for this version)

Submission history

From: William Paul Boyce [view email]
[v1] Mon, 2 Dec 2019 05:08:39 GMT (286kb,D)
[v2] Sat, 11 Jan 2020 06:27:55 GMT (189kb,D)
[v3] Tue, 16 Jun 2020 02:10:31 GMT (194kb,D)

Link back to: arXiv, form interface, contact.