We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Neural and Evolutionary Computing

Title: Uncertainty aware audiovisual activity recognition using deep Bayesian variational inference

Abstract: Deep neural networks (DNNs) provide state-of-the-art results for a multitude of applications, but the approaches using DNNs for multimodal audiovisual applications do not consider predictive uncertainty associated with individual modalities. Bayesian deep learning methods provide principled confidence and quantify predictive uncertainty. Our contribution in this work is to propose an uncertainty aware multimodal Bayesian fusion framework for activity recognition. We demonstrate a novel approach that combines deterministic and variational layers to scale Bayesian DNNs to deeper architectures. Our experiments using in- and out-of-distribution samples selected from a subset of Moments-in-Time (MiT) dataset show a more reliable confidence measure as compared to the non-Bayesian baseline and the Monte Carlo dropout (MC dropout) approximate Bayesian inference. We also demonstrate the uncertainty estimates obtained from the proposed framework can identify out-of-distribution data on the UCF101 and MiT datasets. In the multimodal setting, the proposed framework improved precision-recall AUC by 10.2% on the subset of MiT dataset as compared to non-Bayesian baseline.
Comments: Accepted at ICCV 2019 for Oral presentation
Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG); Machine Learning (stat.ML)
Cite as: arXiv:1811.10811 [cs.NE]
  (or arXiv:1811.10811v3 [cs.NE] for this version)

Submission history

From: Mahesh Subedar [view email]
[v1] Tue, 27 Nov 2018 04:51:54 GMT (1305kb,D)
[v2] Mon, 10 Jun 2019 06:01:04 GMT (1747kb,D)
[v3] Fri, 20 Sep 2019 05:35:30 GMT (1768kb,D)

Link back to: arXiv, form interface, contact.