We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

eess.IV

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Electrical Engineering and Systems Science > Image and Video Processing

Title: Combining Different V1 Brain Model Variants to Improve Robustness to Image Corruptions in CNNs

Abstract: While some convolutional neural networks (CNNs) have surpassed human visual abilities in object classification, they often struggle to recognize objects in images corrupted with different types of common noise patterns, highlighting a major limitation of this family of models. Recently, it has been shown that simulating a primary visual cortex (V1) at the front of CNNs leads to small improvements in robustness to these image perturbations. In this study, we start with the observation that different variants of the V1 model show gains for specific corruption types. We then build a new model using an ensembling technique, which combines multiple individual models with different V1 front-end variants. The model ensemble leverages the strengths of each individual model, leading to significant improvements in robustness across all corruption categories and outperforming the base model by 38% on average. Finally, we show that using distillation, it is possible to partially compress the knowledge in the ensemble model into a single model with a V1 front-end. While the ensembling and distillation techniques used here are hardly biologically-plausible, the results presented here demonstrate that by combining the specific strengths of different neuronal circuits in V1 it is possible to improve the robustness of CNNs for a wide range of perturbations.
Comments: 15 pages with supplementary material, 3 main figures, 2 supplementary figures, 4 supplementary tables
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Neurons and Cognition (q-bio.NC)
Journal reference: Workshop on Shared Visual Representations in Human and Machine Intelligence 2021
Cite as: arXiv:2110.10645 [eess.IV]
  (or arXiv:2110.10645v2 [eess.IV] for this version)

Submission history

From: Tiago Marques [view email]
[v1] Wed, 20 Oct 2021 16:35:09 GMT (852kb,D)
[v2] Tue, 7 Dec 2021 23:51:00 GMT (857kb,D)

Link back to: arXiv, form interface, contact.