Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Teaching Cameras to Feel: Estimating Tactile Physical Properties of Surfaces From Images
(Submitted on 29 Apr 2020 (v1), last revised 20 Sep 2021 (this version, v3))
Abstract: The connection between visual input and tactile sensing is critical for object manipulation tasks such as grasping and pushing. In this work, we introduce the challenging task of estimating a set of tactile physical properties from visual information. We aim to build a model that learns the complex mapping between visual information and tactile physical properties. We construct a first of its kind image-tactile dataset with over 400 multiview image sequences and the corresponding tactile properties. A total of fifteen tactile physical properties across categories including friction, compliance, adhesion, texture, and thermal conductance are measured and then estimated by our models. We develop a cross-modal framework comprised of an adversarial objective and a novel visuo-tactile joint classification loss. Additionally, we develop a neural architecture search framework capable of selecting optimal combinations of viewing angles for estimating a given physical property.
Submission history
From: Matthew Purri [view email][v1] Wed, 29 Apr 2020 21:27:26 GMT (3962kb,D)
[v2] Thu, 7 May 2020 16:43:34 GMT (5471kb,D)
[v3] Mon, 20 Sep 2021 19:03:56 GMT (21990kb,D)
Link back to: arXiv, form interface, contact.