References & Citations
Computer Science > Robotics
Title: Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging
(Submitted on 2 Apr 2020 (v1), last revised 30 Jul 2020 (this version, v2))
Abstract: Material recognition can help inform robots about how to properly interact with and manipulate real-world objects. In this paper, we present a multimodal sensing technique, leveraging near-infrared spectroscopy and close-range high resolution texture imaging, that enables robots to estimate the materials of household objects. We release a dataset of high resolution texture images and spectral measurements collected from a mobile manipulator that interacted with 144 household objects. We then present a neural network architecture that learns a compact multimodal representation of spectral measurements and texture images. When generalizing material classification to new objects, we show that this multimodal representation enables a robot to recognize materials with greater performance as compared to prior state-of-the-art approaches. Finally, we present how a robot can combine this high resolution local sensing with images from the robot's head-mounted camera to achieve accurate material classification over a scene of objects on a table.
Submission history
From: Zackory Erickson [view email][v1] Thu, 2 Apr 2020 17:33:54 GMT (3234kb,D)
[v2] Thu, 30 Jul 2020 19:45:38 GMT (4885kb,D)
Link back to: arXiv, form interface, contact.