We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:


Current browse context:


Change to browse by:

References & Citations

DBLP - CS Bibliography


(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Robotics

Title: Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging

Abstract: Material recognition can help inform robots about how to properly interact with and manipulate real-world objects. In this paper, we present a multimodal sensing technique, leveraging near-infrared spectroscopy and close-range high resolution texture imaging, that enables robots to estimate the materials of household objects. We release a dataset of high resolution texture images and spectral measurements collected from a mobile manipulator that interacted with 144 household objects. We then present a neural network architecture that learns a compact multimodal representation of spectral measurements and texture images. When generalizing material classification to new objects, we show that this multimodal representation enables a robot to recognize materials with greater performance as compared to prior state-of-the-art approaches. Finally, we present how a robot can combine this high resolution local sensing with images from the robot's head-mounted camera to achieve accurate material classification over a scene of objects on a table.
Comments: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2020), 8 pages, 10 figures, 5 tables
Subjects: Robotics (cs.RO); Machine Learning (cs.LG)
Cite as: arXiv:2004.01160 [cs.RO]
  (or arXiv:2004.01160v2 [cs.RO] for this version)

Submission history

From: Zackory Erickson [view email]
[v1] Thu, 2 Apr 2020 17:33:54 GMT (3234kb,D)
[v2] Thu, 30 Jul 2020 19:45:38 GMT (4885kb,D)

Link back to: arXiv, form interface, contact.