Title: Perceptual Material Attributes Arise in Local Material Recognition

Abstract: Material attributes have been shown to provide a discriminative intermediate representation for materials, as they help tame the otherwise challenging intra-class appearance variation that materials exhibit. In the past, however, the space of material attributes has been manually-defined and treated as merely an intermediate feature for other recognition tasks. Recent neuroscience studies on material perception, as well as computer vision research on object and place recognition, show that attributes may in fact arise naturally during the course of higher-level recognition. These results suggest that, analogous to scenes and objects, perceptual material attributes may arise during material recognition. Part of the challenge in investigating this conjecture is that existing attribute discovery methods cannot leverage large amounts of data likely required to cause the attributes to arise. In this paper, we design a novel network architecture and database in order to show that perceptual attributes can in fact arise from large-scale end-to-end material recognition. We focus on local material recognition, from small patches, in order to separate the materials from surrounding object and scene context. We extract the inherent material attributes by adding auxiliary loss functions to a material recognition CNN, enabling perceptual attributes to be produced as a side product. Our results show that the discovered attributes correspond well with semantically-meaningful visual material traits, and enable recognition of previously unseen material categories given only a few examples.