Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Learning to Predict Indoor Illumination from a Single Image
(Submitted on 1 Apr 2017 (v1), last revised 21 Nov 2017 (this version, v3))
Abstract: We propose an automatic method to infer high dynamic range illumination from a single, limited field-of-view, low dynamic range photograph of an indoor scene. In contrast to previous work that relies on specialized image capture, user input, and/or simple scene models, we train an end-to-end deep neural network that directly regresses a limited field-of-view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. We show that this can be accomplished in a three step process: 1) we train a robust lighting classifier to automatically annotate the location of light sources in a large dataset of LDR environment maps, 2) we use these annotations to train a deep neural network that predicts the location of lights in a scene from a single limited field-of-view photo, and 3) we fine-tune this network using a small dataset of HDR environment maps to predict light intensities. This allows us to automatically recover high-quality HDR illumination estimates that significantly outperform previous state-of-the-art methods. Consequently, using our illumination estimates for applications like 3D object insertion, we can achieve results that are photo-realistic, which is validated via a perceptual user study.
Submission history
From: Marc-André Gardner [view email][v1] Sat, 1 Apr 2017 00:50:12 GMT (84644kb,D)
[v2] Thu, 25 May 2017 19:20:01 GMT (28621kb,D)
[v3] Tue, 21 Nov 2017 08:32:24 GMT (27270kb,D)
Link back to: arXiv, form interface, contact.