References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Background Matting: The World is Your Green Screen
(Submitted on 1 Apr 2020 (v1), last revised 10 Apr 2020 (this version, v2))
Abstract: We propose a method for creating a matte -- the per-pixel foreground color and alpha -- of a person by taking photos or videos in an everyday setting with a handheld camera. Most existing matting methods require a green screen background or a manually created trimap to produce a good matte. Automatic, trimap-free methods are appearing, but are not of comparable quality. In our trimap free approach, we ask the user to take an additional photo of the background without the subject at the time of capture. This step requires a small amount of foresight but is far less time-consuming than creating a trimap. We train a deep network with an adversarial loss to predict the matte. We first train a matting network with supervised loss on ground truth data with synthetic composites. To bridge the domain gap to real imagery with no labeling, we train another matting network guided by the first network and by a discriminator that judges the quality of composites. We demonstrate results on a wide variety of photos and videos and show significant improvement over the state of the art.
Submission history
From: Soumyadip Sengupta [view email][v1] Wed, 1 Apr 2020 17:38:55 GMT (22476kb,D)
[v2] Fri, 10 Apr 2020 03:31:38 GMT (22476kb,D)
Link back to: arXiv, form interface, contact.