References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Temporally Coherent Person Matting Trained on Fake-Motion Dataset
(Submitted on 10 Sep 2021)
Abstract: We propose a novel neural-network-based method to perform matting of videos depicting people that does not require additional user input such as trimaps. Our architecture achieves temporal stability of the resulting alpha mattes by using motion-estimation-based smoothing of image-segmentation algorithm outputs, combined with convolutional-LSTM modules on U-Net skip connections.
We also propose a fake-motion algorithm that generates training clips for the video-matting network given photos with ground-truth alpha mattes and background videos. We apply random motion to photos and their mattes to simulate movement one would find in real videos and composite the result with the background clips. It lets us train a deep neural network operating on videos in an absence of a large annotated video dataset and provides ground-truth training-clip foreground optical flow for use in loss functions.
Submission history
From: Ivan Molodetskikh [view email][v1] Fri, 10 Sep 2021 12:53:11 GMT (46394kb,AD)
Link back to: arXiv, form interface, contact.