References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Temporal Modulation Network for Controllable Space-Time Video Super-Resolution
(Submitted on 21 Apr 2021 (v1), last revised 30 Apr 2021 (this version, v2))
Abstract: Space-time video super-resolution (STVSR) aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Recently, deformable convolution based methods have achieved promising STVSR performance, but they could only infer the intermediate frame pre-defined in the training stage. Besides, these methods undervalued the short-term motion cues among adjacent frames. In this paper, we propose a Temporal Modulation Network (TMNet) to interpolate arbitrary intermediate frame(s) with accurate high-resolution reconstruction. Specifically, we propose a Temporal Modulation Block (TMB) to modulate deformable convolution kernels for controllable feature interpolation. To well exploit the temporal information, we propose a Locally-temporal Feature Comparison (LFC) module, along with the Bi-directional Deformable ConvLSTM, to extract short-term and long-term motion cues in videos. Experiments on three benchmark datasets demonstrate that our TMNet outperforms previous STVSR methods. The code is available at this https URL
Submission history
From: Gang Xu [view email][v1] Wed, 21 Apr 2021 17:10:53 GMT (4023kb,D)
[v2] Fri, 30 Apr 2021 01:11:27 GMT (4023kb,D)
Link back to: arXiv, form interface, contact.