Academic Journal
Temporally consistent video colorization with deep feature propagation and self-regularization learning
| Title: | Temporally consistent video colorization with deep feature propagation and self-regularization learning |
|---|---|
| Authors: | Yihao Liu, Hengyuan Zhao, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Yu Qiao, Chao Dong |
| Source: | Computational Visual Media, Vol 10, Iss 2, Pp 375-395 (2024) |
| Publisher Information: | SpringerOpen, 2024. |
| Publication Year: | 2024 |
| Collection: | LCC:Electronic computers. Computer science |
| Subject Terms: | video colorization, temporal consistency, feature propagation, self-regularization, Electronic computers. Computer science, QA75.5-76.95 |
| More Details: | Abstract Video colorization is a challenging and highly ill-posed problem. Although recent years have witnessed remarkable progress in single image colorization, there is relatively less research effort on video colorization, and existing methods always suffer from severe flickering artifacts (temporal inconsistency) or unsatisfactory colorization. We address this problem from a new perspective, by jointly considering colorization and temporal consistency in a unified framework. Specifically, we propose a novel temporally consistent video colorization (TCVC) framework. TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Furthermore, TCVC introduces a self-regularization learning (SRL) scheme to minimize the differences in predictions obtained using different time steps. SRL does not require any ground-truth color videos for training and can further improve temporal consistency. Experiments demonstrate that our method can not only provide visually pleasing colorized video, but also with clearly better temporal consistency than state-of-the-art methods. A video demo is provided at https://www.youtube.com/watch?v=c7dczMs-olE , while code is available at https://github.com/lyh-18/TCVC-Temporally-Consistent-Video-Colorization . |
| Document Type: | article |
| File Description: | electronic resource |
| Language: | English |
| ISSN: | 2096-0433 2096-0662 |
| Relation: | https://doaj.org/toc/2096-0433; https://doaj.org/toc/2096-0662 |
| DOI: | 10.1007/s41095-023-0342-8 |
| Access URL: | https://doaj.org/article/e5b90cabae4f4e36a622bea5ccaf7878 |
| Accession Number: | edsdoj.5b90cabae4f4e36a622bea5ccaf7878 |
| Database: | Directory of Open Access Journals |
| ISSN: | 20960433 20960662 |
|---|---|
| DOI: | 10.1007/s41095-023-0342-8 |
| Published in: | Computational Visual Media |
| Language: | English |