Bibliographic Details
| Title: |
Boosting Image Super-Resolution Via Fusion of Complementary Information Captured by Multi-Modal Sensors |
| Authors: |
Wang, Fan, Yang, Jiangxin, Cao, Yanlong, Cao, Yanpeng, Yang, Michael Ying |
| Publication Year: |
2020 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Computer Vision and Pattern Recognition |
| More Details: |
Image Super-Resolution (SR) provides a promising technique to enhance the image quality of low-resolution optical sensors, facilitating better-performing target detection and autonomous navigation in a wide range of robotics applications. It is noted that the state-of-the-art SR methods are typically trained and tested using single-channel inputs, neglecting the fact that the cost of capturing high-resolution images in different spectral domains varies significantly. In this paper, we attempt to leverage complementary information from a low-cost channel (visible/depth) to boost image quality of an expensive channel (thermal) using fewer parameters. To this end, we first present an effective method to virtually generate pixel-wise aligned visible and thermal images based on real-time 3D reconstruction of multi-modal data captured at various viewpoints. Then, we design a feature-level multispectral fusion residual network model to perform high-accuracy SR of thermal images by adaptively integrating co-occurrence features presented in multispectral images. Experimental results demonstrate that this new approach can effectively alleviate the ill-posed inverse problem of image SR by taking into account complementary information from an additional low-cost channel, significantly outperforming state-of-the-art SR approaches in terms of both accuracy and efficiency.
Comment: 8 pages, 7 figures |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2012.03417 |
| Accession Number: |
edsarx.2012.03417 |
| Database: |
arXiv |
