GuidedNet: A General CNN Fusion Framework via High-Resolution Guidance for Hyperspectral Image Super-Resolution

Ran Ran (1) , Liang-Jian Deng (1) , Tai-Xiang Jiang (2) , Jin-Fan Hu (1) , Jocelyn Chanussot (3, 4, 5) , Gemine Vivone (6)

1 UESTC - School of Mathematical Sciences [Chengdu]
2 Southwestern University of Finance and Economics [Chengdu, China]
3 AIRICAS - Aerospace Information Research Institute
4 Thoth - Apprentissage de modèles à partir de données massives
5 GIPSA-SIGMAPHY - GIPSA - Signal Images Physique
6 IMAA - Institute of Methodologies for Environmental Analysis of the National Research Council [Italy]

Hyperspectral image super-resolution (HISR) is about fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI) to generate a high-resolution hyperspectral image (HR-HSI). Recently, convolutional neural network (CNN)-based techniques have been extensively investigated for HISR yielding competitive outcomes. However, existing CNN-based methods often require a huge amount of network parameters leading to a heavy computational burden, thus, limiting the generalization ability. In this article, we fully consider the characteristic of the HISR, proposing a general CNN fusion framework with high-resolution guidance, called GuidedNet. This framework consists of two branches, including 1) the high-resolution guidance branch (HGB) that can decompose the high-resolution guidance image into several scales and 2) the feature reconstruction branch (FRB) that takes the low-resolution image and the multiscaled high-resolution guidance images from the HGB to reconstruct the high-resolution fused image. GuidedNet can effectively predict the high-resolution residual details that are added to the upsampled HSI to simultaneously improve spatial quality and preserve spectral information. The proposed framework is implemented using recursive and progressive strategies, which can promote high performance with a significant network parameter reduction, even ensuring network stability by supervising several intermediate outputs. Additionally, the proposed approach is also suitable for other resolution enhancement tasks, such as remote sensing pansharpening and single-image super-resolution (SISR). Extensive experiments on simulated and real datasets demonstrate that the proposed framework generates state-of-the-art outcomes for several applications (i.e., HISR, pansharpening, and SISR). Finally, an ablation study and more discussions assessing, for example, the network generalization, the low computational cost, and the fewer network parameters, are provided to the readers. The code link is: https://github.com/Evangelion09/GuidedNet .

Domains: Artificial Intelligence [cs.AI]

Ran Ran, Liang-Jian Deng, Tai-Xiang Jiang, Jin-Fan Hu, Jocelyn Chanussot, et al.. GuidedNet: A General CNN Fusion Framework via High-Resolution Guidance for Hyperspectral Image Super-Resolution. IEEE Transactions on Cybernetics, 2023, 53 (7), pp.4148-4161. ⟨10.1109/TCYB.2023.3238200⟩. ⟨hal-04473668⟩