Hyperspectral and Multispectral Image Fusion Based on Adaptive Wavelet Transform and Dual Spectral–Spatial Branch.
Saved in:
| Title: | Hyperspectral and Multispectral Image Fusion Based on Adaptive Wavelet Transform and Dual Spectral–Spatial Branch. |
|---|---|
| Authors: | Chang, Yanhui1 (AUTHOR), Xiao, Zhiyun1,2 (AUTHOR), Lu, Jiayang1,2 (AUTHOR), Fang, Tao2 (AUTHOR), Bao, Tengfei2 (AUTHOR) btflmqy1314@sjtu.edu.cn |
| Source: | Remote Sensing. Jun2026, Vol. 18 Issue 11, p1726. 33p. |
| Subjects: | Image fusion, Wavelet transforms, Feature extraction, Image reconstruction, Multisensor data fusion, Deep learning, Remote sensing |
| Abstract: | Highlights: What are the main findings? A wavelet-deep learning hybrid fusion framework is proposed, which combines pixel-wise adaptive wavelet, transform with dual-branch spectral–spatial feature extraction, effectively preserving spectral fidelity and spatial details. By introducing an adaptive wavelet transform gating mechanism and cross-scale cross-domain bidirectional attention, dynamic optimization of wavelet basis functions and efficient collaboration between features are achieved, thereby significantly improving reconstruction performance. What are the implications of the main findings? Experimental results show that the proposed method can effectively alleviate the information loss and fusion errors caused by resolution mismatch, providing a robust and well-generalizable paradigm for hyperspectral-multispectral fusion. Experimental results show that the proposed method exhibits strong application performance and provides effective support for downstream tasks such as semantic segmentation and precision agriculture. As the role of remote sensing continues to grow, the fusion technology of low-spatial-resolution hyperspectral images and high-spatial-resolution multispectral images has become increasingly critical. Traditional methods rely on fixed rules and exhibit poor robustness, whereas deep learning methods struggle to establish efficient interactions between local and global information due to the complexity of their underlying networks. Therefore, we propose a deep learning fusion module that combines pixel-wise adaptive wavelet transform with a spectral–spatial dual-branch extraction. Firstly, by utilizing the unique properties of the wavelet transform, it is possible to effectively preserve spectral information and extract spatial edge features, thereby achieving preliminary fusion by leveraging both low-frequency and high-frequency components. To compensate for the lack of nonlinear expression capability in the wavelet transform, a dual-branch parallel extraction of spectral and spatial features is subsequently performed in the deep learning module. The Multi-Scale Group Convolution module (MSGC) is utilized to extract spectral information, while the Spectral Compression and Spatially Guided Gating Module (SCSGM) is employed to extract spatial information, thereby enhancing the data's adaptive capability. A bidirectional attention mechanism is interspersed within the module to capture complementary information across different scales, ultimately reconstructing a high-resolution hyperspectral image. Finally, the proposed fusion strategy demonstrates superior performance in practical image reconstruction, outperforming more than ten state-of-the-art fusion methods. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? A wavelet-deep learning hybrid fusion framework is proposed, which combines pixel-wise adaptive wavelet, transform with dual-branch spectral–spatial feature extraction, effectively preserving spectral fidelity and spatial details. By introducing an adaptive wavelet transform gating mechanism and cross-scale cross-domain bidirectional attention, dynamic optimization of wavelet basis functions and efficient collaboration between features are achieved, thereby significantly improving reconstruction performance. What are the implications of the main findings? Experimental results show that the proposed method can effectively alleviate the information loss and fusion errors caused by resolution mismatch, providing a robust and well-generalizable paradigm for hyperspectral-multispectral fusion. Experimental results show that the proposed method exhibits strong application performance and provides effective support for downstream tasks such as semantic segmentation and precision agriculture. As the role of remote sensing continues to grow, the fusion technology of low-spatial-resolution hyperspectral images and high-spatial-resolution multispectral images has become increasingly critical. Traditional methods rely on fixed rules and exhibit poor robustness, whereas deep learning methods struggle to establish efficient interactions between local and global information due to the complexity of their underlying networks. Therefore, we propose a deep learning fusion module that combines pixel-wise adaptive wavelet transform with a spectral–spatial dual-branch extraction. Firstly, by utilizing the unique properties of the wavelet transform, it is possible to effectively preserve spectral information and extract spatial edge features, thereby achieving preliminary fusion by leveraging both low-frequency and high-frequency components. To compensate for the lack of nonlinear expression capability in the wavelet transform, a dual-branch parallel extraction of spectral and spatial features is subsequently performed in the deep learning module. The Multi-Scale Group Convolution module (MSGC) is utilized to extract spectral information, while the Spectral Compression and Spatially Guided Gating Module (SCSGM) is employed to extract spatial information, thereby enhancing the data's adaptive capability. A bidirectional attention mechanism is interspersed within the module to capture complementary information across different scales, ultimately reconstructing a high-resolution hyperspectral image. Finally, the proposed fusion strategy demonstrates superior performance in practical image reconstruction, outperforming more than ten state-of-the-art fusion methods. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18111726 |
