Unsupervised Scale-Conditioned Hyperspectral and Multispectral Image Fusion via a Frequency–Spatial Dual-Domain Network.
Saved in:
| Title: | Unsupervised Scale-Conditioned Hyperspectral and Multispectral Image Fusion via a Frequency–Spatial Dual-Domain Network. |
|---|---|
| Authors: | Tang, Peng1 (AUTHOR), Zheng, Ke1,2 (AUTHOR) zhengke@lcu.edu.cn, Li, Jiaxin2,3 (AUTHOR), Yu, Haoyang3,4 (AUTHOR), Sun, Xu1,4 (AUTHOR) |
| Source: | Remote Sensing. Sep2026, Vol. 18 Issue 18, p3187. 35p. |
| Subjects: | Image fusion, Spatial resolution, Signal reconstruction, Multispectral imaging, Machine learning |
| Abstract: | Highlights: What are the main findings? A novel unsupervised framework, SCDF-Net, was developed for HSI–MSI fusion under both integer and fractional resolution ratios, operating directly on the native input grids without any ground-truth HR-HSI labels. The two-stage design freezes self-supervised PSF/SRF estimates as physical priors and drives a frequency–spatial dual-domain network with a continuous scale embedding. SCDF-Net attains the lowest SAM on all four datasets and improves PSNR/RMSE in most baseline comparisons at ×1.5, ×2.0, ×2.4, and ×4.0, while the multi-scale self-evaluation further covers ×5.0 and ×6.0 to assess robustness under severe spatial degradation. What are the implications of the main findings? Treating the resolution ratio as an explicit continuous condition establishes a general principle for scale-flexible fusion, removing the interpolation error incurred when fractional cases are forced onto a nearby integer ratio. Coupling learned degradation priors with scale-conditioned dual-domain detail modeling under observation-domain supervision provides a robust, label-free foundation for real cross-sensor Earth-observation tasks where HR-HSI references are unavailable. Hyperspectral–multispectral image fusion reconstructs high-spatial-resolution hyperspectral images (HR-HSIs) by combining low-resolution hyperspectral images (LR-HSIs) with high-resolution multispectral images (HR-MSIs). Many methods only support integer resolution ratios; when the LR-HSI/HR-MSI ratio is fractional, inputs are often resampled to a nearby integer ratio, altering observations and introducing interpolation error. We propose SCDF-Net, an unsupervised Scale-Conditioned Dual-domain Fusion Network that treats the spatial ratio as an explicit conditioning variable and fuses directly on native grids without HR-HSI labels. A degradation network first estimates the point spread function and spectral response function in a self-supervised manner; the learned operators are then frozen as physical priors. Conditioned on a continuous scale embedding, SCDF-Net integrates HSI spectral features and MSI spatial features through coupled frequency- and spatial-domain branches, trained with dual observation-domain consistency and spectral/frequency regularizations. On four benchmarks, baseline comparisons at scale factors ×1.5, ×2.0, ×2.4, and ×4.0 show that SCDF-Net obtains the lowest SAM on all datasets and improves PSNR/RMSE in most reported settings, with clear advantages under fractional scale factors. In addition, the multi-scale self-evaluation is extended to ×5.0 and ×6.0 to assess robustness under more severe spatial degradation. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? A novel unsupervised framework, SCDF-Net, was developed for HSI–MSI fusion under both integer and fractional resolution ratios, operating directly on the native input grids without any ground-truth HR-HSI labels. The two-stage design freezes self-supervised PSF/SRF estimates as physical priors and drives a frequency–spatial dual-domain network with a continuous scale embedding. SCDF-Net attains the lowest SAM on all four datasets and improves PSNR/RMSE in most baseline comparisons at ×1.5, ×2.0, ×2.4, and ×4.0, while the multi-scale self-evaluation further covers ×5.0 and ×6.0 to assess robustness under severe spatial degradation. What are the implications of the main findings? Treating the resolution ratio as an explicit continuous condition establishes a general principle for scale-flexible fusion, removing the interpolation error incurred when fractional cases are forced onto a nearby integer ratio. Coupling learned degradation priors with scale-conditioned dual-domain detail modeling under observation-domain supervision provides a robust, label-free foundation for real cross-sensor Earth-observation tasks where HR-HSI references are unavailable. Hyperspectral–multispectral image fusion reconstructs high-spatial-resolution hyperspectral images (HR-HSIs) by combining low-resolution hyperspectral images (LR-HSIs) with high-resolution multispectral images (HR-MSIs). Many methods only support integer resolution ratios; when the LR-HSI/HR-MSI ratio is fractional, inputs are often resampled to a nearby integer ratio, altering observations and introducing interpolation error. We propose SCDF-Net, an unsupervised Scale-Conditioned Dual-domain Fusion Network that treats the spatial ratio as an explicit conditioning variable and fuses directly on native grids without HR-HSI labels. A degradation network first estimates the point spread function and spectral response function in a self-supervised manner; the learned operators are then frozen as physical priors. Conditioned on a continuous scale embedding, SCDF-Net integrates HSI spectral features and MSI spatial features through coupled frequency- and spatial-domain branches, trained with dual observation-domain consistency and spectral/frequency regularizations. On four benchmarks, baseline comparisons at scale factors ×1.5, ×2.0, ×2.4, and ×4.0 show that SCDF-Net obtains the lowest SAM on all datasets and improves PSNR/RMSE in most reported settings, with clear advantages under fractional scale factors. In addition, the multi-scale self-evaluation is extended to ×5.0 and ×6.0 to assess robustness under more severe spatial degradation. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18183187 |
