PMDet: Patch-Aware Enhancement and Fusion for Multispectral Object Detection.
Saved in:
| Title: | PMDet: Patch-Aware Enhancement and Fusion for Multispectral Object Detection. |
|---|---|
| Authors: | Li, Jie1 (AUTHOR), Sui, Chenhong1,2 (AUTHOR) sui6662015@ytu.edu.cn, Wang, Jing3 (AUTHOR), Zhou, Jun1,3 (AUTHOR) |
| Source: | Remote Sensing. Apr2026, Vol. 18 Issue 7, p1068. 27p. |
| Subjects: | Multispectral imaging, Multisensor data fusion, Infrared imaging, Signal denoising, Spectral imaging |
| Abstract: | Highlights: What are the main findings? A patch-aware enhancement and fusion network (PMDet) is proposed for multispectralobject detection. A unified feature enhancement and aggregation mechanism is designed to improve cross-modal alignment and enable robust deep semantic fusion. What are the implications of the main findings? The proposed method achieves consistent improvements in detection accuracy and robustness across multiple multispectral benchmarks. The patch-aware modeling strategy provides an effective and efficient paradigm for cross-modal feature fusion. Multispectral object detection addresses the limitations of single-modal approaches by fusing complementary information from visible and infrared images, thereby improving robustness in complex environments. However, the inter-modal representations are inherently misaligned due to sensing discrepancies, and the complementary cues they provide are often imbalanced, making it difficult to exploit modality-specific information effectively. Moreover, directly merging features from different modalities can introduce noise and artifacts that deteriorate the detection performance. To this end, this paper proposes a patch-aware enhancement and fusion network for multispectral object detection (PMDet). This method employs a dual-stream backbone equipped with the patch-aware Feature Enhancer (FE) module for cross-modal features alignment and enhancement. FE not only reinforces the feature representation of key regions but also helps to suppress local noise and enhance the model's perception of fine textures and differences. Building on these enriched features, the patch-based Feature Aggregator (FA) module allows for efficient inter-modal feature interaction and semantic fusion with noise resistance. Specifically, both FE and FA modules leverage the shifted-patch design to preserve computational efficiency while enabling long-range modeling. In this regard, PMDet couples multi-scale cross-modal semantic enhancement with deep semantic fusion to form a stable and discriminative multimodal representation pipeline. Experiments on FLIR, LLVIP, and VEDAI demonstrate that the method outperforms mainstream approaches in detection accuracy and robustness, and ablation studies further verify the effectiveness of each module. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? A patch-aware enhancement and fusion network (PMDet) is proposed for multispectralobject detection. A unified feature enhancement and aggregation mechanism is designed to improve cross-modal alignment and enable robust deep semantic fusion. What are the implications of the main findings? The proposed method achieves consistent improvements in detection accuracy and robustness across multiple multispectral benchmarks. The patch-aware modeling strategy provides an effective and efficient paradigm for cross-modal feature fusion. Multispectral object detection addresses the limitations of single-modal approaches by fusing complementary information from visible and infrared images, thereby improving robustness in complex environments. However, the inter-modal representations are inherently misaligned due to sensing discrepancies, and the complementary cues they provide are often imbalanced, making it difficult to exploit modality-specific information effectively. Moreover, directly merging features from different modalities can introduce noise and artifacts that deteriorate the detection performance. To this end, this paper proposes a patch-aware enhancement and fusion network for multispectral object detection (PMDet). This method employs a dual-stream backbone equipped with the patch-aware Feature Enhancer (FE) module for cross-modal features alignment and enhancement. FE not only reinforces the feature representation of key regions but also helps to suppress local noise and enhance the model's perception of fine textures and differences. Building on these enriched features, the patch-based Feature Aggregator (FA) module allows for efficient inter-modal feature interaction and semantic fusion with noise resistance. Specifically, both FE and FA modules leverage the shifted-patch design to preserve computational efficiency while enabling long-range modeling. In this regard, PMDet couples multi-scale cross-modal semantic enhancement with deep semantic fusion to form a stable and discriminative multimodal representation pipeline. Experiments on FLIR, LLVIP, and VEDAI demonstrate that the method outperforms mainstream approaches in detection accuracy and robustness, and ablation studies further verify the effectiveness of each module. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18071068 |
