FCAT: Frequency-Domain Cross-Attention for All-Weather Multispectral Object Detection in Low-Altitude UAV Security Inspection of Urban and Industrial Areas.
Saved in:
| Title: | FCAT: Frequency-Domain Cross-Attention for All-Weather Multispectral Object Detection in Low-Altitude UAV Security Inspection of Urban and Industrial Areas. |
|---|---|
| Authors: | Li, Kewei1,2 (AUTHOR), Zhong, Ziyi1,2 (AUTHOR), Luo, Ziyue1,2,3 (AUTHOR), Tian, Haishan1,2,4 (AUTHOR) tianhaishan@hunnu.edu.cn, Wang, Kui3,4,5 (AUTHOR), Jiang, Han1,5 (AUTHOR), Xiang, Deyuan1,2 (AUTHOR), Tang, Weiwei1,2,3 (AUTHOR) |
| Source: | Remote Sensing. Mar2026, Vol. 18 Issue 5, p826. 24p. |
| Subjects: | Fast Fourier transforms, Object recognition (Computer vision), Aerial surveillance, Police surveillance, Computer performance |
| Abstract: | Highlights: What are the main findings? We propose a frequency-based cross-attention transformer (FCAT), a novel multispectral object detection framework that fuses infrared (IR) and visible (RGB) features via a cross-attention mechanism operating in the frequency domain, significantly improving detection accuracy under diverse lighting and adverse weather conditions. Spatial-domain features are mapped to the frequency domain using the fast Fourier transform (FFT), and the scaled dot product attention is estimated via element-wise product operations, which effectively reduce computational complexity while maintaining the powerful global context modeling ability. We introduce OPVM-VIRD, a new low-altitude UAV multispectral dataset containing 20,025 annotated instances captured at altitudes of 30–200 m across varied scenarios and weather conditions, offering a realistic and challenging benchmark for all-weather aerial surveillance. What are the implications of the main findings? FCAT achieves cutting-edge results on the M3FD, FLIR, and self-built dataset OPVMVIRD: Compared with the ICAF method, mAP50 increases by 10.8%, and the model parameter has only 85.26 M, which is at least 29% lower than the mainstream model. This model is more adaptable for deployment on resource-limited UAV platforms. The proposed frequency-domain cross-attention mechanism provides a new paradigm for multimodal feature fusion that takes into account accuracy, robustness, and computational efficiency; this meets the actual needs of all-weather and all-time security inspection in urban and industrial areas. The OPVM-VIRD dataset focuses on the multispectral sensing tasks of UAVs at lowaltitude and close-to-actual operating scenarios, complements existing aerial multispectral data resources, and helps promote more practical and deployable cross-spectral vision research. UAVs are widely used for all-weather, round-the-clock security inspections in urban and industrial areas. However, pure visible-light systems fail at night or in adverse weather conditions, while pure infrared methods are limited by thermal noise, low spatial resolutions, and high false alarm rates. Multispectral images render the task of object detection highly reliable and robust by providing complementary target feature information. This study suggests a frequency-based cross-attention transformer (FCAT) for multispectral object detection as a solution to this issue. This approach collects cross-modal complementary characteristics, effectively learns and integrates global contextual information via the cross-attention mechanism, and greatly increases multispectral object detection accuracy. At the same time, spatial-domain features are mapped to the frequency domain via the Fourier transform, and the scaled dot product attention is estimated via element-wise product operations, which break through the limitation of traditional spatial-domain matrix multiplication and effectively reduce the computational cost of the model. Additionally, this study independently builds a multi-scene multi-time climate visible–infrared dataset (OPVM-VIRD), which contains 20,025 target instances, to address the issue of the lack of all-weather cross-spectral data in object detection tasks from the perspective of UAVs. Experimental findings from the OPVM-VIRD, M3FD, and FLIR datasets demonstrate that our proposed approach outperforms prevailing state-of-the-art multispectral object detection algorithms on public benchmarks, while the FCAT model achieves an mAP50 score of 94.7% on our custom-built dataset—10.8% higher than ICAF. At the same time, the number of FCAT parameters is 85.26 M, which is significantly lower than that of mainstream models, such as ICAF. Therefore, the FCAT is a change detection strategy with strong model generalization abilities, and it has important application value in the all-day and all-weather security patrol of cities and industrial parks carried out by UAVs. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? We propose a frequency-based cross-attention transformer (FCAT), a novel multispectral object detection framework that fuses infrared (IR) and visible (RGB) features via a cross-attention mechanism operating in the frequency domain, significantly improving detection accuracy under diverse lighting and adverse weather conditions. Spatial-domain features are mapped to the frequency domain using the fast Fourier transform (FFT), and the scaled dot product attention is estimated via element-wise product operations, which effectively reduce computational complexity while maintaining the powerful global context modeling ability. We introduce OPVM-VIRD, a new low-altitude UAV multispectral dataset containing 20,025 annotated instances captured at altitudes of 30–200 m across varied scenarios and weather conditions, offering a realistic and challenging benchmark for all-weather aerial surveillance. What are the implications of the main findings? FCAT achieves cutting-edge results on the M3FD, FLIR, and self-built dataset OPVMVIRD: Compared with the ICAF method, mAP50 increases by 10.8%, and the model parameter has only 85.26 M, which is at least 29% lower than the mainstream model. This model is more adaptable for deployment on resource-limited UAV platforms. The proposed frequency-domain cross-attention mechanism provides a new paradigm for multimodal feature fusion that takes into account accuracy, robustness, and computational efficiency; this meets the actual needs of all-weather and all-time security inspection in urban and industrial areas. The OPVM-VIRD dataset focuses on the multispectral sensing tasks of UAVs at lowaltitude and close-to-actual operating scenarios, complements existing aerial multispectral data resources, and helps promote more practical and deployable cross-spectral vision research. UAVs are widely used for all-weather, round-the-clock security inspections in urban and industrial areas. However, pure visible-light systems fail at night or in adverse weather conditions, while pure infrared methods are limited by thermal noise, low spatial resolutions, and high false alarm rates. Multispectral images render the task of object detection highly reliable and robust by providing complementary target feature information. This study suggests a frequency-based cross-attention transformer (FCAT) for multispectral object detection as a solution to this issue. This approach collects cross-modal complementary characteristics, effectively learns and integrates global contextual information via the cross-attention mechanism, and greatly increases multispectral object detection accuracy. At the same time, spatial-domain features are mapped to the frequency domain via the Fourier transform, and the scaled dot product attention is estimated via element-wise product operations, which break through the limitation of traditional spatial-domain matrix multiplication and effectively reduce the computational cost of the model. Additionally, this study independently builds a multi-scene multi-time climate visible–infrared dataset (OPVM-VIRD), which contains 20,025 target instances, to address the issue of the lack of all-weather cross-spectral data in object detection tasks from the perspective of UAVs. Experimental findings from the OPVM-VIRD, M3FD, and FLIR datasets demonstrate that our proposed approach outperforms prevailing state-of-the-art multispectral object detection algorithms on public benchmarks, while the FCAT model achieves an mAP50 score of 94.7% on our custom-built dataset—10.8% higher than ICAF. At the same time, the number of FCAT parameters is 85.26 M, which is significantly lower than that of mainstream models, such as ICAF. Therefore, the FCAT is a change detection strategy with strong model generalization abilities, and it has important application value in the all-day and all-weather security patrol of cities and industrial parks carried out by UAVs. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18050826 |
