CITMR: Contrastive Image–Text–Motion Retrieval for Autonomous Driving Critical Scenarios Extraction.
Saved in:
| Title: | CITMR: Contrastive Image–Text–Motion Retrieval for Autonomous Driving Critical Scenarios Extraction. |
|---|---|
| Authors: | Peng, Kun1 (AUTHOR), Li, Shanke1 (AUTHOR), Hui, Fei2 (AUTHOR) feihui@chd.edu.cn, Liu, Xiyao2 (AUTHOR) liuxiyao@chd.edu.cn, Shriniwas, Arkatkar (AUTHOR) sarkatkar@ced.svnit.ac.in |
| Source: | Journal of Advanced Transportation. 5/13/2026, Vol. 2026, p1-14. 14p. |
| Subjects: | Contrastive learning, Motion analysis, Image retrieval, Acquisition of data, Autonomous vehicles |
| Abstract: | Extracting critical scenarios from the vast and complex driving environments is a crucial step in the iterative upgrade process of autonomous driving systems. Existing image–text‐based retrieval methods provide a good understanding of the environment but overlook the critical information regarding the impact of the environment on the self‐vehicle. To address these issues, we propose a contrastive image–text–motion retrieval (CITMR) cross‐modal learning framework that uses descriptive text as input to retrieve critical scenarios. The framework employs image, text, and motion encoders to extract features from different modalities and uses contrastive loss to enable cross‐modal information comparison and interaction. Finally, the vehicle's motion and its textual description are supplemented in the collected autonomous driving dataset, and a scenario dataset containing image–text–motion pairs is constructed for model validation. Experimental results show that CITMR achieves retrieval performance for text‐critical scenarios of 0.8684 and 0.8557 on two test datasets, outperforming the baseline methods. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Advanced Transportation is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Extracting critical scenarios from the vast and complex driving environments is a crucial step in the iterative upgrade process of autonomous driving systems. Existing image–text‐based retrieval methods provide a good understanding of the environment but overlook the critical information regarding the impact of the environment on the self‐vehicle. To address these issues, we propose a contrastive image–text–motion retrieval (CITMR) cross‐modal learning framework that uses descriptive text as input to retrieve critical scenarios. The framework employs image, text, and motion encoders to extract features from different modalities and uses contrastive loss to enable cross‐modal information comparison and interaction. Finally, the vehicle's motion and its textual description are supplemented in the collected autonomous driving dataset, and a scenario dataset containing image–text–motion pairs is constructed for model validation. Experimental results show that CITMR achieves retrieval performance for text‐critical scenarios of 0.8684 and 0.8557 on two test datasets, outperforming the baseline methods. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 01976729 |
| DOI: | 10.1155/atr/4985228 |
