A Relative Orbital Motion-Guided Framework for Generating Multimodal Visual Data of Spacecraft.
Saved in:
| Title: | A Relative Orbital Motion-Guided Framework for Generating Multimodal Visual Data of Spacecraft. |
|---|---|
| Authors: | Li, Wanyun1 (AUTHOR), Huo, Yurong1 (AUTHOR), Zhu, Qinyu1 (AUTHOR), Lu, Yao1 (AUTHOR), Fang, Yuqiang1 (AUTHOR) fangyuqiang@nudt.edu.cn, Zhang, Yasheng1 (AUTHOR) |
| Source: | Remote Sensing. Apr2026, Vol. 18 Issue 8, p1177. 23p. |
| Subjects: | Synthetic data, Relative motion, Space debris, Image processing, Space vehicles |
| Abstract: | Highlights: What are the main findings? A novel, physically grounded framework is proposed to generate large-scale synthetic visual datasets for non-cooperative spacecraft. It incorporates relative orbital motion simulation and end-to-end imaging degradation to produce realistic multimodal data (RGB, mask, depth, normal) with precise 6-DoF pose annotations for four representative spacecraft types. The generated dataset, comprising 8000 high-fidelity samples, effectively bridges the domain gap between synthetic and real on-orbit imagery. It specifically addresses the critical data scarcity issue in spacecraft visual perception, which arises from limited and inaccessible real-world data. What are the implications of the main findings? This work provides a crucial, open-access data foundation and simulation tool for the development, benchmarking, and validation of data-driven algorithms (e.g., for pose estimation, segmentation, tracking) in on-orbit servicing and space debris removal, potentially accelerating related research and technology maturation. The proposed integrated pipeline establishes a new paradigm for generating physically accurate and task-specific synthetic data in remote sensing by integrating motion physics, multimodal ground truth, and sensor degradation. This paradigm can be adapted to other space and Earth observation applications facing similar data scarcity challenges. The advancement of on-orbit servicing and space debris removal missions has established high-precision visual perception for non-cooperative spacecraft as a key research focus. However, the availability of high-quality, diverse spacecraft image datasets is severely limited due to extreme on-orbit imaging conditions, data confidentiality, and morphological diversity of targets, significantly constraining the advancement of data-driven algorithms in this domain. To address this challenge, we propose a relative orbital motion-guided framework for generating multimodal visual data of spacecraft. The proposed method integrates an orbital dynamics model into the synthetic data generation pipeline to simulate typical relative motion patterns between the camera and the target in a realistic orbital environment, thereby generating image sequences characterized by continuous spatiotemporal evolution. Targeting four representative spacecraft—Tiangong, Spacedragon, ICESat, and Cassini—this work simultaneously generates a dataset comprising 8000 samples, each containing four strictly aligned modalities: RGB images, instance segmentation masks, depth maps, and surface normal maps, along with precise 6-degree-of-freedom (6-DoF) pose ground truth. Furthermore, an end-to-end physical image degradation model is developed to accurately simulate the complete imaging chain—from optical diffraction and aberrations to sensor sampling and noise—thereby effectively narrowing the domain gap between synthetic and real data. By addressing three key aspects—physical motion modeling, synchronous multimodal ground truth, and imaging degradation simulation—this work provides a crucial data foundation for training, testing, and validating data-driven on-orbit perception algorithms. [ABSTRACT FROM AUTHOR] |
| Copyright of Remote Sensing is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Highlights: What are the main findings? A novel, physically grounded framework is proposed to generate large-scale synthetic visual datasets for non-cooperative spacecraft. It incorporates relative orbital motion simulation and end-to-end imaging degradation to produce realistic multimodal data (RGB, mask, depth, normal) with precise 6-DoF pose annotations for four representative spacecraft types. The generated dataset, comprising 8000 high-fidelity samples, effectively bridges the domain gap between synthetic and real on-orbit imagery. It specifically addresses the critical data scarcity issue in spacecraft visual perception, which arises from limited and inaccessible real-world data. What are the implications of the main findings? This work provides a crucial, open-access data foundation and simulation tool for the development, benchmarking, and validation of data-driven algorithms (e.g., for pose estimation, segmentation, tracking) in on-orbit servicing and space debris removal, potentially accelerating related research and technology maturation. The proposed integrated pipeline establishes a new paradigm for generating physically accurate and task-specific synthetic data in remote sensing by integrating motion physics, multimodal ground truth, and sensor degradation. This paradigm can be adapted to other space and Earth observation applications facing similar data scarcity challenges. The advancement of on-orbit servicing and space debris removal missions has established high-precision visual perception for non-cooperative spacecraft as a key research focus. However, the availability of high-quality, diverse spacecraft image datasets is severely limited due to extreme on-orbit imaging conditions, data confidentiality, and morphological diversity of targets, significantly constraining the advancement of data-driven algorithms in this domain. To address this challenge, we propose a relative orbital motion-guided framework for generating multimodal visual data of spacecraft. The proposed method integrates an orbital dynamics model into the synthetic data generation pipeline to simulate typical relative motion patterns between the camera and the target in a realistic orbital environment, thereby generating image sequences characterized by continuous spatiotemporal evolution. Targeting four representative spacecraft—Tiangong, Spacedragon, ICESat, and Cassini—this work simultaneously generates a dataset comprising 8000 samples, each containing four strictly aligned modalities: RGB images, instance segmentation masks, depth maps, and surface normal maps, along with precise 6-degree-of-freedom (6-DoF) pose ground truth. Furthermore, an end-to-end physical image degradation model is developed to accurately simulate the complete imaging chain—from optical diffraction and aberrations to sensor sampling and noise—thereby effectively narrowing the domain gap between synthetic and real data. By addressing three key aspects—physical motion modeling, synchronous multimodal ground truth, and imaging degradation simulation—this work provides a crucial data foundation for training, testing, and validating data-driven on-orbit perception algorithms. [ABSTRACT FROM AUTHOR] |
|---|---|
| ISSN: | 20724292 |
| DOI: | 10.3390/rs18081177 |
