View in EDS

TGDF: Task-oriented grasping through dense detection and foundation models.

Saved in:
Bibliographic Details
Title: TGDF: Task-oriented grasping through dense detection and foundation models.
Authors: Wang, Hanwen1 (AUTHOR) whw2022111391@bupt.edu.cn, Wen, Yike1 (AUTHOR) 2023141257@bupt.cn, Zhang, Ying1 (AUTHOR) yingzhang_bupt@bupt.edu.cn
Source: Pattern Recognition Letters. Jun2026, Vol. 204, p15-20. 6p.
Subjects: Generalization, Object manipulation, Machine learning, Robots, Detection algorithms
Abstract: • TGDF, a multi-stage framework, merges dense grasp detection and foundation models to generate executable, taskaligned grasp poses. • TGDF exhibits strong generalization, enabling zero-shot deployment in novel scenes. • Real-world task-oriented grasping experiments across 9 task categories achieved 82.22% success, showing our approach's generalization. [Display omitted] Task-oriented grasping is a crucial prerequisite for robotic manipulation, especially for service robots in household environments. However, current methods struggle to model the relationship among task language instructions, observations, and grasp poses. Moreover, most existing approaches are trained on closed-set datasets, resulting in limited generalization to novel scenarios. To address these limitations, we propose TGDF, a multi-stage framework for task-oriented grasp detection. The foundation model in TGDF is capable of perceiving and understanding both human task requirements and the task environment. By combining dense and effective grasp candidate perceptions, it can filter out the unique grasp pose that is specific to the task. The generalization capability of the foundation model enables zero-shot, task-oriented grasping in the real world. Experiments conducted in real-world settings demonstrate that TGDF achieves fine-grained scene understanding and effectively generalizes to novel environments for task-oriented grasping, even with minimal language prompts. Notably, TGDF achieves a task success rate of up to 82.22% in real-robot experiments. A video demo of TGDF is available at https://wykbupt.github.io. [ABSTRACT FROM AUTHOR]
Copyright of Pattern Recognition Letters is the property of Elsevier B.V. and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Database: Engineering Source
Description
Abstract:• TGDF, a multi-stage framework, merges dense grasp detection and foundation models to generate executable, taskaligned grasp poses. • TGDF exhibits strong generalization, enabling zero-shot deployment in novel scenes. • Real-world task-oriented grasping experiments across 9 task categories achieved 82.22% success, showing our approach's generalization. [Display omitted] Task-oriented grasping is a crucial prerequisite for robotic manipulation, especially for service robots in household environments. However, current methods struggle to model the relationship among task language instructions, observations, and grasp poses. Moreover, most existing approaches are trained on closed-set datasets, resulting in limited generalization to novel scenarios. To address these limitations, we propose TGDF, a multi-stage framework for task-oriented grasp detection. The foundation model in TGDF is capable of perceiving and understanding both human task requirements and the task environment. By combining dense and effective grasp candidate perceptions, it can filter out the unique grasp pose that is specific to the task. The generalization capability of the foundation model enables zero-shot, task-oriented grasping in the real world. Experiments conducted in real-world settings demonstrate that TGDF achieves fine-grained scene understanding and effectively generalizes to novel environments for task-oriented grasping, even with minimal language prompts. Notably, TGDF achieves a task success rate of up to 82.22% in real-robot experiments. A video demo of TGDF is available at https://wykbupt.github.io. [ABSTRACT FROM AUTHOR]
ISSN:01678655
DOI:10.1016/j.patrec.2026.03.011