SAGP: Semantic Affordance-Guided Grasp Planning via Coarse-Zone VLM Reasoning

2026-07-31

AI Summary

This work addresses the limitation of existing geometric grasp planning methods, which often ignore functional semantics and consequently generate physically feasible but functionally inappropriate grasps—such as gripping a cup rim or a knife blade—leading to downstream task failure. To overcome this, the authors propose a training-free, coarse-grained region abstraction mechanism that segments object point clouds via PCA alignment and distance-driven DBSCAN clustering. Grasp candidates are then reranked by integrating zero-shot semantic reasoning from a pretrained vision-language model with geometric constraints, reachability, and task-alignment signals. Evaluated on the YCB dataset using a Franka Panda robot, the approach maintains high grasp success rates while significantly improving functional appropriateness, particularly for asymmetric objects with handles.

Abstract

Geometry-based grasp planners ensure physically valid grasps but ignore functional semantics, often generating grasps that are antipodal and collision-free yet practically inappropriate, for example, gripping a mug by its rim, a knife by the blade, or a bottle near its cap. These inconsistencies cause the downstream task to fail even when traditional grasp metrics are met. Existing vision-language model (VLM) approaches either depend on fine-grained, category-specific part segmentation or attempt to directly infer grasp poses, with the latter prone to spatial hallucinations. As a result, no practical, training-free framework has yet been proposed that robustly links high-level semantic reasoning to geometric grasp planning. We introduce Semantic Affordance-Guided Grasp Planning (SAGP), a training-free pipeline built on a coarse-zone abstraction layer. The method first partitions the object point cloud into spatial regions (top, middle, bottom, lateral sides, and protrusions) by applying PCA-based alignment followed by distance-driven DBSCAN clustering, entirely bypassing learned segmentation. A pre-trained VLM then assesses the grasp quality of each region through a structured zero-shot query, and the resulting zone-wise scores are fused with geometric, reachability, and task-alignment signals to re-rank antipodal grasp candidates. Experiments on YCB objects in PyBullet with a Franka Panda robot show that SAGP preserves the high success rate of geometry-only planning while substantially improving the functional appropriateness of selected grasps, particularly on asymmetric, handle-bearing objects where geometry alone is uninformative. The introduced coarse-zone abstraction offers an effective, training-free bridge between VLM-based reasoning and geometric grasp planning, without the need for fine-grained part segmentation.

Problem

• semantic affordance
• grasp planning
• vision-language model
• functional appropriateness
• coarse-zone abstraction

Innovation

• Semantic Affordance
• Coarse-Zone Abstraction
• Vision-Language Model (VLM)
• Grasp Planning
• Training-Free