SpatialAfford: Teaching Compact VLMs Where to Look and Where to Ground for Affordance

2026-08-01

AI Summary

Existing compact vision-language models struggle to localize fine-grained, context-dependent affordance regions. This work proposes SpatialAfford, a two-stage framework that first employs Spatial Attention Alignment (SAA) to guide the model toward genuine affordance areas and then refines coordinate prediction through spatially aware GRPO optimization. By decoupling attention guidance from coordinate regression, SpatialAfford introduces a novel spatial reasoning paradigm—“attend first, then localize.” Integrating a compact autoregressive vision-language model with sequence-level supervision and reinforcement learning, the method significantly outperforms larger-scale baselines across multiple benchmarks, including ShareRobot-Bench, ReasonAff, and PartAfford, with its 4B-parameter variant surpassing models exceeding 7B parameters.

Abstract

Affordance grounding aims to localize the functional region for interaction, such as the handle to grasp or the button to press, rather than the whole object. This makes it more challenging than generic visual grounding because the target region is smaller, more ambiguous, and more dependent on task context, especially for compact vision-language models (VLMs) used in embodied settings. Recent sequence-level supervision and reinforcement learning improve coordinate prediction quality, yet compact autoregressive VLMs still lack reliable affordance-aware visual focus before coordinate generation: the model can produce better coordinate tokens while its cross-modal attention remains diffuse and weakly anchored to the true affordance evidence. To address it, we propose SpatialAfford, a two-stage framework that first aligns attention to the ground-truth affordance region through Spatial Attention Alignment (SAA), then refines coordinate prediction with Spatial-Aware GRPO. By explicitly teaching the model where to look before optimizing where to ground, SpatialAfford turns affordance grounding from a purely output-constrained objective into attention-grounded spatial reasoning. Across ShareRobot-Bench, ReasonAff, and PartAfford, SpatialAfford consistently improves affordance grounding, with a compact 4B model outperforming stronger 7B+ baselines.

Problem

• affordance grounding
• compact VLMs
• visual attention
• spatial reasoning
• embodied AI

Innovation

• SpatialAfford
• affordance grounding
• spatial attention alignment
• compact VLMs
• visual grounding