Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception

2026-08-02

AI Summary

Existing reinforcement learning approaches for structured visual perception tasks provide only response-level supervision, which fails to disentangle the individual contributions of bounding boxes in multi-object predictions, resulting in coarse optimization granularity. This work proposes the MCR-GRPO framework, which introduces a Marginal Contribution Reward (MCR) mechanism—the first of its kind—to estimate each predicted box’s marginal contribution to the overall matching set value via leave-one-out evaluation. By integrating permutation-invariant matching, count-aware normalization, and hierarchical localization, the framework allocates normalized box-level advantages to their corresponding generation tokens, enabling fine-grained credit assignment. This approach overcomes the limitations of response-level supervision and significantly outperforms existing GRPO baselines across diverse tasks, including referring expression comprehension, open-vocabulary detection, segmentation, and counting, achieving state-of-the-art performance.

Abstract

Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-object binding, object cardinality preservation, and precisely localized grounding and segmentation outputs. However, existing group-relative reinforcement learning methods provide only response-level supervision, creating a granularity mismatch for structured multi-object prediction: a single advantage is broadcast to all tokens in a response, without distinguishing individual box contributions. To address this mismatch, we propose MCR-GRPO, a marginal contribution assignment framework that derives box-level credit directly from each sampled response. Specifically, Marginal Contribution Reward (MCR) estimates each predicted box's contribution through a leave-one-out comparison, measuring how the matched set value changes when the box is removed from the response. After within-response normalization, records that improve the set value receive positive credit, while redundant or harmful ones are suppressed. To make marginal attribution stable and informative, we further introduce a Continuous Matched Set Value Evaluator that integrates permutation-invariant matching, count-aware normalization, and graded localization. MCR-GRPO maps normalized box-level marginal advantages to the token spans that generated each box, preserving GRPO's response-level comparison while enabling box-aware optimization of structured multi-object grounding. Experiments across REC, DOD, segmentation, and counting benchmarks show state-of-the-art performance over prior GRPO-based baselines.

Problem

• structured visual perception
• multimodal large language models
• credit assignment
• object grounding
• reinforcement learning

Innovation

• Marginal Contribution Reward
• Structured Visual Perception
• Box-level Credit Assignment
• Multimodal Large Language Models
• GRPO