ST-Veto: Spatio-Temporal Token Veto for Diffusion MLLMs via Taylor Prediction and Visual Grounding

2026-07-20

AI Summary

Existing diffusion-based multimodal large language models (dMLLMs) suffer from the absence of effective confidence estimation and self-correction mechanisms during inference, limiting their generation quality. This work proposes ST-Veto, a training-free decoding strategy that, for the first time, integrates second-order Taylor expansion to predict confidence trends with visual attention quality assessment. Leveraging the order-agnostic generation property of diffusion models, ST-Veto dynamically identifies and replaces unstable or weakly grounded tokens at each decoding step. The approach enables joint spatiotemporal token-level optimization, achieving up to a 9% absolute accuracy gain across multiple dMLLMs and multimodal reasoning benchmarks. It significantly outperforms existing decoding and vision-language reasoning methods while introducing no additional training or computational overhead.

Abstract

Vision Language Models (VLMs) achieve strong reasoning with Chain-of-Thought (CoT) prompting but incur high sequential-generation cost, error accumulation, and limited self-correction. Diffusion Multimodal Large Language Models (dMLLMs) unmask tokens in an order-agnostic process, improving efficiency and enabling iterative refinement, yet their reasoning and how to enhance it remain underexplored. We propose a training-free method, Spatio-Temporal Token Veto (ST-Veto), which leverages the ability to observe all token positions at each diffusion step. Rather than relying only on current-step confidence, ST-Veto vetoes temporally unstable tokens via second-order Taylor prediction of confidence dynamics and filters weakly grounded tokens using image-attention mass, swapping them with safer candidates. Across multiple dMLLMs and multimodal reasoning benchmarks, ST-Veto consistently outperforms standard decoding policies and prior VLM reasoning methods, improving accuracy by up to 9% with no additional training or generation cost. Analyses show that ST-Veto steers generation toward higher-confidence, better-grounded paths.

Problem

• Diffusion Multimodal Large Language Models
• reasoning
• token stability
• visual grounding
• confidence dynamics

Innovation

• Diffusion MLLMs
• Token Veto
• Taylor Prediction
• Visual Grounding
• Spatio-Temporal Reasoning