STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs

2026-07-19

AI Summary

This work addresses the semantic alignment gap between understanding and generation in existing unified multimodal models, which often leads to inconsistencies between linguistic descriptions and visual outputs. To bridge this gap, the authors propose STBridge, a shared-target alignment framework that replaces task-specific pathways with a unified shared-target channel. STBridge establishes a coherent information flow from target representation to realization through a “align-then-optimize” strategy. The framework leverages supervised fine-tuning to construct a shared target representation and employs sequential reinforcement learning to refine a target-centric coordination mechanism. Experimental results demonstrate that STBridge significantly outperforms baseline models across comprehension, generation, and editing tasks, effectively closing the semantic-visual discrepancy.

Abstract

Unified multimodal models (UMMs) aim to integrate visual understanding and generation within a single architecture, but architectural unification alone does not ensure semantic consistency. A model may describe the intended target correctly while generating an inconsistent edit. This exposes an understanding-generation alignment gap: linguistic and visual outputs live in different spaces, yet should be governed by the same target semantics. We study this gap in image editing, where an instruction defines a target state that can be both described and visually realized. Given a source image and an edit instruction, we compare a UMM's target caption with its edited image to test whether the two outputs converge on the same result. Our analysis shows that existing UMMs remain weakly aligned, especially for fine-grained entities, attributes, spatial relations, and local details, indicating that semantic unification is not achieved by architecture alone. To bridge this gap, we propose STBridge, a shared-target alignment framework that connects understanding and generation through a common target state. Here the target caption expresses the desired visual result, while the edited image realizes it visually, replacing separate task-specific paths with a shared information flow from target expression to target realization. STBridge follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, and sequential reinforcement learning further refines target-centered coordination. Across visual understanding, image generation, and image editing benchmarks, STBridge consistently improves over the initialization model. Alignment analysis confirms that STBridge narrows the gap between what the model describes and what it generates, demonstrating shared-target alignment as an effective post-training strategy for bridging understanding and generation in UMMs.

Problem

• understanding-generation alignment
• unified multimodal models
• semantic consistency
• image editing
• target semantics

Innovation

• shared-target alignment
• unified multimodal models
• understanding-generation gap
• image editing
• reinforcement learning