ArXiv

STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs

2026-07-19 · 08:53 UTC ·Ye Wang, Hongjun Wang, Hao Fang... ·1 min read
Authors
Ye Wang, Hongjun Wang, Hao Fang...
Categories
cs.CV
arXiv
https://arxiv.org/abs/2607.17140v1
PDF
https://arxiv.org/pdf/2607.17140v1

Brief

STBridge proposes shared-target alignment to bridge the understanding–generation gap in unified multimodal models by treating a target caption as the common semantic state that the edited image must realize. The method uses supervised fine-tuning followed by sequential reinforcement learning to align caption and image outputs, producing consistent improvements on visual understanding, image generation, and editing benchmarks and reducing fine-grained mismatches.

Why it matters

STBridge (Ye Wang et al., arXiv 2026-07-19) introduces a shared-target alignment framework that connects a model's target caption and edited image via a common target state, replacing separate task-specific paths with a single information flow from target expression to target realization.

Key details

  • Training follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, then sequential reinforcement learning refines target-centered coordination between understanding and generation.
  • Evaluation finds that STBridge consistently improves the initialization UMM on visual understanding, image generation, and image-editing benchmarks and narrows the description–generation alignment gap that prior UMMs exhibited for fine-grained entities, attributes, spatial relations, and local details.
Source evidence

Abstract

Unified multimodal models (UMMs) aim to integrate visual understanding and generation within a single architecture, but architectural unification alone does not ensure semantic consistency. A model may describe the intended target correctly while generating an inconsistent edit. This exposes an understanding-generation alignment gap: linguistic and visual outputs live in different spaces, yet should be governed by the same target semantics. We study this gap in image editing, where an instruction defines a target state that can be both described and visually realized. Given a source image and an edit instruction, we compare a UMM's target caption with its edited image to test whether the two outputs converge on the same result. Our analysis shows that existing UMMs remain weakly aligned, especially for fine-grained entities, attributes, spatial relations, and local details, indicating that semantic unification is not achieved by architecture alone. To bridge this gap, we propose STBridge, a shared-target alignment framework that connects understanding and generation through a common target state. Here the target caption expresses the desired visual result, while the edited image realizes it visually, replacing separate task-specific paths with a shared information flow from target expression to target realization. STBridge follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, and sequential reinforcement learning further refines target-centered coordination. Across visual understanding, image generation, and image editing benchmarks, STBridge consistently improves over the initialization model. Alignment analysis confirms that STBridge narrows the gap between what the model describes and what it generates, demonstrating shared-target alignment as an effective post-training strategy for bridging understanding and generation in UMMs.