Xichen Pan highlights a structural mismatch in modern text-to-image pipelines: prompts are encoded once by large pretrained LLMs while denoising of intermediate noisy latents is left to a newly trained backbone. He proposes exploring pretrained multimodal priors for denoising and announces RepFusion, with full details on arXiv:2606.14700 and the project page.
Author @xichen_pan (posted 2026-06-16) identifies a mismatch in modern text-to-image systems: large pretrained LLMs typically encode the text prompt once, while a separately trained generative backbone handles the evolving noisy latent states.
Modern text-to-image models are increasingly powered by large pretrained LLMs.
But there is a curious mismatch: the LLM typically encodes the prompt only once, while the evolving noisy latent states are handled entirely by a newly trained generative backbone.
Can pretrained multimodal prior participate in the denoising process?
Introducing RepFusion. (1/12)
📄 arxiv.org/abs/2606.14700
🌐 xichenpan.com/repfusion/