Scaling Properties of Text Conditioning in Visual Generation

2026-07-31 · 17:56 UTC ·Zilong Chen, Chaorui Deng, Kunchang Li... ·1 min read

Brief

The paper measures how text conditioning affects diffusion-model loss by introducing two metrics—GPG (white-box likelihood) and ED (black-box attribute). Converged diffusion loss falls roughly linearly with GPG and follows a power law with ED. Leveraging semantic and geometric prompt annotations plus a prompter trained via supervised, cold-start, and verifier-gated distillation, the system outperforms open-weight baselines and rivals closed-weight models.

Why it matters

Chen et al. (published 2026-07-31) show that converged diffusion loss in text-conditioned visual generation decreases with the amount of structured language in prompts: it scales approximately linearly with a white-box likelihood metric (GPG) and follows a power law with a black-box attribute metric (ED).

Key details

Using these scaling insights, the authors boost 'diffusability' via structured prompts (semantic + geometric annotations) and improve 'promptability' by training a prompter (supervised fine-tuning, cold-start, verifier-gated on-policy distillation); their system outperforms all evaluated open-weight models and matches or surpasses top closed-weight models on most compositional, reasoning, and world-knowledge benchmarks (code: github.com/heheyas/context-scaling, models/demo linked).

Abstract

We study empirical scaling properties for text conditioning in visual generation. Such properties have rarely been measured because diffusion loss does not scale with the number of tokens in natural-language prompts. Surprisingly, we find that the converged diffusion loss scales with the amount of structured language in the prompt. To quantify structured language, we adapt two complementary measures: a white-box likelihood metric (GPG) and a black-box attribute metric (ED). Across controlled training runs, the converged diffusion loss decreases approximately linearly with GPG and follows a power law with ED. Guided by these scaling properties, we improve \emph{diffusability} by constructing structured prompts with semantic and geometric annotations derived from images, and improve \emph{promptability} by training a prompter through supervised fine-tuning, cold-start, and verifier-gated on-policy distillation. The resulting system outperforms all evaluated open-weight models on nearly every compositional, reasoning, and world-knowledge benchmark, while matching or surpassing the strongest closed-weight models on most evaluations.

Comment: Code: https://github.com/heheyas/context-scaling Models: https://huggingface.co/collections/heheyas/context-scaling Demo: https://heheyas-context-scaling.hf.space/ Project page: https://heheyas.github.io/context-scaling