Twitter/X

Author @ypatil125 asserts that running RL asynchronously is the key to faster…

2026-07-03 · 19:10 UTC ·@ypatil125 ·0 min read

Brief

@ypatil125 promotes asynchronous RL for faster, cheaper training of open‑weight models, citing Applied Compute's research. Applied Compute derived a closed‑form staleness predictor whose simulated predictions match production measurements to within a fraction of a step, and they report those simulations produced insights that preserve high GPU utilization during RL runs without degrading ML performance.

Why it matters

Author @ypatil125 asserts that running RL asynchronously is the key to faster, cheaper training runs for open-weight models, based on extensive internal research.

Key details

  • Applied Compute says they derived a closed-form formula that predicts staleness in an async RL stack in advance; their predictions match measured staleness from production RL runs within a fraction of a step.
  • Their simulations and analytical work produced operational insights that let them maintain high GPU utilization during RL runs without sacrificing ML performance.
Source evidence

Running RL asynchronously is the key to faster and cheaper training runs. We've been doing A LOT of research here to make the most performant RL training stack for open weight models.

Check it out!

Applied Compute (@appliedcompute)

Controlling staleness in an async RL stack has not been well understood, so we derived a closed-form formula that predicts staleness in advance. Our predictions match measured staleness from production RL runs within a fraction of a step. Building these simulations has led to key insights on how we maintain high GPU utilization during our RL runs without sacrificing ML performance.

— https://nitter.net/appliedcompute/status/2073119551336919102#m