Mindforge built 562 “source-free” cleanroom environments for open-source…

Brief

Mindforge is an automated pipeline that converts open-source command-line programs into "source-free" training environments by giving agents only a compiled executable and documentation (no source). It produced 562 cleanroom setups and 1,001 GLM-5.2 whole-life-cycle trajectories (avg. 181 turns, 177K tokens), and yields agents that run longer, fail less, and more often turn reasoning into working code.

Why it matters

Mindforge built 562 “source-free” cleanroom environments for open-source command-line programs across six compiled languages (including Go, Rust, C, C++) with reproducibility and source-leakage checks, and generated 1,001 whole-life-cycle trajectories from a teacher agent (GLM-5.2) averaging 181 turns and 177K tokens each (spec exploration 99%, design 87%).

Key details

Behavioral analysis shows agents trained with Mindforge work ~2× longer per task (peak 830-turn, 209M-token run), have lower command failure rates, and nearly double the rate at which reasoning and failure recovery produce actual code edits, closing most of the gap to frontier agents.

Mindforge is an automated pipeline we built that converts open-source command-line programs into "source-free" training environments: the agent is given only a compiled reference executable and its documentation — never the source code. To succeed, it must work through the entire software-engineering life cycle: infer the specification from observed behavior, design an architecture, implement it, hunt down its own bugs, write tests, and iterate to a passing build.

With this pipeline we:

🔧 Automatically constructed 562 cleanroom environments across 6 compiled languages (Go, Rust, C, C++, and more), with reproducibility and source-leakage checks at every step

🔧 Collected 1,001 whole-life-cycle trajectories from a teacher agent (GLM-5.2) — averaging 181 turns and 177K tokens each, covering spec exploration (99%), design (87%), debugging, testing, and refinement.

🔧 Refined them with infrastructure-noise recovery and surgical reasoning rewrites, so every training example is clean supervision

And the behavior analysis shows it genuinely learned to engineer: it works ~2× longer per task (up to an 830-turn, 209M-token run) while its command failure rate drops, and it nearly doubles the rate at which reasoning and failure recovery turn into actual code edits — closing most of the gap to frontier agents.