MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

2026-07-29 · 17:23 UTC ·Yihao Chen, Shi Chang, Khaled Chawa... ·1 min read

Brief

MindForge creates source-free program-synthesis environments (compiled executable + docs) and uses GLM-5.2 to generate whole-life-cycle trajectories to fine-tune Qwen3.6-27B; this raises ProgramBench pass rate from 37.98% to 49.51% and yields consistent gains across seven unseen SE benchmarks, showing competitive results versus much larger models. Summary based on abstract; full text not reviewed.

Why it matters

MindForge (published 2026-07-29) builds source-free training environments by converting open-source command-line programs into setups exposing only a compiled executable and documentation, and uses GLM-5.2 as a teacher to synthesize trajectories; fine-tuning Qwen3.6-27B on these trajectories raised ProgramBench average test pass rate from 37.98% to 49.51% (absolute +11.53 points).

Key details

The fine-tuned Qwen3.6-27B improved consistently on seven unseen SE benchmarks with reported absolute gains: +31.00 on RepoZero-C2Rust, +14.16 on DeepSWE, +10.70/+4.56 on NL2Repo-Bench (with/without tests), +5.04 on SWE-bench Verified, +5.93 on SWE-bench Pro, +5.22 on SWE-bench Multilingual, and +4.94 on FeatBench; training repos were disjoint from ProgramBench.

Abstract

Coding agents have made substantial progress on software engineering tasks that modify existing codebases, including bug fixing and feature implementation. However, constructing a complete program from scratch remains a major challenge: even the frontier models evaluated on ProgramBench fully resolve fewer than 1% of tasks. One obstacle is the lack of scalable training environments for this from-scratch setting, spanning the whole software engineering life cycle, as existing environment-construction frameworks focus only on a single phase in software development. To address this gap, we introduce MindForge, an automated pipeline that converts open-source command-line programs into source-free environments that expose only a compiled reference executable and its documentation. Using MindForge, we construct training environments from repositories disjoint from those in ProgramBench, and curate a high-quality data recipe consisting of program synthesis trajectories using GLM-5.2 as the teacher agent. Fine-tuning Qwen3.6-27B on these trajectories increases its ProgramBench average test pass rate from 37.98% to 49.51%, achieving performance comparable to substantially larger frontier models. Moreover, the fine-tuned model consistently improves over the base model across all seven unseen software engineering benchmarks, spanning long-horizon repository generation and translation, bug fixing, feature implementation, and cross-language issue resolution, with absolute gains of 31.00 points on RepoZero-C2Rust, 14.16 on DeepSWE, 10.70/4.56 on NL2Repo-Bench (with/without tests), 5.04 on SWE-bench Verified, 5.93 on SWE-bench Pro, 5.22 on SWE-bench Multilingual, and 4.94 on FeatBench.