ArXiv

GDPevo: Evaluating Agent Self-Evolution on Real Business Tasks

Authors
Leijun Zhou, Zhihao Liu, Xiang Qu...
Categories
cs.AI
arXiv
https://arxiv.org/abs/2608.03764v1
PDF
https://arxiv.org/pdf/2608.03764v1

Brief

GDPevo presents an evolution-focused benchmark and automated pipeline for agent self-evolution using GDP-related enterprise workflows; it applies rule hybridization to split workflows into atomic business rules and recombine them into held-out tests. V1 offers 120 tasks (12 groups); V2 can reach 240 tasks in two days. Experiments on four agents show up to 16.44 pp gains but a large gap to a 91.6% oracle.

Why it matters

GDPevo is an evolution-native benchmark for agent self-evolution grounded in GDP-related enterprise workflows; V1 contains 120 tasks across 12 groups (5 training + 5 held-out test tasks per group) and uses rule hybridization to decompose/recombine atomic business rules so test gains are attributable to prior experience.

Key details

  • The fully automated pipeline can expand the suite to 240 tasks in 24 groups (V2) within two days to mitigate data contamination, and the authors publicly release the pipeline, benchmark, and evaluations at https://github.com/Prism-Shadow/GDPevo.
  • Evaluation of four agents (each a harness + model) under four supervision types shows self-evolution yields held-out accuracy gains up to 16.44 percentage points, but the best evolved agents remain well below the oracle ceiling of 91.6%.
Source evidence

Abstract

Agent self-evolution updates an agent's persistent state from prior experience and reuses it to solve related tasks more effectively. Evaluating self-evolution is difficult: existing benchmarks provide limited coverage of economically valuable task domains, do not always design training and test tasks such that test-time gains can be attributed to training experience, and remain vulnerable to data contamination. We present GDPevo, an evolution-native benchmark grounded in GDP-related enterprise workflows, together with the fully automated data pipeline that generates it. Its core mechanism, rule hybridization, decomposes each enterprise workflow into atomic business rules, distributes subsets of these rules across training tasks, and recombines them in held-out test tasks so that test-time gains are attributable. GDPevo spans CRM, ERP, finance, healthcare, legal, and data-centric workflows. Its V1 release contains 120 tasks in 12 groups, with five training and five held-out test tasks per group. Full automation enables the pipeline to expand the suite to 240 tasks in 24 groups (V2) within two days, providing a practical response to contamination. Using GDPevo, we evaluate four agents, each comprising a harness and a model, under four supervision types. Self-evolution consistently improves held-out accuracy by up to 16.44 percentage points. But the best evolved agents remain far below the fully informed oracle ceiling of 91.6%, indicating that the self-evolution ability of current agents remains far from fully realized. We publicly release the pipeline, benchmark, and full evaluation results at https://github.com/Prism-Shadow/GDPevo.