ArXiv

EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions

2026-06-22 · 17:39 UTC ·Jincheng Zhong, Weizhi Wang, Che Jiang... ·1 min read
Authors
Jincheng Zhong, Weizhi Wang, Che Jiang...
Categories
cs.CL, cs.SE
arXiv
https://arxiv.org/abs/2606.23654v1
PDF
https://arxiv.org/pdf/2606.23654v1

Brief

EnterpriseClawBench is a benchmark built from proprietary, real-world workplace agent sessions that yields 852 reproducible tasks, each annotated with fixtures, rewritten prompts, role and skill taxonomies, hard rules, and semantic rubrics. Because the underlying enterprise content is private, the paper releases the construction/evaluation protocol and code; the best tested setup scores 0.663, underscoring the need for multi-dimensional evaluation metrics.

Why it matters

EnterpriseClawBench reproduces 852 tasks from real workplace agent sessions (paper posted 2026-06-22); each task is paired with recovered fixtures, rewritten prompts, role classes, skill subclasses, hard rules, and semantic rubrics. The raw enterprise data are not released; the authors publish a reusable construction and evaluation protocol and code at https://github.com/FrontisAI/EnterpriseClawBench.

Key details

  • Benchmark results show the best configuration reaches 0.663 (Codex combined with GPT-5.5). The authors highlight that enterprise-agent evaluation must report harness–model combinations, artifact delivery quality, visual quality, cost, runtime, and skill-transfer behavior rather than a single aggregated score.
Source evidence

Abstract

Enterprise agents increasingly operate inside workspaces: they read heterogeneous files, invoke tools, and deliver business artifacts. We introduce EnterpriseClawBench, an enterprise agent benchmark constructed from proprietary, real-world agent sessions. Starting from a large archive of workplace sessions, the EnterpriseClawBench produces 852 reproducible tasks, each paired with recovered fixtures, rewritten prompts, role classes, skill subclasses, hard rules, and semantic rubrics. Because the sessions contain internal enterprise content, we do not release the benchmark data; instead, our reusable contribution is the construction and evaluation protocol. On EnterpriseClawBench, the best configuration reaches only 0.663 (Codex with GPT-5.5). These results show that enterprise agent evaluation must report harness--model combinations, artifact delivery, visual quality, cost, runtime, and skill-transfer behavior, rather than collapsing performance into a single score. Code: https://github.com/FrontisAI/EnterpriseClawBench