DungeonBench: A Benchmark for Rules-Rich Tactical Reasoning in Dungeons & Dragons Combat

2026-07-31 · 16:03 UTC ·Ismayil Ismayilov, Atakan Kara, Kaan Oktay ·1 min read

Brief

DungeonBench is a Dungeons & Dragons combat benchmark that simulates most combat-relevant 2014 SRD rules and exposes per-step tactical observations plus enumerated legal options. It offers Encounter (single-fight) and Day (linked encounters with persistent HP, spell slots, consumables, and short-rest timing) tracks, and evaluation shows frontier language models win many local fights but struggle with resource budgeting and long-run tactical discipline.

Why it matters

DungeonBench (Ismayilov, Kara, Oktay; arXiv 2026-07-31) is a benchmark for rules-rich tactical reasoning built to cover the vast majority of combat-relevant 2014 System Reference Document mechanics that the simulator can resolve; each decision step exposes a full tactical observation, a pending decision, and an indexed list of executable options (movement, attacks, spells, reactions, objectives, preparation, scarce resources).

Key details

DungeonBench provides two tracks—Encounter (single fights) and Day (linked encounters with persistent hit points, spell slots, consumables, preparation, and short-rest timing)—and a single engine-generated decision stream that supports heuristic controllers, language-model policies, learned option rankers, and masked-action RL; evaluations show frontier LMs often win direct encounters but fail on resource budgeting, rest timing, and rule-aware tactical discipline.

Abstract

Games and simulators make valuable benchmarks by turning decisions into measurable outcomes, but many current suites under-test rules-rich tactical reasoning: the ability to choose well when geometry, timing, resources, objectives, and rule interactions all matter at once. We introduce DungeonBench, a benchmark for tactical reasoning in Dungeons & Dragons combat, built to cover the vast majority of combat-relevant 2014 System Reference Document content whose effects can be resolved by the simulator while retaining mechanics that simplified combat simulators often abstract away. At each step, DungeonBench exposes a complete tactical observation, a pending decision, and an indexed list of executable options spanning movement, attacks, spells, reactions, objectives, preparation, and scarce resources. The task is to value legal choices whose consequences depend on action economy, creature traits, battlefield geometry, timing windows, and future encounters. DungeonBench has two tracks: Encounter, which evaluates local tactical play in single fights, and Day, which links encounters through persistent hit points, spell slots, consumables, preparation, and short-rest timing, forcing policies to trade off immediate tactical advantage against future survivability. The same engine-generated decision stream supports heuristic controllers, language-model policies, learned option rankers, and masked-action reinforcement-learning agents. We evaluate frontier language-model policies on this shared decision stream. Results show that full tactical observations do not saturate the benchmark: frontier policies often win direct encounters, but linked encounter days expose failures in resource budgeting, rest timing, and rule-aware tactical discipline.