Abstract
Minimal Markovization via Stable Quotients in Holonomy-Cover Decision Processes
- Authors
- Zuyuan Zhang, Yongshan Chen, Mahdi Imani...
- Categories
- cs.LG
Brief
Holonomy-cover decision processes address partial observability where visible dynamics are Markov and hidden modes undergo fixed permutations. The paper defines the stable quotient as the coarsest reward- and successor-preserving observation abstraction, shows (observation, stable class) yields an exact finite Markov state, proves memory minimality under reachability/separation, gives exponential error bounds under resettable diagnostics, and presents a Holonomy Memory RL pipeline with experiments matching the quotient oracle.
Why it matters
Zhang et al. (arXiv 2026-07-29) construct the stable quotient—the coarsest observation-wise abstraction that preserves one-step rewards and quotient successors—and prove that (current observation, stable class) is an exact finite Markov state for holonomy-cover decision processes.
Key details
- They prove minimality of memory: with correct initialization and under reachability plus pairwise decision separation at a maximizing observation, exact class tracking requires exactly the minimal number of memory symbols—no arbitrary finite-memory controller can use fewer.
- With resettable diagnostics, nearest-prototype class inference has exponentially decaying error; they introduce Holonomy Memory Reinforcement Learning (ordered edge transports, local class coordinates, then finite-MDP RL), and experiments recover exact state compression and perfect paired-order accuracy using three decision-time memory states (matching the quotient oracle).