Humanity’s Last Problem

Ben Hylak’s March 16, 2026 essay presents AI-agent monitoring as a defining operational problem of the near future. Borrowing from Kurt Vonnegut’s 'Player Piano,' he argues that as agents become responsible for a growing share of economic activity, their errors will become both more frequent in strange ways and more consequential in practice. The article’s technical core is a two-part argument: rising capabilities increase system complexity, making comprehensive evals impossible, and those same systems are being deployed into domains where errors carry outsized cost, from production infrastructure and security to law, medicine, and finance. Hylak contends that production behavior, not lab testing, increasingly becomes the real source of truth for agent reliability. He broadens the claim into a social prediction, suggesting many knowledge-work functions—from translation to software engineering—are already being automated, leaving humans with the narrowing task of recognizing when an agent’s output can still be improved.