Twitter/X

@johnloeber argues that if AI's rapid progress raises the probability we live in…

2026-08-02 · 19:15 UTC ·@johnloeber ·2 min read

Brief

John Loeber contends that AI progress, if taken as evidence we're in a simulation, probabilistically points to being inside ephemeral evaluation runs rather than a high‑effort core simulation. He cites Andrej Karpathy's example of Opus 5 using a 1M‑token (~$10) budget and ~2 hours to write ~5,500 lines of JS to render LOTR, showing both LLM creative stamina and a persistent inability to natively audit video/game outputs.

Why it matters

@johnloeber argues that if AI's rapid progress raises the probability we live in a simulation, it more likely means we're inside one of many short‑lived evaluation runs ('ephemeral evals') rather than a single 'core, high‑effort' simulation.

Key details

  • Andrej Karpathy reports Opus 5, given the first paragraph of Lord of the Rings with a 1,000,000‑token budget (~$10) and ~2 hours of run time, produced ~5,500 lines of JavaScript to procedurally render and animate the scene, producing janky but functional results.
  • Karpathy and Loeber highlight that LLMs' stamina enables highly custom, on‑demand worlds (e.g., 'ephemeral GTA of X'), but current models struggle to audit and iterate visual/game outputs because they cannot natively perceive video or play games, causing slow screenshot‑based checks and errors.
Source evidence

If you look at the ascendant progress of AI and think this probabilistically suggests we live in a simulation, then it probabilistically also follows that we do not live in the core, high-effort simulation, but just in one of many ephemeral evals

Andrej Karpathy (@karpathy)

We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It's kind of janky but fun. But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all.

I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free". There might be a lot more. But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand.

Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

Video

— https://nitter.net/karpathy/status/2083749667410727319#m