Twitter/X

Andrej Karpathy ran the same bug through five models and showed that the paid…

2026-08-05 · 12:00 UTC ·@phosphenq ·1 min read

Brief

Phosphen (@phosphenq, 2026-08-05) highlights an Andrej Karpathy demo that ran one bug across five models—showing the $200 tier plus free Claude, Gemini, Grok and DeepSeek detected it—and explains why top models confidently lie about recent events and why fresh chats improve accuracy. Phosphen urges watching the trimmed video, reading the guide, and building a first loop; they also note a Claude Code run that spawned 105 agents, cost 5,044,822 tokens, and ran 36 minutes.

Why it matters

Andrej Karpathy ran the same bug through five models and showed that the paid $200 tier found it, and free models Claude, Gemini, Grok and DeepSeek also found it.

Key details

  • Karpathy demonstrates why the so-called 'smartest' model confidently hallucinates about recent information and why starting a fresh chat makes the next answer more accurate.
  • Phosphen (@phosphenq) says they’d been using about 10% of the tool, cut the video intro to the good part, and highlights a Claude Code run that spawned 105 agents, cost 5,044,822 tokens, and took 36 minutes — a run most Claude Code users haven’t tried.
Cleaned source text

you use chatgpt every day, and andrej karpathy just proved you're doing almost all of it wrong.

he ran the same bug through five models, and the $200 tier found it, and so did claude, gemini, grok and deepseek for free.

he shows why your smartest model confidently lies about anything recent, and why a fresh chat makes the next answer more accurate.

i watched the whole thing and realized i'd been using maybe 10% of the tool.

i cut the intro so it opens where it gets good.

watch it tonight, then read the guide below and build your first loop.

Video

Phosphen (@phosphenq)

Article

How to Graph Engineering: The Complete Guide

Claude Code ships a feature that spawns 105 agents on a single question. I ran it this week. It cost 5,044,822 tokens and took 36 minutes.

Most people who use Claude Code every day have never run one.

— https://nitter.net/phosphenq/status/2083991226827567419#m