Twitter/X

Gavin Baker: companies cutting reported AI spend can still increase token usage…

2026-08-04 · 19:02 UTC ·@patrick_oshag ·1 min read

Brief

Gavin Baker argues that reported cuts in AI spend aren’t a sign of falling demand: firms route traffic from expensive proprietary models to cheaper open‑source models and inference clouds, lowering spend (90%→30% gross‑margin shift) while token output and GPU hours can stay the same or increase. Patrick O’Shaughnessy’s interview (his seventh with Gavin) situates this insight among GPU price dynamics, financing, Claude’s market role, and datacenter strategy.

Why it matters

Gavin Baker: companies cutting reported AI spend can still increase token usage and GPU hours by shifting from high-margin proprietary tokens (~90% gross margin) to cheaper open‑source tokens (~30% gross margin); tokens require the same compute to produce, so cost per token falls but compute consumption may rise.

Key details

  • Patrick O'Shaughnessy’s seventh conversation with Gavin covers why old GPUs are getting more expensive, open‑source vs frontier models, Claude’s outsized market influence, financing the AI buildout (cash flow vs debt), the memory supply war, Nvidia’s new playbook, China/open source dynamics, and SpaceX as a data‑center (video timestamps 0:00–71:11).
Source evidence

Gavin explains why companies cutting their AI spend is not bad for AI demand.

With open source models and inference clouds, AI spending can decrease even as token usage and GPU hours increase:

“You're literally just shifting tokens from really expensive tokens with like 90% gross margins to tokens with maybe let's call it a 30% gross margin. And that's where the savings are coming from, but the tokens cost the same amount of compute to produce.

All these big public companies are like, ‘Oh my god, my AI spend is 20X'd. I've burned my budget in three months.’

So they set up a router and that cuts their AI spend, but it may actually increase the amount of tokens that they are generating just by shifting them to cheaper open source tokens, and that's just more compute.

A company getting smarter about which model to use for which task, that may lead to a destabilization of their spend or even a decline, but it has nothing to do with the amount of GPU compute hours they are effectively consuming.”

Video

Patrick OShaughnessy (@patrick_oshag)

My seventh conversation with @GavinSBaker.

It's about the gap between what the market is doing and what companies are seeing. It's been a tough month or so for public AI names, but there's no sign of a slowdown on the ground in Silicon Valley.

We discuss:
- Why old GPUs are getting more expensive
- Open source vs the frontier
- Claude as Wall Street's Walter Cronkite
- Whether the buildout gets funded out of cash flow or debt
- SpaceX as a data center company
- And more

Enjoy!

TIMESTAMPS
0:00 Intro
1:17 AI Selloff vs. Fundamentals
10:20 Financing the AI Buildout
18:18 GPU Prices Keep Rising
24:23 Claude Moves Markets
29:01 What Could Break the Thesis
36:59 The Memory Supply War
42:08 Nvidia’s New Playbook
50:14 China and Open Source
61:51 Data Centers and Regulation
71:11 SpaceX and Orbital Compute

Video

— https://nitter.net/patrick_oshag/status/2084610308279288141#m