Twitter/X

Octen's launch reported a 6 ms gap between P50 and P90 search latency, a very…

2026-07-21 · 14:21 UTC ·@aakashgupta ·1 min read

Brief

Octen's search claims a 62 ms P50 latency with only a 6 ms P50–P90 spread, achieved by rebuilding index, ranking, and serving layers to support agent fan-out of ~50 sub-queries. @aakashgupta and Kuan Zou argue that low, consistent latency is critical because agents chain dozens of searches and variance would otherwise compound across steps.

Why it matters

Octen's launch reported a 6 ms gap between P50 and P90 search latency, a very tight spread that prevents latency variance from compounding across chained agent queries.

Key details

  • Octen achieved a 62 ms P50 search latency, which @aakashgupta says 'fits inside an agent's thinking loop.'
  • Because agents fan a question into roughly 50 sub-queries, Octen rebuilt the index, ranking, and serving layers to handle that fan-out (point highlighted by Kuan Zou / @KZouAPT).
Cleaned source text

The most interesting number in Octen's launch is the small one: a 6ms gap between P50 and P90 search latency.

Agents chain dozens of searches per task, so variance compounds. A spread that tight means step 40 of a chain behaves like step 1.

Search was built for one person typing one query. An agent fans a question into 50 sub-queries at once, so Octen rebuilt the index, ranking, and serving layer for that fan-out. At 62ms P50, search fits inside an agent's thinking loop.

Kuan Zou (@KZouAPT)

Article

We stayed quiet and built the FASTEST search on Earth

Search was built for humans. You type one query, scan the results, refine, and try again.

Thirty years of infrastructure is designed around that loop.

AI agents DON'T work that way. An agent wants to

— https://nitter.net/KZouAPT/status/2079569508549673409#m