Reader · no content
No body text on file.
Open the original to read the full piece.
Anjney Midha's Plan to Radically Lower the Price of Compute
Brief
Midha also emphasized the technical and epistemic contours of progress in AI. He broke the model development pipeline into pretraining, mid‑training, deployment and a continuous verifiable feedback loop; where feedback is objectively verifiable (unit tests, PR approvals, lab measurements like X‑ray diffraction) capabilities improve fastest. He used examples from software engineering and his Periodic Labs materials program to show how verification closes the loop and reduces hallucination. Midha pushed back on the idea that frontier models are already at parity — saying frontiers are multiple (software engineering, consumer chat, video, materials) and that model+‘harness’ co‑design (tools and orchestration layered atop models) is critical to real‑world performance and cost efficiency. He warned executives against naive sandboxing: technical literacy is essential because black‑box deployment leads to misuse (hallucinations, prompt injection) and suboptimal cost/ROI. The hosts agreed the conversation pointed toward commodification at the user level — customers will demand cheap, reliable services — while AMP’s grid aims to deliver that by coordinating capacity, forecasting demand, and resisting speculative financialization of compute.
Why it matters
Anjney Midha (guest, Speaker 5) founded AMP PBC to standardize compute into a fungible “grid” and sell consumption as grid credits; the system is software-only (a BORG-like translation layer) that lets researchers ignore underlying chip types and raises utilization from typical industry levels (<70%) toward ~90–96% (Midha cites lab examples reaching ~95–96%).
Key details
- Midha traced the frontier-model recipe as four steps—pretraining, mid‑training, post‑training/deployment and a continuous verifiable feedback loop—and emphasized verifiable feedback (unit tests, PR approvals for code; lab synthesis + XRD verification for materials work at his Periodic Labs) as the fastest path to capability gains.
- Midha recounted his Anthropic role: he wrote an early check, helped raise ~ $100M in an initial angel-heavy round (late 2020/early 2021), and later the team secured a ~$4 billion compute/capital partnership with Amazon; he argued Anthropic (~5,000+ employees) is not at parity with Google/DeepMind/OpenAI in day-to-day performance despite headline proximity.
- AMP’s technical approach (Midha) is to make heterogeneous compute fungible via software translation; his cofounder Sebastian Lobo — who led Google’s internal BORG — previously increased Google cluster utilization from ~62% to ~99%, an engineering precedent AMP is replicating for other labs.
- Midha explained the economic root of the bottleneck: long-term leased GPU capacity and spiky research demand create massive waste, inflating the effective per-GPU-hour cost from marketed ~$2.50 to an effective ~$25–$28 after overprovisioning; AMP reallocates unused reserved capacity to other users to recover that gap.
- He warned leaders not to outsource technical literacy: sandboxing a black‑box model is insufficient because teams can still misuse models (hallucinations, prompt injection); Midha argues executives should understand end‑to‑end systems (he teaches Stanford CS153 Frontier Systems and posts lectures online).