Twitter/X

Alex Atallah and OpenRouter launched Ori Eval, a new CLI announced 2026-08-03…

2026-08-03 · 15:49 UTC ·@alexatallah ·0 min read

Brief

Ori Eval is a newly released CLI from Alex Atallah/OpenRouter (announced 2026-08-03) that packages their customer-facing process for writing evaluations and selecting models. It lets developers run evaluations across 500+ OpenRouter models using benchmarks, real-world usage metrics and OpenRouter APIs to compare performance on tasks, and the team asks for user feedback.

Why it matters

Alex Atallah and OpenRouter launched Ori Eval, a new CLI announced 2026-08-03 that helps users write evals and pick models for tasks by leveraging OpenRouter's data.

Key details

  • Ori Eval exposes 500+ models available on OpenRouter and uses benchmarks plus real-world usage data and OpenRouter APIs to evaluate model outputs within your codebase.
  • The tool is brand new and the team is soliciting feedback from users; announcement includes a curl install/reference link to openrouter.ai/skills/spawn-o…
Source evidence

We spend a ton of time with customers struggling to write evals and figure out the best model for each task.

This CLI wraps that experience into a single tool, allowing you to leverage all 500+ models on OpenRouter, their benchmarks, and real-world usage data.

It's brand new. Give us feedback!

OpenRouter (@OpenRouter)

Introducing Ori Eval: the easiest way to write your first eval.

There's no definitive best model, only the best model for each task. Ori Eval leverages OpenRouter's APIs for each task in your codebase, and then evaluates the results.

curl -fsSL openrouter.ai/skills/spawn-o…

— https://nitter.net/OpenRouter/status/2084301100078027143#m