On 2026-08-04 Alex Atallah announced he incorporated community feedback into…

Brief

Alex Atallah posted on 2026-08-04 that community feedback has been integrated into OpenRouter’s Ori Eval (openrouter.ai/ori/eval). He emphasized Ori is more than evals, described Ori Eval’s features—running agents on prompts, asserting tool usage, and LLM-judged grading to detect regressions—and invited cloud-agent builders to DM Jacky to trial a new eval approach.

Key details

Atallah said “Ori is way bigger than just evals,” invited cloud-agent builders to DM Jacky to try a new eval workflow, and described Ori Eval as running agents on prompts, asserting tool calls, and grading answers with an LLM judge to catch regressions and pick the best model.

Thanks for all your feedback here, now incorporated in openrouter.ai/ori/eval

But Ori is way bigger than just evals!

If you're building cloud agents, DM Jacky again to try out something new :)

Link

Ori Eval: Find the Best Model for Your Project | OpenRouter

Find the best model for your project. Ori Eval runs your agent and model on your prompts, asserts on the tools it called, and grades the answers with an LLM judge, so you catch regressions and pick...
openrouter.ai

Alex Atallah (@alexatallah)

We're working on an exciting new way to write evals.

DM Jacky if you'd like to try it out!

— https://nitter.net/alexatallah/status/2079603435867869652#m

Brief

Why it matters

Key details