On 2026-08-07 Matt Shumer (@mattshumer_) proposes an automated loop to preserve…

Brief

Matt Shumer (2026-08-07) recommends an iterative, automated benchmarking loop (Gauntlet Loop-style blind comparisons against Opus 4.8) to adapt and validate skills for Claude Opus 5. He also advises deleting existing skill files and shifting prompts from procedural instructions to outcome-focused requests, arguing Opus 5 excels if used differently from prior Claude versions.

Why it matters

On 2026-08-07 Matt Shumer (@mattshumer_) proposes an automated loop to preserve skills when migrating to Claude Opus 5: 1) run a first update pass to adapt skills to Opus 5, 2) create test suites/benchmarks per skill, 3) run tests on Opus 5 with updated skills vs Opus 4.8 with old skills, 4) have a separate model blind-compare (Gauntlet Loop-style), 5) iterate until Opus 5+updated skills consistently outperforms Opus 4.8.

Key details

Operational advice: delete ALL existing skills/MCPs/Claude.md and start fresh, and change prompting style—stop instructing step-by-step and instead state the desired outcome ("just say what you want done").

Claimed evaluation: Claude Opus 5 is "way better than people give it credit for," but will perform poorly if used like previous Claude models.

For those who want to keep their skills while using Opus 5, here's a trick you can try (let me know how it goes):

Write a loop (have a model do this!) that has Opus 5:
- do a first update pass on your skills to make them better for Opus 5
- creates a suite of test tasks for each skill, like a benchmark
- runs each test task on Opus 5 with the new skill, and Opus 4.8 with the old one
- has a separate model blind-compare the outcomes (Gauntlet Loop-style)
- repeatedly iterate until Opus 5 + updated skills is consistently better on the suites than Opus 4.8 + old skills

Matt Shumer (@mattshumer_)

Claude Opus 5 is way better than people give it credit for.

If you're using it like previous Claude models, it's going to suck.

Two things make a huge difference:
- Delete ALL your skills/MCPs/Claude.md/etc. Start fresh.
- Stop telling it how to do the thing. Just say what you want done.

These two small changes make a massive difference.

— https://nitter.net/mattshumer_/status/2085776751150485642#m