two things are true here.
this is an impressive score and it’s clear that arc agi 3 is saturated.
this is not apples to apples with other scores you’ve seen because arc themselves don’t allow any type
of harness. also, it’s only on the test set, which is much easier.
this is more a problem with arc imo.
we’re not going to discount a cure for cancer because it a model used a harness or wasn’t a ‘pure llm’, whatever that is.
🍓🍓🍓 (@iruletheworldmo)
arc agi 3 has fallen. btw, ilyas model crushes this with no harness.
happy hunting asi chasers. this is what the singularity feels like. yeeeehaww.
— https://nitter.net/iruletheworldmo/status/2085122893432672501#m