François Chollet says that during the 2022–2024 base‑LLM scaling era he expected…

Brief

François Chollet recounts that although he expected a plateau for base LLMs in 2022–2024, the late‑2024 o3 test‑time compute demo convinced him newer models show fluid intelligence and could allow unbounded scaling. Still, he expects the dominant long‑term architecture (~15 years) to shift toward symbolic learning because present methods are 4–6 orders of magnitude inefficient, and ongoing research will produce new techniques.

Why it matters

François Chollet says that during the 2022–2024 base‑LLM scaling era he expected a capability plateau, but after the late‑2024 o3 test‑time compute demo he changed his view: the new models showed genuine fluid intelligence and could enable unbounded capability scaling (“There will be no wall”).

Key details

Chollet predicts that roughly 15 years out, future AI will not be based on the LLM stack but will move toward symbolic learning as the optimal final form; he calls this a risky/contrarian belief compared with the safer bet of LRMs.

He claims current techniques are 4–6 orders of magnitude away from optimality in data efficiency and test‑time compute efficiency, yet argues continued research investment means technique limits will be overcome by new methods.

In the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs).

In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall." I talked about it at length on Twitter and in a blog post.

However, looking ahead, I still do not believe that future AI (say, in 15 years) will be based on the LLM stack. I believe it will necessarily have to move closer to its optimal, final form -- symbolic learning. Obviously this is a risky and contrarian belief -- the safe bet would be LRMs. But let's see.

The only meaningful difference is efficiency, not task-specific skill. I believe current techniques are 4-6 orders of magnitude away from optimality in terms of data efficiency and test-time compute efficiency. But far future AI will be near-optimal.

François Chollet (@fchollet)

The limitations of specific techniques are predictable and correspondingly lead to plateaus for those techniques. But there is always the next technique, building on top of the pile that's already available.

There is enough research investment that there will be no wall.

— https://nitter.net/fchollet/status/1870194696477388836#m