Twitter/X

François Chollet (post dated 2026-08-02) says AI initially used a "patch (1)"…

2026-08-02 · 10:30 UTC ·@fchollet ·1 min read

Brief

François Chollet argues that after the field applied a widely demoed "patch (1)" (first shown December 2024) to avoid stalling, long-term progress will require a shift to "patch (2)." He frames two options: active inference for test-time adaptation (incremental progress) or replacing SGD with MDL and discrete program search, which would abandon current deep-learning curve fitting.

Why it matters

François Chollet (post dated 2026-08-02) says AI initially used a "patch (1)" approach that was demoed in December 2024 (nine months after it began) and has since become "completely ubiquitous."

Key details

  • He presents two remedies: (1) active inference—models adapt their learned programs at test time, which he views as potentially meaningful but only incremental; (2) replace SGD with a More Robust training principle such as MDL, requiring moving away from deep-learning curve fitting toward discrete program search (an approach he has advocated for years).
Source evidence

To address the limits of deep learning and avoid stalling, the field of AI started by applying patch (1), which started being demoed 9 months later in December 2024 and has now become completely ubiquitous. However, long term, it is simply inevitable that AI will move to patch (2).

François Chollet (@fchollet)

There are essentially two main options to remedy this:

  1. Find ways to perform active inference, so that the model adapts its learned program in contact with a new data distribution at test time. Would likely lead to some meaningful progress, but it isn't the ultimate solution, more of an incremental improvement.

  2. Change the training mechanism to something more robust than SGD, such as the MDL principle. This would pretty much require moving away from deep learning (curve fitting) altogether and embracing discrete program search instead (which I have advocated for many years as a way to tackle reasoning problems...)

— https://nitter.net/fchollet/status/1766044732830999014#m