My December 2024 takes were "there will be no wall", "if the only bottleneck is test-time search, we will see continued scaling in the future", and "the world is once again about to run out of GPUs"
To this day, I do expect post-training scaling and TTA scaling to continue to deliver. At least 10-100x from current levels.
nitter.net/fchollet/status/187019…
François Chollet (@fchollet)
The limitations of specific techniques are predictable and correspondingly lead to plateaus for those techniques. But there is always the next technique, building on top of the pile that's already available.
There is enough research investment that there will be no wall.
— https://nitter.net/fchollet/status/1870194696477388836#m