On the first point - Epoch finds that in language models, pretraining algorithmic progress has been around half as impactful as compute scale up. Naively, if compute scale up stopped, progress would slow down by 3x. This is a decent amount, but not enough to say “2030 or bust”

Now, maybe you think without compute scale up, algorithmic progress would be slower. But note that even if scale up of the largest training runs has to stop, algorithmic progress is more dependent on experimental compute, which may hit limits later (tho I’m not sure when tbh)

And add to that Moore’s law would keep going, adding 2x as much compute (for both training & experiments) every 1-2 years. Not the sort of scale up in compute we see now, but not nothing.

Okay, what about the second point about unhobblings? The word “unhobblings” makes it sound like some sort of minor thing that’s pretty easy - in reality, we’re talking about everything from RLHF to all sorts of agent scaffolding, advancements for reasoning models, etc

These can be a huge deal! And they aren’t necessarily so simple that all the main advances would be achieved in a few years. (I probably should have used the term “post-training enhancements” instead.) and if compute scale up slows down, more efforts will go here

Add to all that, we might not be *that* far from AGI by the time the compute scale up slows down, and AI researchers would have help in AI R&D from all sorts of pre-AGI systems that are substantially more advanced than current systems.

Considering all that, not hard to imagine that by 2030, we’re 2-3 years away from AGI at today’s rate, and with the compute slow down we wind up getting to AGI sometime in the mid- or late- 2030s

@daniel_271828 yeah very much agree, bimodal agi theory seems quite arbitrary and abandons the same extrapolation that makes this decade compelling luckily i doubt these gentlemen actually believe we won’t have agi this decade