Everything has limits, of course. But if you've not yet seen a deceleration, it's reasonable to expect that at the least you're in the middle of the sigmoid, not the top.
We have though. The velocity remains high but the acceleration is decreasing.
The improvement between gpt 2-4 where staggering. to 5, 5.6? Much less so.
The results improve of course, but the difference is no longer mind blowing.
Most of the improvement now has come from agentic harnessing, which is unrelated to the acceleration of the model but the tooling around the use of the model.
So we are seeing a deceleration and the speed is still high due to high prev acceleration but its not growing at the same rate as before and the edges are starting to show themselves
I literally don’t understand, and have a deep distrust of people who say we haven’t seen deceleration.
The diminishing returns are colossal. GPT 4 is more than 3 years old now, and while Sol is definitely better, it’s better at a tremendous cost. And 3 years of development. And it still fails in nearly all the same places as 4.0.
I’m not some “zero AI” person, I don’t think this industry will collapse into nothing and we will go back to a no LLM world.
But to look at this industry, which basically began in November 2022, almost 4 years and a trillion+ dollars later and say “oh yeah, definitely riding that exponential growth still” is some of the wildest shit I’ve ever seen, and it’s so common.
I'm the other way around. For doing real, useful work, these things were barely more than toys even just a year ago. Now they're very capable when used well, and still getting better at a fast pace.
