Microsoft CTO Kevin Scott is of course not a reliable source due to conflict of interest and his position in the US corporate world.
If anything, the fact that he is doing damage control PR around “LLM scaling laws” suggests something is amiss. Let’s see how things develop.
Given Microsoft’s investment in OpenAI and strong marketing of its own Microsoft Copilot AI features, the company has a strong interest in maintaining the perception of continued progress, even if the tech stalls.
I believe this sums it up.
TLDR: he thinks the techniques are fine and you can just brute force them for the foreseeable future.
Yeah… Why does it sound dumb in so many level?
Because he’s a salesman, and he’s selling you bullshit.
What the experts are now saying is that it looks like the LLM approach to AI will require exponentially larger amounts of training data (and data processing) to achieve linear growth. Next generation AI models will cost ten times as much to train, and the generation after that will cost ten times as much again.
The whole thing is a giant con. Kevin is just trying to keep investor confidence floating for a little longer.
And the harder the sell, the worse the product.
lol I honestly needed to open the article to parse the title. That’s why I posted.
But I’m definitely of the belief that you need a hell of a lot more architecture than they have to go meaningfully further. Humans are a hell of a lot more complicated than a bit like of neurons.
