Creating GPT-4 was once an all-hands-on-deck operation at OpenAI, involving hundreds of engineers and researchers. But now, thanks to advancements made during the development of GPT-4.5, the company says it could rebuild that same model with just five to ten people.
In a recent company podcast, OpenAI CEO Sam Altman posed a surprising question to a few of his top engineers: How many people would it take to retrain GPT-4 today? The answer stunned even him.
“It took hundreds of people, almost all of OpenAI’s effort, to build GPT-4,” Altman said. But retraining it now? According to Alex Paino, the engineer who led pretraining for GPT-4.5, that number has dropped dramatically. “It’d probably take five to ten people,” he said.
That’s not because GPT-4 has gotten any smaller or simpler. Rather, the OpenAI team now has a clearer roadmap — battle-tested tools, established infrastructure, and lessons learned from scaling GPT-4.5, which Paino described as “10 times as smart as GPT-4.” And as he noted, their GPT-4o model — trained using the insights from GPT-4.5 — was built with a leaner, more efficient team.
OpenAI researcher Daniel Selsam added another insight: once something has been done, even once, it unlocks a massive shortcut for those who follow. “Just finding out someone else did something — it becomes immensely easier. I feel like just the fact that something is possible is a huge cheat code,” he said.
Altman highlighted another major shift: compute is no longer the primary bottleneck for OpenAI’s best models. “It is a crazy update,” he remarked. “For so long, we lived in a world where compute was always the limiting factor.”
But that world is changing fast. With Big Tech giants like Microsoft, Google, Amazon, and Meta expected to pour $320 billion into AI infrastructure this year, access to raw computing power is expanding at a historic pace. OpenAI itself recently closed the largest private tech funding round ever, securing $30 billion from SoftBank and another $10 billion from various investors, bringing its valuation to a staggering $300 billion.
Nvidia CEO Jensen Huang echoed this during an earnings call, predicting that AI models, particularly those capable of reasoning, will require exponentially more compute. “Reasoning models can consume 100x more compute. Future reasoning can consume much more compute,” he said.
Yet, computing won’t get us to the next level. Selsam pointed out that we’re approaching a new ceiling: data. While models like GPT-4 are highly efficient at processing information, there’s only so much insight they can extract from the available training data. As computing continues to grow faster than data, the bottleneck begins to shift.
“To push beyond that,” Selsam said, “we’ll need algorithmic innovations, new methods to squeeze deeper insights from the same data.”
