For a few years, the AI world operated on a simple, almost religious belief: if you throw more data and more compute at a model, it will inevitably get smarter. This wasn't just a guess; it was based on the first wave of scaling laws is a set of empirical observations describing how a model's performance improves predictably as model size, dataset size, and compute increase. But as it turns out, these laws aren't universal truths-they're more like guidelines that break the moment you try to use them in the real world.
The problem is that we often confuse "minimizing loss" with "making a useful product." While the math might tell us that a bigger model reduces a specific error metric, it doesn't always mean the AI is better at reasoning, safer to use, or cheaper to run. When we push these models into production, the neat power laws start to crumble.
The Chinchilla Shock: Bigger Isn't Always Better
Early on, researchers followed the Kaplan scaling laws, which suggested that model size was the primary lever for performance. This led to a "billion-parameter arms race" where the goal was simply to build the biggest brain possible. However, DeepMind stepped in with the Chinchilla study, which fundamentally changed how we think about compute allocation. They discovered that most models were "under-trained"-they had massive architectures but weren't given enough data to actually saturate their potential.
The Chinchilla findings proved that for every single single unit of compute increase, you should scale the model size and the amount of training data in equal proportions. If you have 10x more compute, you don't just make the model 10x bigger; you make the model about 3.1x larger and the dataset 3.1x larger. To prove this, DeepMind built Chinchilla, a 70-billion parameter model. Despite being far smaller than the 175-billion parameter GPT-3, Chinchilla crushed it because it was fed 1.4 trillion tokens-far more than the 300 billion tokens used for their previous, larger model, Gopher.
| Strategy | Primary Focus | Key Outcome | Risk |
|---|---|---|---|
| Kaplan Scaling | Model Size | Rapid loss reduction in early stages | Data inefficiency (Under-training) |
| Chinchilla Optimality | Balanced Model/Data | Maximum training efficiency | Higher data acquisition costs |
| Overtraining | Inference Performance | Cheaper, faster production models | Diminishing returns on training compute |
The Gap Between Training Loss and Production Reality
There is a massive difference between "Chinchilla optimality" and what actually works in a product. Chinchilla optimality focuses on the most efficient way to reach a certain level of loss during training. But once a model is deployed, the goal changes. You don't care how much compute it took to train; you care how fast it responds to a user and how accurate it is during inference.
This realization led to the rise of "overtraining." For example, the LLaMA models were intentionally trained on far more data than the Chinchilla laws would suggest is "optimal." Why? Because spending extra compute during the training phase makes the model much more capable during the inference phase. In some cases, researchers have pushed dataset sizes up to 32x beyond the Chinchilla-optimal limit. Essentially, the "laws" of efficient training fail when the actual objective is a high-quality user experience.
Where RL Scaling Completely Breaks Down
If pretraining is a well-mapped highway, Reinforcement Learning (RL) is a jungle. In pretraining, we have rigorous power laws that let us predict exactly how much better a model will get if we add more tokens. In RL, those predictions almost completely vanish.
RL doesn't follow a simple linear or power-law relationship because it's incredibly unstable. A single token in a sequence can cause a massive spike in the loss expression, leading to numerical instability-especially when using complex architectures like Mixture-of-Experts (MoE) models. Because of this variance, the "best practices" for RL are often anecdotal. What works for one model might fail spectacularly for another, meaning researchers have to test everything "the hard way" through expensive trial and error rather than relying on a mathematical formula.
The Danger Zone: Adversarial and Safety Scaling
One of the most worrying discoveries is that safety doesn't scale the same way capabilities do. You might think that as a model gets smarter, it becomes easier to align or keep safe. In reality, jailbreak scaling laws follow a different mathematical beast entirely.
When a user tries to bypass a model's safety filters using adversarial prompts, the success rate doesn't grow slowly. Instead, it can grow exponentially based on the number of inference-time samples. While benign capabilities scale predictably, adversarial attacks behave like a spin-glass system. Short, weak prompts might follow a power law, but long, strategically crafted injections act like strong magnetic fields, pushing the model toward unsafe clusters with exponential speed. This means your safety guardrails can't just "scale up" alongside the model's intelligence; they require a completely different strategy.
The New Frontier: Test-Time Scaling
We are now seeing a shift from training-time scaling to test-time scaling. Instead of just making the model bigger before it ships, we give it more compute while it is thinking. This is the secret behind reasoning models that perform multiple internal passes or "chains of thought" to solve a complex math problem.
This represents a fundamental shift in the efficiency frontier. The compute used for a single inference pass to "reason" is a different cost-benefit calculation than the compute used to train the model on a trillion tokens. It suggests that the next leap in AI won't come from simply adding more GPUs to a training cluster, but from optimizing how models use compute at the moment of execution.
Summary of Scaling Failures
If we look at the patterns, it's clear that scaling laws are context-dependent. They work great in a lab setting where the goal is to minimize a mathematical loss function on a static dataset. But they break when they hit real-world friction: finite high-quality data, the need for low-latency inference, the instability of RL, and the unpredictability of human attackers.
The lesson here is that the "universal law" of scaling is actually a collection of specific assumptions. When those assumptions change-such as moving from training to deployment-the law changes with them. Scaling is still a powerful tool, but it is no longer a magic wand that guarantees progress.
What exactly is Chinchilla optimality?
Chinchilla optimality is the idea that for a fixed compute budget, the most efficient way to train a model is to scale the number of parameters and the number of training tokens in equal proportions. It debunked the idea that simply making a model larger is the best way to improve performance.
Why do researchers "overtrain" models if it's not optimal?
Overtraining is done because Chinchilla optimality only minimizes training loss. In the real world, we want models that are highly capable but small enough to run quickly during inference. By training a small model on far more data than "optimal," we get a model that performs better in production and is cheaper to run for the end user.
Do scaling laws work for everything in AI?
No. While they work well for pretraining (predicting loss based on data/compute), they fail in Reinforcement Learning (RL) due to high variance and instability, and they follow entirely different, often exponential, paths in adversarial and safety contexts.
What is test-time scaling?
Test-time scaling refers to increasing the amount of computation a model uses during the inference phase (when it's answering a prompt) to improve its reasoning capabilities, rather than just relying on the knowledge baked in during training.
Why is RL scaling so much harder than pretraining scaling?
Pretraining is predictable because it's based on power laws of data and compute. RL is unstable because single tokens can dominate the loss expression, leading to policy gradient variance. This makes it nearly impossible to predict performance improvements without actually running the experiment.
mark nine
April 26, 2026 AT 15:54basically just saying that throwing more money at the problem only works until it doesn't
the shift to test-time compute is where the real magic is happening right now since it lets smaller models punch way above their weight class without needing another 100k h100s just to train it once
Scott Perlman
April 27, 2026 AT 02:08this is great stuff
Sandi Johnson
April 27, 2026 AT 10:02oh wow imagine thinking a math formula would actually predict how a chaotic neural net behaves in the wild
truly shocking that the "laws" of AI are actually just vibes and a bit of guesswork
Eva Monhaut
April 27, 2026 AT 15:23The pivot toward test-time scaling is a breathtaking evolution in the field. It is like shifting from memorizing a textbook to actually learning how to think through a puzzle in real-time. This approach breathes new life into the efficiency debate by prioritizing cognitive agility over sheer brute-force scale. It transforms the model from a static encyclopedia into a dynamic reasoning engine, which is a far more elegant path toward intelligence than just piling on more parameters. I love how this acknowledges the nuance of human-like deliberation. By allowing the model a moment to pause and refine its internal logic, we are seeing a kaleidoscope of emergent capabilities that pretraining alone could never spark. This is a shimmering glimpse into a future where quality of thought outweighs the quantity of data. It makes the whole endeavor feel less like an industrial assembly line and more like an artistic pursuit of logic. We are finally moving past the era of mindless expansion and into an era of sophisticated refinement. The implications for accessibility are wonderful since smaller, smarter models are easier to deploy. Truly a vibrant shift in perspective for everyone involved in the AI odyssey.
mark nine
April 29, 2026 AT 14:50spot on. the efficiency gains from inference-time compute are way more sustainable for actual dev shops than trying to chase that chinchilla dragon
Rakesh Kumar
April 30, 2026 AT 20:18OH MY GOD! I never realized the safety side was so terrifying! The fact that jailbreaks scale exponentially is absolutely wild! It's like we're building a skyscraper and realizing the locks on the doors get easier to pick the taller the building gets! This is just mind-blowing!