Scaling Law

Scaling laws are empirical rules for estimating how an AI's performance changes as you scale up training data, parameter count (a measure of model size), and the computing power used for training.
It's a statistical trend drawn from many experiments, not a formula guaranteeing that increasing these inputs always makes an AI smarter — more a rough guide to what performance level to expect under given conditions.
Put simply, performance keeps improving along a fairly consistent pattern, even as the pace of improvement gradually slows.
On top of that, you can't just blindly increase data, parameters, and computing power all at once and expect the best results.
How you split a limited budget across these three factors matters a lot. Early in AI development, some models had far more money poured into parameters than into training data to match.
Later research found a more effective approach: increasing parameter count and training data at roughly the same rate is the efficient way to scale.

This principle was published by an AI research group and reshaped the direction of AI development that followed.
Before that, nobody could say for certain how large an AI needed to be to actually get smarter. This principle let developers estimate roughly how much budget and data a target performance level would need.
Some of the world's most widely used, high-performance conversational AI systems emerged from major investment guided by this principle, combined with other techniques to align responses with how humans naturally converse.