Tag: scaling laws
Transfer and Emergence: When LLM Capabilities Appear at Scale
Discover how large language models develop sudden, unpredictable capabilities through emergence. Learn why scaling laws don't predict task performance jumps and what this means for AI safety and development.
Data-Centric vs Model-Centric Scaling: How to Actually Improve LLM Quality
Discover why the industry is shifting from model-centric to data-centric scaling for LLMs. Learn how data quality and compression outperform raw parameter count.
Do Thinking Tokens Change LLM Scaling Laws?
Discover how Thinking Tokens alter LLM scaling laws. Learn about Test-Time Scaling (TTTS), its impact on reasoning accuracy, and the trade-offs between speed and intelligence.
- 10
- Read More
Scaling Laws in Practice: When to Stop Training Large Language Models
Learn when to stop training LLMs using scaling laws. We explore Chinchilla optimal points, overtraining strategies, and practical metrics for compute-efficient model development.
Predicting Performance Gains from Scaling Large Language Models
Discover how scaling laws predict LLM performance gains. Learn about compute-optimal training, the Chinchilla effect, and test-time scaling strategies for efficient AI development.
Why LLM Scaling Laws Fail: The Hidden Limits of Model Growth
Explore why LLM scaling laws fail in real-world production. Learn about Chinchilla optimality, overtraining, and the limits of compute-driven AI growth.
Tokens per Parameter: How Much Data Large Language Models Really Need
Large language models need far more data than most people think. The key is tokens per parameter - and the magic number is 20. Learn why more data beats more parameters and how scaling laws shape today’s AI.