A central question in AI is whether scaling up current approaches will produce general intelligence.
Scaling Laws
Research found that model performance improves predictably as models, data and compute grow. Larger models also showed new abilities that smaller ones lacked.
The Scaling View
Some argue that continued scaling — together with improvements like reinforcement learning, longer reasoning and better data — will keep expanding capabilities towards general intelligence.
The Sceptical View
Others argue current methods have fundamental limits:
- Reliance on vast data, and possible shortages of high-quality text.
- Weaknesses in reliable reasoning and planning.
- Difficulty learning continuously from experience.
- Lack of grounding in the physical world.
They argue new ideas will be needed.
Beyond Pre-Training
Recent gains have come not only from bigger pre-training runs but also from techniques applied after training and at inference time, such as reasoning with more computation.
Practical Constraints
Compute, energy, chips and cost limit how far scaling can go, and shape who can build frontier systems.
The Honest Answer
Nobody knows for certain. Watching which capabilities improve, and which stubbornly don't, is the best evidence.