What does “Scaling Law” mean?

An attention pattern where each token attends only to a fixed-size window of nearby tokens to reduce compute.

A training setup where a model improves by generating data through interactions or competitions with copies of itself.

How clearly an AI system's logic, data sources, and limitations are documented and understandable.

A shift in real-world input data over time that can degrade model performance.

How well a model performs on new, unseen data outside the training set.

A vector showing how much each parameter should change to reduce loss.