What is Gradient?

A training hyperparameter controlling how much parameters change each update step.

An empirical relationship showing how performance improves with model size, data, or compute.

An attention pattern where each token attends only to a fixed-size window of nearby tokens to reduce compute.

A generative setup where a generator and discriminator train against each other.

How well a model performs on new, unseen data outside the training set.

An optimization method that updates parameters in the direction that reduces error.