AI Glossary Term
What does “Gradient” mean?
Definition
A vector showing how much each parameter should change to reduce loss.
How Gradient works
Example
If increasing one weight slightly makes the loss go up, its gradient is positive and training lowers that weight. With millions of weights, the gradient is a list of such directions, one per weight.
Why it matters
The gradient tells training how to improve. Without it, adjusting millions of parameters would be a guessing game.
Common questions
Is a gradient a number or a vector?
For a single parameter it is a number. For a whole model it is a vector with one entry per parameter.
What is a vanishing gradient?
When gradients shrink toward zero in early layers, so those layers barely learn. Better activations and architectures reduce this.
Who computes gradients?
Software using automatic differentiation, which applies backpropagation behind the scenes.