AI Glossary Term

What does “Learning Rate” mean?

Definition

A training hyperparameter controlling how much parameters change each update step.

Browse the full AI Glossary Explore all free AI guides

How Learning Rate works

Example

If you set the learning rate too high, each update overshoots, and the loss bounces around or rises. Set too low, training crawls and may stop improving long before it reaches a good result.

Why it matters

It is one of the most influential training settings. Many failed training runs are traced to a poor learning rate.

Common questions

What is a learning-rate schedule?

A plan that changes the rate during training, such as warming up first and then decaying gradually.

Typical values?

They vary widely by model and optimizer. Values like 0.001 or smaller are common starting points, but there is no universal answer.

How do I find a good one?

Try several values and compare validation results, or use a learning-rate finder tool.

GradientA vector showing how much each parameter should change to reduce loss.HyperparameterA configuration value set before training, such as learning rate, batch size, or depth.GeneralizationHow well a model performs on new, unseen data outside the training set.Parameter-Efficient Fine-Tuning (PEFT)Methods that adapt models by training a small subset of added parameters.Beam SearchA decoding algorithm that keeps the top several candidate sequences at each step to find higher-probability outputs.Loss FunctionA mathematical objective that quantifies prediction error during training.

Learn more in our free guides

Deep Learning Reinforcement Learning Machine Learning Basics Supervised Learning

Test yourself

Deep Learning Quiz Reinforcement Learning Quiz Machine Learning Basics Quiz