AI Glossary Term

What does “Loss Function” mean?

Definition

A mathematical objective that quantifies prediction error during training.

Browse the full AI GlossaryExplore all free AI guides

How Loss Function works

Diagram for Loss Function, a repeating loop: Model predicts; then Loss function scores the error; then Optimizer lowers the loss; then Weights update; then back to the start.Model predictsLoss function scores the errorOptimizer lowers the lossWeights updaterepeat

Example

In a house-price model, the loss might be the average squared difference between predicted and actual prices. Training tries to make that single number as small as possible.

Why it matters

The loss defines what the model is rewarded for. A loss that does not match your real goal can lead to a model that scores well but is not useful.

Common questions

Is loss the same as accuracy?

No. Loss is a smooth training signal; accuracy is a human-friendly measure. They often move together but not always.

What are common loss functions?

Mean squared error for numbers and cross-entropy for classes and next-token prediction.

Can I design my own?

Yes, and teams often do when the standard ones do not reflect their goals.

Related terms

Training LossThe model error value computed during training and optimized downward over time.Cross-Entropy LossA common objective function used to train classification models by penalizing incorrect probabilities.BackpropagationThe core training algorithm that updates model weights by propagating prediction errors backward through the network.Learning RateA training hyperparameter controlling how much parameters change each update step.LLM-as-JudgeUsing a language model to score or compare outputs from other models during evaluation.BiasA consistent pattern of error or unfairness in data or model behavior.

Learn more in our free guides

Focal Loss for Imbalanced DetectionTriplet Loss and Metric LearningSigned Distance FunctionsSiamese Networks and Triplet Loss

Test yourself

Focal Loss for Imbalanced Detection QuizTriplet Loss and Metric Learning QuizSigned Distance Functions Quiz

See also

LoRA (Low-Rank Adaptation)Machine Learning (ML)Memory (Agent Memory)Large Language Model (LLM)Mixture of Experts (MoE)LatencyModel CardLabel Smoothing