AI Glossary Term
What does “Normalization” mean?
Definition
Transforming values to a consistent scale to improve optimization stability.
How Normalization works
Example
A model using age (about 20 to 80) and income (thousands to hundreds of thousands) can be dominated by income's large numbers. Rescaling both to similar ranges lets training treat them fairly.
Why it matters
Consistent scales make optimization more stable and faster. Layer normalization is also a standard part of transformer networks.
Common questions
Normalization vs standardization?
Normalization often means scaling to a fixed range such as 0 to 1. Standardization shifts to mean 0 and standard deviation 1. Usage varies.
Do I apply it to test data?
Yes, using the same scaling values computed from the training data, to avoid leakage.
What are batch and layer normalization?
Methods inside neural networks that normalize activations to keep training stable.