Overfitting and Underfitting
Overfitting is when a model memorizes its training data and fails on new examples; underfitting is when it is too simple to capture the real pattern.
Overview
Hitting the sweet spot between them is the central challenge of machine learning.
Deep Dive
Every model is fit to a finite training set, but the goal is to perform well on unseen data. An overfit model treats noise and quirks of the training set as if they were real signal: it might score 99% on training data yet collapse to 70% on a test set. An underfit model is the opposite problem, too rigid to capture the underlying structure, so it does poorly on both training and test data. The gap between training and test performance is the telltale sign. Underfitting shows as high error everywhere (high bias); overfitting shows as low training error but high test error (high variance). The skill is recognizing which problem you have, because the fixes pull in opposite directions.
Technical Insight
Overfitting and underfitting are two ends of the bias-variance tradeoff. Bias is error from oversimplified assumptions; variance is error from being too sensitive to the specific training sample. A tiny linear model has high bias and low variance (underfits); a huge unconstrained model has low bias and high variance (overfits). Total expected error roughly decomposes as bias-squared plus variance plus irreducible noise. Practitioners detect the problem by comparing training-set accuracy against a held-out validation set, watching where the two curves diverge.
Strategic Impact
Clearer decisions
It helps you separate clear technical claims from marketing language.
Cost and budget
You can ask better implementation questions before spending money or time.
Team and workflow
Teams with shared understanding make better product, policy, and learning decisions.
The Future of Overfitting and Underfitting
These concepts remain foundational, but very large neural networks have complicated the classic picture. Modern models can have far more parameters than data points yet still generalize well, a surprising regime sometimes called 'double descent' where test error drops again after the overfitting peak. Research increasingly focuses on why over-parameterized models generalize, the role of implicit regularization in optimizers, and better automated detection of distribution shift. Expect richer diagnostics that flag overfitting in production as real-world data drifts away from training data.
Real-World Implementation
A spam filter that flags every email containing a specific sender's name because that sender happened to spam heavily in training data, missing new spammers entirely (overfitting).
A house-price model using only square footage and ignoring location, bedrooms, and condition, so it misses badly in expensive neighborhoods (underfitting).
A medical image classifier that learns to detect a hospital's scanner watermark instead of the disease, and fails at other hospitals (overfitting to a spurious feature).
Plotting training loss versus validation loss during training and stopping when validation loss starts rising while training loss keeps falling (catching overfitting early).
Risks & Guardrails
Different teams may use the same term differently, so define scope early.
Benchmarks can look strong while real-world performance is uneven.
Ignoring data quality and evaluation plans often creates fragile outcomes.
Implementation Roadmap
Start with a plain-language definition of the outcome you need.
Pick one success metric and one failure condition before testing.
Run a small pilot with representative data, not a polished demo set.
Document where Overfitting and Underfitting helps and where simpler methods are better.
Keep Exploring
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Overfitting and Underfitting quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
Next guide
Backpropagation
Frequently asked questions
What is Overfitting and Underfitting?
Overfitting is when a model memorizes its training data and fails on new examples; underfitting is when it is too simple to capture the real pattern. Hitting the sweet spot between them is the central challenge of machine learning.
A model scores 98% on its training data but only 71% on a held-out test set. What is the most likely problem?
A large gap where training accuracy is high but test accuracy is much lower is the classic signature of overfitting: the model fit noise in the training data rather than the general pattern.
Which scenario best describes underfitting?
An underfit model is too simple to capture the underlying pattern, so it performs poorly everywhere, including on the data it was trained on.
In the bias-variance tradeoff, high variance is most associated with which outcome?
High variance means the model changes a lot depending on the exact training data it sees, which leads to overfitting. High bias is the opposite, oversimplified end.
A loan-default model uses only an applicant's age and ignores income, debt, and credit history. It performs poorly on both training and new data. What should you do?
Poor performance everywhere signals underfitting. The fix is to increase model capacity, often by adding informative features, so it can represent the true relationship.
Why is a held-out validation or test set essential for detecting overfitting?
Overfitting is invisible if you only look at training accuracy. Evaluating on unseen data exposes the gap between memorization and genuine generalization.