AI Glossary Term

What does “Evaluation Set” mean?

Definition

A held-out dataset used to measure model quality after training.

Browse the full AI Glossary Explore all free AI guides

How Evaluation Set works

Example

A team sets aside 10% of its labeled data before training and never lets the model learn from it. After training is finished, they measure accuracy on that held-out set to get an honest estimate of performance on new data.

Why it matters

Testing on data the model has already seen overstates how good it is. A clean evaluation set tells you whether it will work in the real world.

Common questions

Is an evaluation set the same as a test set?

Usually yes. Some teams use the terms differently, with evaluation meaning any held-out data, including validation.

How big should it be?

Large enough that scores are stable and cover the cases you care about. Rare cases may need deliberate extra coverage.

What ruins an evaluation set?

Leakage: if examples from it appear in training, scores become inflated.

BenchmarkA standardized test or dataset used to measure and compare model performance.DatasetA collection of structured or unstructured examples used for training, validation, or testing.Validation SetA dataset used during development to tune models and prevent overfitting.GeneralizationHow well a model performs on new, unseen data outside the training set.HyperparameterA configuration value set before training, such as learning rate, batch size, or depth.Synthetic DataArtificially generated data used to augment, simulate, or protect sensitive training data.

Learn more in our free guides

AI Evaluation Basics LLM Evaluations Linear Probing and Frozen Feature Evaluation Mean Opinion Score Evaluation

Test yourself

AI Evaluation Basics Quiz LLM Evaluations Quiz Linear Probing and Frozen Feature Evaluation Quiz