基本ガイド
Train, Validation, and Test Splits
Training, validation and test partitions serve different roles in model development: fitting parameters, choosing a design, and estimating performance after those choices.
このページでは3 分で読めます
概要
Keeping the final test data out of tuning helps prevent an optimistic result. The right split also respects time, shared people or devices, class balance, and the actual deployment population.
ディープダイブ
A training set is used to fit model parameters from examples. A validation set informs choices made during development, such as feature selection, hyperparameters, architecture or a decision threshold. A test set is reserved to estimate performance after those choices have been made. Test results should not become a repeated tuning signal: after many adjustments made in response to the test score, that set is no longer untouched. There is no universally correct 80/10/10 or 60/20/20 ratio. The amount of data, rarity of outcomes, expected deployment shift and evaluation precision all matter. State the partition method and counts. For a rare classification target, stratification can keep class proportions reasonably represented across splits, but it does not by itself stop the same person's records from appearing in training and test. Group-aware splitting keeps related observations, such as visits from one patient or images from one device, together. For forecasting and other time-dependent tasks, randomly mixing future observations into training can make evaluation unrealistically easy. Use earlier data to predict later periods and consider a gap if labels or features overlap across the boundary. A split should also resemble the intended deployment setting: a model trained on one city and tested on the same city's nearby dates may not tell us how it will perform in a different region. Split before fitting preprocessing. Scalers, imputers, feature selectors and encoders must learn from training data in each development fold, then transform held-out data using those fitted settings. Scikit-learn's common-pitfalls guide calls out the error of computing a mean or selecting features on the full dataset before splitting. Pipelines can help keep these operations inside the correct fold. Preserve the random seed or time cutoff and document any exclusions. Report test uncertainty and subgroup results where important, and obtain a fresh independent evaluation set if the final test has already guided repeated revisions.
戦略的影響
より明確な判決
これは、明確な技術的主張とマーケティング言語を区別するのに役立ちます。
費用と予算
お金や時間を費やす前に、実装に関するより良い質問をすることができます。
チームとワークフロー
共通の理解を持ったチームは、製品、ポリシー、学習に関する意思決定をより適切に行うことができます。
The Future of Train, Validation, and Test Splits
As datasets grow more connected, split design will become more important than a familiar percentage rule. Evaluations increasingly need to keep households, patients, products or geographic clusters together and test later time windows. Automated pipelines can enforce boundaries, but teams must still decide which relationships and deployment shifts matter. Future benchmarks should publish partition definitions and leakage checks along with scores. A clean split cannot guarantee performance after a major change in population or policy; it only makes the stated test more credible. Ongoing monitoring and new holdouts will remain necessary when models are retrained or repurposed.
現実世界の実装
A classifier is fitted on training rows, its threshold is chosen with validation rows, and the final score is reported once on untouched test rows.
A hospital keeps all visits from one patient in the same partition so repeated visits cannot leak identity-related signals.
A demand forecaster trains on earlier months and tests on later months rather than shuffling all dates.
A team stratifies a rare-label sample while checking separately that customers and time periods do not cross partitions.
リスクとガードレール
チームが異なれば、同じ用語の使用方法も異なる可能性があるため、範囲を早めに定義してください。
ベンチマークは好調に見えても、実際のパフォーマンスにはばらつきがある場合があります。
データの品質と評価計画を無視すると、多くの場合、脆弱な結果が生じます。
実装ロードマップ
必要な結果を平易な言葉で定義することから始めます。
テストする前に、成功指標と失敗条件を 1 つ選択します。
洗練されたデモセットではなく、代表的なデータを使用して小規模なパイロットを実行します。
Document where Train, Validation, and Test Splits helps and where simpler methods are better.
探検を続けましょう
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Train, Validation, and Test Splits quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
よくある質問
What is Train, Validation, and Test Splits?
Training, validation and test partitions serve different roles in model development: fitting parameters, choosing a design, and estimating performance after those choices. Keeping the final test data out of tuning helps prevent an optimistic result. The right split also respects time, shared people or devices, class balance, and the actual deployment population.
Which partition should normally supply examples for fitting a model's learned parameters?
Training data are used to fit parameters; validation and test data have separate selection and evaluation roles.
A team compares two architectures and chooses a probability threshold. Which partition can guide those development choices?
Validation results can guide architecture, hyperparameter and threshold choices while preserving the final test for later estimation.
Why should a final test set be used sparingly after model choices are fixed?
Repeated tuning after seeing the final score compromises the claim that the set remained independent of model selection.
A dataset contains several visits per patient. Which split rule prevents patient overlap between train and test?
Group-aware splitting keeps related observations together so one patient's information does not appear on both sides.
What does class stratification help preserve during a classification split?
Stratification aims to keep class mixes represented; it does not replace group or time-aware splitting.
学び続ける
関連ガイド
このトピックのために選ばれたその他のガイド