基礎知識指南

訓練集、驗證集與測試集劃分的最佳實踐

Train, validation, and test partitions serve different purposes: fitting, model selection, and final evaluation.

聆聽此指南AI 語音敘述 · ~3 min

本頁閱讀時間3分鐘
  1. 概述
  2. 深入探討
  3. 現實世界的實施
  4. The Future of Train, Validation and Test Split Best Practices
  5. 付諸實踐
  6. 不斷探索
  7. 常見問題

概述

Split design must match the deployment setting and prevent leakage across related people, groups, time periods, or near-duplicate records; a random split is not universally appropriate.

深入探討

A training set is used to fit model parameters. Validation data help choose hyperparameters, features, thresholds, or model variants. A final test set estimates performance after those choices are settled. Repeatedly tuning against the test set makes it part of model selection and can produce an optimistic estimate. The split strategy should reflect how predictions will be used. Random stratified splits can be useful when future examples are independent and come from a similar population. If the same person, household, device, or source appears in multiple rows, group-aware splitting may be needed to estimate performance on new groups. For future prediction, time-based splitting usually better reflects deployment than shuffling historical events. Scikit-learn documents GroupKFold and TimeSeriesSplit for such settings. All preprocessing that learns from data—such as scaling, imputation, feature selection, or vocabulary building—should be fitted using training data within each fold, then applied to held-out data. Otherwise information from validation or test can leak into training. Duplicate and near-duplicate examples across partitions can also inflate evaluation. Check split membership after deduplication and before augmentation or oversampling. There is no universal ratio such as 70/15/15. Choose sizes based on data volume, class balance, uncertainty, and the evaluation goal. Preserve a final holdout where feasible, report the split method and random seed, and consider confidence intervals or repeated cross-validation for development. A held-out test is still only an estimate for the population and time period it represents.

現實世界的實施

A hospital holds out entire hospitals when the goal is to assess transfer to an unseen hospital.

A forecasting model trains on earlier dates and evaluates on later dates using a time-series split.

A scaler is fitted only on the training fold and then applied to validation data.

Image crops or augmented versions stay with their original image in one partition.

The Future of Train, Validation and Test Split Best Practices

Evaluation practice is moving toward more explicit separation of development data from final, temporally and institutionally meaningful tests. Future benchmarks should document population, time, grouping, preprocessing, and overlap checks, not just a split percentage. As data distributions shift, external or prospective evaluation may be needed. Automated split tools can help implement a design, but they cannot choose the right deployment target without domain knowledge. Transparent split manifests can make these choices easier to audit and reproduce across model updates over time.

付諸實踐

為什麼這很重要

  • 更明確的決策. 它可以幫助您將清晰的技術聲明與行銷語言分開。
  • 成本與預算. 在花費金錢或時間之前,您可以提出更好的實施問題。
  • 團隊與工作流程. 具有共同理解的團隊可以做出更好的產品、政策和學習決策。

留意

  • 不同的團隊可能會以不同的方式使用相同術語,因此請儘早定義範圍。
  • 基準測試可能看起來很強大,但實際效能卻參差不齊。
  • 忽視數據品質和評估計劃通常會產生脆弱的結果。

後續步驟

  1. 從您需要的結果的簡單語言定義開始。
  2. 在測試之前選擇一種成功指標和一種失敗條件。
  3. 使用代表性資料運行小型試點,而不是完善的演示集。
  4. Document where Train, Validation and Test Split Best Practices helps and where simpler methods are better.

不斷探索

免費電子報

獲取每日 AI 簡報

每個工作日上午提供三則經過驗證的 AI 新聞,以淺顯易懂的英文撰寫。永久免費,無廣告。

每個工作日一封電子郵件。一鍵取消訂閱。我們絕不出售或分享您的地址。

測試一下自己

Take the Train, Validation and Test Split Best Practices quiz

每個答案皆有即時回饋,通過課程後可獲得具備可驗證 ID 的可分享證書。

開始測驗

支持免費的 AI 教育。AI Understanding 是一個 501(c)(3) 非營利組織——絕無廣告,絕無付費牆。 捐款

常見問題

What is Train, Validation and Test Split Best Practices?

Train, validation, and test partitions serve different purposes: fitting, model selection, and final evaluation. Split design must match the deployment setting and prevent leakage across related people, groups, time periods, or near-duplicate records; a random split is not universally appropriate.

What happens if a test set is repeatedly used to choose model variants?

Repeated decisions based on test results leak test information into selection.

When is a group-aware split appropriate?

Group splits keep related records together when evaluating unseen groups.

Why use a time-based split for a future forecasting task?

Time-based evaluation better mimics predicting future periods.

Where should a scaler be fitted during cross-validation?

Fitting preprocessing on all data can leak held-out information.

What does a held-out test score estimate?

A test result is an estimate tied to its sample and conditions.