Guide-linked quiz · Medium Level

Train, Validation, and Test Splits Quiz

Tests fitting versus tuning versus final evaluation, preprocessing leakage, group and time splits, stratification, ratios, and reused test sets.

Related guide pathsTrain Validation Test Split
Question 1 of 8

Which partition should normally supply examples for fitting a model's learned parameters?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Which partition should normally supply examples for fitting a model's learned parameters?
    • The final untouched test partition
    • The training partition
    • Only the deployment population after launch
    • All partitions combined before model selection
  2. A team compares two architectures and chooses a probability threshold. Which partition can guide those development choices?
    • The final test set, repeated until the score looks best
    • A production set labeled only after release
    • An unrelated benchmark with different labels and population
    • A validation set or cross-validation within development data
  3. Why should a final test set be used sparingly after model choices are fixed?
    • Repeated test-driven revisions turn it into development feedback
    • Each test run erases the labels
    • The test set is needed to fit model weights
    • A final test cannot measure any subgroup
  4. A dataset contains several visits per patient. Which split rule prevents patient overlap between train and test?
    • Shuffle each visit independently and stratify labels
    • Put the latest visit of every patient into test and earlier visits into train
    • Assign every visit from one patient to the same partition
    • Sort records by file name before splitting
  5. What does class stratification help preserve during a classification split?
    • Identical timestamps in each partition
    • The same fitted preprocessing statistics in every split
    • Complete separation of repeated users by itself
    • Approximate label proportions across partitions
  6. Why should a scaler or feature selector be fitted after the split, using training data only?
    • Validation and test rows must never be transformed
    • Fitting on the full dataset can leak held-out information into development
    • Training rows have no feature distribution before splitting
    • A scaler always changes the target labels
  7. For a demand forecast, which test design most closely reflects future use?
    • Shuffle all dates so future demand appears in training
    • Use only the first week for both training and testing
    • Fit on earlier periods and evaluate on later periods
    • Split by the highest sales value regardless of date
  8. Why does the guide avoid prescribing one fixed train/validation/test percentage for every project?
    • Data volume, rare outcomes, dependence and deployment shift affect the design
    • A fixed ratio prevents the need for confidence intervals
    • Any ratio is equivalent if the model is large enough
    • Every project should instead use all rows for both training and testing

Keep testing yourself

More quizzes picked for your level.

View all quizzes