Guide-linked quiz · Medium Level
Train, Validation, and Test Splits Quiz
Tests fitting versus tuning versus final evaluation, preprocessing leakage, group and time splits, stratification, ratios, and reused test sets.
Question 1 of 8
Which partition should normally supply examples for fitting a model's learned parameters?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- Which partition should normally supply examples for fitting a model's learned parameters?
- The final untouched test partition
- The training partition
- Only the deployment population after launch
- All partitions combined before model selection
- A team compares two architectures and chooses a probability threshold. Which partition can guide those development choices?
- The final test set, repeated until the score looks best
- A production set labeled only after release
- An unrelated benchmark with different labels and population
- A validation set or cross-validation within development data
- Why should a final test set be used sparingly after model choices are fixed?
- Repeated test-driven revisions turn it into development feedback
- Each test run erases the labels
- The test set is needed to fit model weights
- A final test cannot measure any subgroup
- A dataset contains several visits per patient. Which split rule prevents patient overlap between train and test?
- Shuffle each visit independently and stratify labels
- Put the latest visit of every patient into test and earlier visits into train
- Assign every visit from one patient to the same partition
- Sort records by file name before splitting
- What does class stratification help preserve during a classification split?
- Identical timestamps in each partition
- The same fitted preprocessing statistics in every split
- Complete separation of repeated users by itself
- Approximate label proportions across partitions
- Why should a scaler or feature selector be fitted after the split, using training data only?
- Validation and test rows must never be transformed
- Fitting on the full dataset can leak held-out information into development
- Training rows have no feature distribution before splitting
- A scaler always changes the target labels
- For a demand forecast, which test design most closely reflects future use?
- Shuffle all dates so future demand appears in training
- Use only the first week for both training and testing
- Fit on earlier periods and evaluate on later periods
- Split by the highest sales value regardless of date
- Why does the guide avoid prescribing one fixed train/validation/test percentage for every project?
- Data volume, rare outcomes, dependence and deployment shift affect the design
- A fixed ratio prevents the need for confidence intervals
- Any ratio is equivalent if the model is large enough
- Every project should instead use all rows for both training and testing
Keep testing yourself
More quizzes picked for your level.