Nested Cross-Validation Quiz

Answers and explanations appear as you take the quiz.

1. Within one outer split, which data may the inner hyperparameter search use?
   • All examples, including the outer test fold
   • Only the outer training data
   • Only the outer test labels
   • The full dataset after fitting preprocessing globally
2. Why can reporting the best inner validation score overstate model performance?
   • Inner cross-validation evaluates every candidate on its own training rows
   • Averaging across folds removes all uncertainty from candidate rankings
   • The best inner score already includes an independent outer evaluation
   • Choosing the best setting also selects favorable variation in validation scores
3. Five outer folds, three inner folds and ten candidate settings require how many inner candidate fits in the guide's example?
   • 150
   • 18
   • 50
   • 30
4. A scaler is fitted on the full dataset before nested cross-validation begins. Which problem remains?
   • Using more outer folds automatically removes preprocessing leakage
   • Unsupervised preprocessing cannot use information relevant to evaluation
   • Preprocessing has already accessed held-out information
   • A scaler can safely see test inputs whenever it does not see test labels
5. Different outer folds select different hyperparameters. How should this be interpreted?
   • It proves the cross-validation code is broken
   • It can occur because each selection step sees different training data
   • It means only the highest-scoring outer model is valid
   • It requires replacing outer evaluation with training accuracy
6. A dataset contains many rows per customer, and deployment targets unseen customers. Which design addresses that dependence?
   • Shuffle all rows separately in the outer loop only
   • Group customers only after evaluation is complete
   • Fit the feature selector globally before splitting
   • Keep customer groups separated in both inner and outer splits
7. After evaluating a fixed nested procedure, how does the guide recommend obtaining a final model?
   • Run the selection procedure on development data and fit the chosen configuration
   • Deploy the outer-fold model with the highest observed test score
   • Choose the configuration from the single best inner-fold result
   • Choose final settings by comparing candidates on the reserved test set
8. Why is the spread of outer-fold scores not automatically a confidence interval?
   • Each outer score is an independent measurement from an unrelated training set
   • The raw standard deviation of fold scores is itself a confidence interval
   • The fitted models share much of their training data
   • Averaging fold scores removes all sampling uncertainty