引導式測驗 · 硬 等級
巢狀交叉驗證測驗
在實用的嵌套交叉驗證工作流程中檢查內折疊和外折疊角色、預處理洩漏和擬合計數計算。
問題 1 的 8
在一次外部切分中,內部超參數搜尋可以使用哪些資料?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- 在一次外部切分中,內部超參數搜尋可以使用哪些資料?
- All examples, including the outer test fold
- Only the outer training data
- Only the outer test labels
- The full dataset after fitting preprocessing globally
- 為什麼報告最佳內部驗證分數會高估模型效能?
- Inner cross-validation evaluates every candidate on its own training rows
- Averaging across folds removes all uncertainty from candidate rankings
- The best inner score already includes an independent outer evaluation
- Choosing the best setting also selects favorable variation in validation scores
- 五個外部折、三個內部折與十個候選設定,在指南範例中需要多少次內部候選擬合?
- 150
- 18
- 50
- 30
- 在開始嵌套交叉驗證前,先在完整資料集上擬合比例因子。還剩下什麼問題?
- Using more outer folds automatically removes preprocessing leakage
- Unsupervised preprocessing cannot use information relevant to evaluation
- Preprocessing has already accessed held-out information
- A scaler can safely see test inputs whenever it does not see test labels
- 不同的外層折疊會選擇不同的超參數。這應該如何詮釋?
- It proves the cross-validation code is broken
- It can occur because each selection step sees different training data
- It means only the highest-scoring outer model is valid
- It requires replacing outer evaluation with training accuracy
- A dataset contains many rows per customer, and deployment targets unseen customers. Which design addresses that dependence?
- Shuffle all rows separately in the outer loop only
- Group customers only after evaluation is complete
- Fit the feature selector globally before splitting
- Keep customer groups separated in both inner and outer splits
- After evaluating a fixed nested procedure, how does the guide recommend obtaining a final model?
- Run the selection procedure on development data and fit the chosen configuration
- Deploy the outer-fold model with the highest observed test score
- Choose the configuration from the single best inner-fold result
- Choose final settings by comparing candidates on the reserved test set
- Why is the spread of outer-fold scores not automatically a confidence interval?
- Each outer score is an independent measurement from an unrelated training set
- The raw standard deviation of fold scores is itself a confidence interval
- The fitted models share much of their training data
- Averaging fold scores removes all sampling uncertainty