Permutation Feature Importance Quiz

Answers and explanations appear as you take the quiz.

1. A higher-is-better score falls from 0.82 to 0.70 after one input is shuffled. Which importance value follows?
   • 0.12
   • 0.70
   • 0.82
   • 1.52
2. During a basic permutation-importance test, which part of the dataset is deliberately rearranged?
   • All feature columns and outcomes together
   • The fitted model's parameter values
   • One feature's values across evaluation rows
   • Only the outcome labels while all inputs stay fixed
3. Two nearly identical sensor columns each receive low individual permutation importance. Which explanation is consistent with the guide?
   • Neither sensor can contain useful information
   • All low-importance features must be deleted immediately
   • The metric has necessarily been calculated backward
   • The model can use one sensor when the other is shuffled
4. Why should an analyst repeat the shuffle for each input?
   • To retrain the model on every permutation
   • To examine variation caused by different random rearrangements
   • To ensure importance values sum to one
   • To force every input to have positive importance
5. How does permutation importance differ from fitting a new model after removing a feature?
   • Permutation importance keeps the existing fitted model fixed
   • Permutation importance always changes the outcome labels
   • Feature removal cannot change predictive performance
   • Feature removal and shuffling are mathematically identical
6. An input has negative permutation importance across some shuffles. What happened in those measurements?
   • The baseline score was necessarily negative
   • The feature was uncorrelated with every other input
   • The shuffled score was higher than the baseline score
   • The absolute magnitude of the importance was calculated instead of its sign
7. A feature ranks highly in a permutation report. Which claim would exceed what the test establishes?
   • The fitted model's score dropped when the feature was shuffled
   • The result depends on the chosen metric
   • The ranking should be interpreted with the evaluation data
   • Changing that feature in the real world must cause the outcome to change
8. Before treating a permutation ranking as useful evidence, which check does the guide recommend?
   • Confirm that training-set accuracy alone is high
   • Confirm that the fitted model predicts usefully on evaluation data
   • Confirm that every importance is greater than zero
   • Confirm that individual importances sum to one