Permutation Feature Importance

Permutation importance asks a practical question: how much does this fitted model's score change when information in one input is disrupted? First measure the model on an evaluation dataset. Shuffle one feature's values across rows, leaving the other features and outcome labels unchanged, then score the same fitted model again. Restore the data and repeat for other features.

With a higher-is-better metric, importance is the original score minus the shuffled score. A hypothetical drop from 0.82 to 0.70 therefore gives an importance of 0.12. The units follow the chosen metric. This is not automatically a percentage contribution to the prediction, and importance values do not need to sum to one.

Repeat shuffles because different random rearrangements can produce different results. Report the average and variation, along with the metric and evaluation dataset. A value near zero can mean the model makes little use of that feature under this test. It can also arise when another feature supplies similar information.

Correlated inputs are a major interpretation problem. A model may continue predicting well after one of two similar sensors is shuffled. Removing both sensors could have a much larger effect. Shuffling can also produce implausible input combinations, so domain knowledge matters when interpreting the experiment.

Assess predictive performance before interpreting feature rankings. A poorly performing model cannot reliably explain which inputs would matter to a better model. Scikit-learn provides permutation_importance for inspecting fitted estimators. Using held-out data focuses the analysis on the model's behavior beyond its training cases. The result describes this model, dataset and metric; it does not establish a causal relationship in the world.