Lift and Gain Charts

Start with a held-out set containing a defined positive outcome and a model score for each case. Sort cases from highest to lowest predicted score. A cumulative gains curve asks, after acting on the first x percent of the ranked population, what percentage of all actual positives has been found? If ranking is no better than random selection, its expected gain at x percent is x percent, the diagonal reference line. A useful ranking may rise above that line early. Consider an invented test set of 1,000 people with 100 actual positives. The top 200 scored people contain 50 positives. At 20% of the population, cumulative gain is 50/100 = 50%. Random selection would be expected to find about 20% of the positives in 20% of the population. Lift at that capacity is 50%/20% = 2.5. The positive rate within the selected group is 50/200 = 25%, compared with a 10% overall base rate. Microsoft's Azure Machine Learning documentation defines gain as the captured positive share at a chosen population fraction and lift as gain divided by the random baseline at that fraction. Lift varies with the fraction selected; do not quote one lift number without its cutoff. A model may be useful for a team able to review 10% of cases but add little value when nearly everyone must be contacted. Compare models at the capacity the application actually has. The charts evaluate ranking against observed labels, not probability calibration or the value of an intervention. A person might have signed up without a message, so a high gain among selected people does not prove that outreach caused their response. Costs, benefits, privacy and burden matter. Use time-relevant held-out data, keep the outcome definition consistent, and inspect performance across relevant groups. A selected fraction with impressive lift can still be too small to find most positives or too costly to act on.