Base Rates and False Positives in AI Detection

A base rate is how common a condition is before a test is applied. Sensitivity is the share of people with the condition whom the test flags. The false-positive rate is the share of people without the condition whom it incorrectly flags. These quantities answer different questions. A detector may have high sensitivity and a low false-positive rate while still producing many false alarms if the condition is uncommon. Consider an illustrative population of 1,000 items where 1 percent truly meet a criterion. If a detector has 90 percent sensitivity, it flags about 9 of the 10 true cases. If its false-positive rate is 5 percent, it also flags about 50 of the 990 cases that do not meet the criterion. There would be roughly 59 flags, of which only 9 are true positives. The exact numbers are hypothetical, but the arithmetic shows why the proportion of correct flags depends on prevalence as well as test performance. This is why asking only whether a detector is 'accurate' is not enough. Request the confusion matrix or the sensitivity and specificity at the threshold used, the evaluation population, and the prevalence assumed. Then estimate the positive predictive value for the real population. If prevalence differs across settings, the same detector score can imply different probabilities that a flagged case is actually positive. For consequential decisions, a detector flag should be treated as a prompt for additional evidence. A human reviewer can examine context and apply the relevant standard, but reviewers also need clear procedures and should not treat the algorithm's output as an authoritative verdict. Provide an appeal path and document how evidence was considered. In education, for example, a detector score alone cannot establish authorship; other evidence and a fair process matter. Measure false positives across relevant groups and conditions.