Conformal Prediction Quiz

Answers and explanations appear as you take the quiz.

1. What does a conformal prediction method output instead of a single label?
   • A prediction set or interval with a coverage guarantee
   • A retrained model with higher accuracy
   • A single point estimate with no uncertainty
   • A loss value for the training set
2. Which assumption underpins the standard conformal coverage guarantee?
   • The model is a neural network
   • Data points are exchangeable
   • Features are independent of each other
   • The labels are perfectly balanced
3. What is the role of the calibration set?
   • To train the underlying model
   • To pick the learning rate
   • To compute nonconformity scores that set the threshold
   • To remove outliers from training data
4. If you set alpha = 0.1, what coverage are you targeting?
   • 10%
   • 50%
   • 99%
   • 90%
5. What typically happens to the prediction set when the model is very uncertain about an input?
   • The set grows larger, containing more candidate labels
   • The set becomes empty
   • The threshold is ignored
   • The model is automatically retrained
6. Why is conformal prediction called 'distribution-free'?
   • It only works on Gaussian data
   • It requires no parametric assumption about the data distribution for its coverage guarantee
   • It discards the data distribution entirely
   • It assumes a uniform distribution
7. A split-conformal classifier has 99 calibration examples and targets 90% marginal coverage. Which rank is used for the calibration quantile?
   • The 10th ordered score
   • The 99th score only
   • The 90th ordered calibration score (ceil((99+1) × 0.9))
   • The average of all labels
8. In classification, what can a nonconformity score such as 1 minus the candidate label probability represent?
   • The number of training epochs
   • The probability that the calibration set is exchangeable
   • The class frequency in the test set
   • How poorly the model supports that candidate label for the example