Quiz collegato alla guida · Medio Livello
What AI Confidence Scores Actually Mean Quiz
Explains calibration, population-level meaning, threshold choices, subgroup checks, and why a model's confidence score is not a guarantee.
Domanda 1 di 8
Un classificatore binario calibrato assegna una fiducia di 0,8 a molti casi. Cosa dovrebbe accadere in quel gruppo nel tempo?
Anteprima di ogni domanda8 domande
Le risposte e le spiegazioni vengono visualizzate mentre rispondi al quiz.
- Un classificatore binario calibrato assegna una fiducia di 0,8 a molti casi. Cosa dovrebbe accadere in quel gruppo nel tempo?
- About 80 percent should be positive, across a sufficiently large representative group
- Every individual case must be correct
- The model must abstain on every case
- The score must equal accuracy on every subgroup
- Cosa dovresti chiedere prima di interpretare il badge '90% di fiducia' di un prodotto?
- Whether the number uses a large font
- Whether the score is a calibrated probability and what outcome it predicts
- Whether the product has a dark theme
- Whether the model was written in Python
- Perché i dati di calibrazione dovrebbero essere separati dai dati di addestramento del modello?
- To make every output equal 0.5
- To remove the need for labels
- To reduce overly optimistic estimates from examples the model already fit
- To guarantee no future distribution shift
- Un modello è calibrato complessivamente ma non per un gruppo linguistico. Cosa succede?
- The overall score is sufficient for every user
- The language group must be removed from evaluation
- Calibration is irrelevant
- The workflow should inspect subgroup outcomes and adjust use or calibration
- Cosa misura la discriminazione in questo contesto?
- How well the model ranks cases with different outcomes
- Whether a score is displayed as a percentage
- Whether a user can change the threshold
- La velocità di esecuzione del modello
- When choosing an automation threshold, what should matter?
- The number of decimal places in the score
- The costs of false accepts and false rejects in the actual workflow
- The model's training language only
- Whether the threshold is a round number
- Why are token probabilities not automatically a factuality score for an entire generated answer?
- They cannot be stored
- They are always zero
- They measure token likelihoods, not a validated probability that the response is correct
- They only apply to images
- A low-confidence case could cause serious harm. Which workflow response is appropriate?
- Hide the score and auto-approve it
- Round the score upward
- Assume calibration makes the case correct
- Escalate or abstain according to a tested workflow