Quiz collegato alla guida · Medio Livello

What AI Confidence Scores Actually Mean Quiz

Explains calibration, population-level meaning, threshold choices, subgroup checks, and why a model's confidence score is not a guarantee.

Percorsi guida correlatiWhat AI Confidence Scores Mean
Domanda 1 di 8

Un classificatore binario calibrato assegna una fiducia di 0,8 a molti casi. Cosa dovrebbe accadere in quel gruppo nel tempo?

Anteprima di ogni domanda8 domande

Le risposte e le spiegazioni vengono visualizzate mentre rispondi al quiz.

  1. Un classificatore binario calibrato assegna una fiducia di 0,8 a molti casi. Cosa dovrebbe accadere in quel gruppo nel tempo?
    • About 80 percent should be positive, across a sufficiently large representative group
    • Every individual case must be correct
    • The model must abstain on every case
    • The score must equal accuracy on every subgroup
  2. Cosa dovresti chiedere prima di interpretare il badge '90% di fiducia' di un prodotto?
    • Whether the number uses a large font
    • Whether the score is a calibrated probability and what outcome it predicts
    • Whether the product has a dark theme
    • Whether the model was written in Python
  3. Perché i dati di calibrazione dovrebbero essere separati dai dati di addestramento del modello?
    • To make every output equal 0.5
    • To remove the need for labels
    • To reduce overly optimistic estimates from examples the model already fit
    • To guarantee no future distribution shift
  4. Un modello è calibrato complessivamente ma non per un gruppo linguistico. Cosa succede?
    • The overall score is sufficient for every user
    • The language group must be removed from evaluation
    • Calibration is irrelevant
    • The workflow should inspect subgroup outcomes and adjust use or calibration
  5. Cosa misura la discriminazione in questo contesto?
    • How well the model ranks cases with different outcomes
    • Whether a score is displayed as a percentage
    • Whether a user can change the threshold
    • La velocità di esecuzione del modello
  6. When choosing an automation threshold, what should matter?
    • The number of decimal places in the score
    • The costs of false accepts and false rejects in the actual workflow
    • The model's training language only
    • Whether the threshold is a round number
  7. Why are token probabilities not automatically a factuality score for an entire generated answer?
    • They cannot be stored
    • They are always zero
    • They measure token likelihoods, not a validated probability that the response is correct
    • They only apply to images
  8. A low-confidence case could cause serious harm. Which workflow response is appropriate?
    • Hide the score and auto-approve it
    • Round the score upward
    • Assume calibration makes the case correct
    • Escalate or abstain according to a tested workflow