Quiz collegato alla guida · Medio Livello

Quiz sulla valutazione off-policy da dati registrati

Applica l'inverse propensity weighting e identifica sovrapposizione, varianza e requisiti di logging che limitano le stime di policy controfattuale.

Domanda 1 di 8

Cosa stima la valutazione off-policy?

Anteprima di ogni domanda8 domande

Le risposte e le spiegazioni vengono visualizzate mentre rispondi al quiz.

  1. Cosa stima la valutazione off-policy?
    • A target policy's outcomes using data generated by another policy
    • The training accuracy of the logging model
    • A model's causal feature effects
    • The total number of logged requests
  2. Perché l'IPS divide per la propensione di registrazione?
    • To make rewards nonnegative
    • To reweight observed actions according to how likely the old policy was to choose them
    • To eliminate the need for outcomes
    • To estimate the context distribution only
  3. Cosa succede quando la politica di registrazione assegna una probabilità molto piccola a un'azione scelta?
    • The candidate policy is unbiased automatically
    • The action outcome becomes known with certainty
    • The inverse weight can become large and increase variance
    • The reward is set to zero
  4. Cosa significa la mancanza di supporto per un'azione target?
    • Every reward is missing
    • The action is always optimal
    • The target policy equals the logger
    • The logs contain little or no evidence for that action in the context
  5. Perché riportare la dimensione campionaria effettiva con stime ponderate?
    • It changes the propensity values
    • It reveals whether a few large weights dominate the information
    • It measures the number of policy actions possible
    • It proves the reward model is correct
  6. What tradeoff can clipping IPS weights create?
    • Guaranteed unbiasedness
    • Higher sample size with no assumptions
    • Lower variance but potential bias
    • No effect on the estimate
  7. What must be logged to support ordinary IPS evaluation?
    • The candidate policy's future decisions
    • Only model predictions
    • Only final reward totals
    • Action probabilities and outcomes with relevant context
  8. When might a controlled experiment be needed instead of relying on OPE?
    • When the candidate policy acts outside historical support
    • Whenever a metric is positive
    • When propensities are accurately logged
    • When the target matches the logging policy