quiz bu lëkkale ak tegtal · Diggante Niveau

Quiz buy jàngat model yi ci net bi ak yi nekkul ci net bi

Wuutale performance done yu njëkk yi ak njeexte yi ci saasi te ràññee xalaat yi ak kaaraange yi ci replay offline ak ci jàngat yi ci net bi.

Laaj 1 de 8

What does offline evaluation directly measure?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. What does offline evaluation directly measure?
    • Candidate metrics on a specified historical or held-out dataset
    • Causal impact for every future user
    • The complete product effect in production
    • A guarantee of fairness
  2. Why can historical recommender logs bias offline evaluation?
    • Every candidate was shown equally
    • Prior exposure determined which items could receive interactions
    • Click labels are always complete
    • Offline scores include live randomization
  3. What can a properly randomized online experiment estimate?
    • A model's exact causal mechanism
    • Universal performance for every population
    • The average effect of assignment for the eligible tested population under design assumptions
    • The offline test-set loss
  4. Why run offline checks before online exposure?
    • They ensure historical labels are unbiased
    • They replace all live testing
    • They guarantee no guardrail regression
    • They can catch broken or clearly poor candidates with lower user risk
  5. Which rollout pattern limits exposure while measuring a candidate live?
    • Guaranteeing randomized assignment
    • Limiting exposure while monitoring a candidate's live behavior
    • Measuring only historical accuracy
    • Removing the need for rollback
  6. Why predeclare primary and guardrail metrics?
    • It makes every metric statistically independent
    • It guarantees enough sample size
    • It reduces post-hoc selection and clarifies success and harm criteria
    • It removes the need for experiment logs
  7. What can offline-online disagreement reveal?
    • That the model is causal
    • That the online test must be false
    • That offline data are useless
    • Distribution, logging, metric or product-interaction mismatch
  8. Why does an online experiment not guarantee impact in every future context?
    • The result is tied to its tested population, period and conditions
    • Randomization is never useful
    • The experiment measures only latency
    • Online tests cannot record assignment