গাইড লিঙ্ক ক্যুইজ · মাঝারি স্তর
লগড ডেটা কুইজ থেকে অফ-পলিসি মূল্যায়ন
বিপরীত প্রবণতা ওজন প্রয়োগ করুন এবং ওভারল্যাপ, বৈচিত্র্য এবং লগিং প্রয়োজনীয়তা সনাক্ত করুন যা বিপরীত নীতি অনুমানকে সীমাবদ্ধ করে।
প্রশ্ন 1 এর 8
What does off-policy evaluation estimate?
প্রতিটি প্রশ্নের পূর্বরূপ দেখুন8 প্রশ্ন
আপনি কুইজ নেওয়ার সাথে সাথে উত্তর এবং ব্যাখ্যাগুলি উপস্থিত হয়।
- What does off-policy evaluation estimate?
- A target policy's outcomes using data generated by another policy
- The training accuracy of the logging model
- A model's causal feature effects
- The total number of logged requests
- Why does IPS divide by the logging propensity?
- To make rewards nonnegative
- To reweight observed actions according to how likely the old policy was to choose them
- To eliminate the need for outcomes
- To estimate the context distribution only
- What happens when the logging policy assigns a very small probability to a chosen action?
- The candidate policy is unbiased automatically
- The action outcome becomes known with certainty
- The inverse weight can become large and increase variance
- The reward is set to zero
- What does lack of support mean for a target action?
- Every reward is missing
- The action is always optimal
- The target policy equals the logger
- The logs contain little or no evidence for that action in the context
- Why report effective sample size with weighted estimates?
- It changes the propensity values
- It reveals whether a few large weights dominate the information
- It measures the number of policy actions possible
- It proves the reward model is correct
- What tradeoff can clipping IPS weights create?
- Guaranteed unbiasedness
- Higher sample size with no assumptions
- Lower variance but potential bias
- No effect on the estimate
- What must be logged to support ordinary IPS evaluation?
- The candidate policy's future decisions
- Only model predictions
- Only final reward totals
- Action probabilities and outcomes with relevant context
- When might a controlled experiment be needed instead of relying on OPE?
- When the candidate policy acts outside historical support
- Whenever a metric is positive
- When propensities are accurately logged
- When the target matches the logging policy