Guide-linked quiz · Medium Level
CUPED Variance Reduction Quiz
Apply CUPED's pre-period adjustment and identify how covariate timing, predictive strength and randomization affect valid inference.
Question 1 of 8
Which variable is appropriate for a standard CUPED adjustment?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- Which variable is appropriate for a standard CUPED adjustment?
- A pre-treatment measurement correlated with the experiment outcome
- A post-treatment variable changed by assignment
- A future outcome observed after the test
- A control-group label chosen after results
- What does theta=Cov(X,Y)/Var(X) represent in a common CUPED adjustment?
- The treatment assignment probability
- The linear coefficient used to remove predictable outcome variation
- The fraction of users excluded
- The model's accuracy
- What happens when the pre-period covariate has little correlation with the outcome?
- Power is guaranteed to increase fourfold
- The treatment effect becomes zero
- CUPED may provide little variance reduction
- Random assignment is invalid
- Why avoid using a treatment-affected covariate?
- It prevents calculating a mean
- It always increases sample size
- It makes variance exactly zero
- Conditioning on it can introduce post-treatment bias
- What does centering X by its mean do in Y_adj=Y-theta(X-mean X)?
- Changes the random assignment
- Subtracts predictive deviations while retaining the outcome's mean scale
- Converts Y into a probability
- Removes all sampling uncertainty
- What core design feature does CUPED not replace?
- A numerical outcome
- A pre-period covariate
- Valid random assignment
- An experiment window
- Why check baseline covariate coverage across groups?
- It removes the need for confidence intervals
- It determines the model's layer count
- It guarantees a higher treatment effect
- Differential missingness can affect the adjusted analysis
- How should CUPED results commonly be communicated?
- Alongside the unadjusted estimate and the prespecified adjustment method
- As proof every user benefited
- Without describing the covariate
- As a replacement for the outcome definition
Keep testing yourself
More quizzes picked for your level.