Straight-Through Estimator Quiz

Answers and explanations appear as you take the quiz.

1. How does the straight-through estimator handle a hard operation?
   • Replace the forward result with a probability vector
   • Stop all gradients before the operation
   • Use the hard result forward and approximate the backward gradient with an identity or smooth proxy
   • Compute an exact derivative for every discontinuity
2. Why can rounding block ordinary gradient descent?
   • Its derivative is zero almost everywhere and undefined at jumps
   • Rounding always increases loss
   • The input becomes categorical
   • It removes the training labels
3. What is the bias of a straight-through gradient estimate?
   • It always equals the exact gradient
   • It is unbiased for every hard function
   • It is a random label rather than a gradient
   • It uses a surrogate gradient that differs from the true derivative
4. Which example can use an STE?
   • A continuous linear layer only
   • A quantization step that rounds activations during the forward pass
   • A dataset split with no model computation
   • A metric computed after training
5. What does `x + stop_gradient(hard(x) - x)` achieve conceptually?
   • A hard forward value with gradients that behave like the identity on x
   • A smooth forward value with no gradient
   • A random category sampled from x
   • A hard value with exact derivative of the rounding function
6. Why might a clipped pass-through gradient be used?
   • To make the forward value continuous
   • To turn the estimator unbiased
   • To limit gradient flow outside a chosen range for stability
   • To replace the model’s activation function
7. Does an STE compute the true derivative of a hard threshold?
   • Yes, because thresholding is smooth
   • No; it substitutes a practical gradient approximation
   • Yes, if the model is quantized
   • It computes no forward value
8. What is a sensible evaluation after training with an STE?
   • Evaluate only the surrogate gradient
   • Assume the estimator proves accuracy
   • Remove the hard operation at inference without testing
   • Measure task performance using the actual hard quantized or thresholded operation