引導式測驗 · 中等 等級
直通式估算器測驗
測試硬前向值、代理梯度、偏差、量化使用、實現邏輯、部署評估和梯度裁剪。
問題 1 的 8
直通估計器如何處理困難操作?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- 直通估計器如何處理困難操作?
- Replace the forward result with a probability vector
- Stop all gradients before the operation
- Use the hard result forward and approximate the backward gradient with an identity or smooth proxy
- Compute an exact derivative for every discontinuity
- 为什么舍入可以阻止普通的梯度下降?
- Its derivative is zero almost everywhere and undefined at jumps
- Rounding always increases loss
- The input becomes categorical
- It removes the training labels
- 直通梯度估計的偏差是多少?
- It always equals the exact gradient
- It is unbiased for every hard function
- It is a random label rather than a gradient
- It uses a surrogate gradient that differs from the true derivative
- 哪個範例可以使用 STE?
- A continuous linear layer only
- A quantization step that rounds activations during the forward pass
- A dataset split with no model computation
- A metric computed after training
- `x + stop_gradient(hard(x) - x)`在概念上實現了什麼?
- A hard forward value with gradients that behave like the identity on x
- A smooth forward value with no gradient
- A random category sampled from x
- A hard value with exact derivative of the rounding function
- Why might a clipped pass-through gradient be used?
- To make the forward value continuous
- To turn the estimator unbiased
- To limit gradient flow outside a chosen range for stability
- To replace the model’s activation function
- Does an STE compute the true derivative of a hard threshold?
- Yes, because thresholding is smooth
- No; it substitutes a practical gradient approximation
- Yes, if the model is quantized
- It computes no forward value
- What is a sensible evaluation after training with an STE?
- Evaluate only the surrogate gradient
- Assume the estimator proves accuracy
- Remove the hard operation at inference without testing
- Measure task performance using the actual hard quantized or thresholded operation