가이드 연계 퀴즈 · 중간 레벨

간단한 견적 퀴즈

하드 포워드 값, 서로게이트 그래디언트, 바이어스, 양자화 사용, 구현 로직, 배포된 평가 및 그래디언트 클리핑을 테스트합니다.

관련 가이드 경로직접 추정기
질문 1 의 8

How does the straight-through estimator handle a hard operation?

모든 질문 미리보기8 질문

퀴즈를 풀면 정답과 설명이 나타납니다.

  1. How does the straight-through estimator handle a hard operation?
    • Replace the forward result with a probability vector
    • Stop all gradients before the operation
    • Use the hard result forward and approximate the backward gradient with an identity or smooth proxy
    • Compute an exact derivative for every discontinuity
  2. Why can rounding block ordinary gradient descent?
    • Its derivative is zero almost everywhere and undefined at jumps
    • Rounding always increases loss
    • The input becomes categorical
    • It removes the training labels
  3. What is the bias of a straight-through gradient estimate?
    • It always equals the exact gradient
    • It is unbiased for every hard function
    • It is a random label rather than a gradient
    • It uses a surrogate gradient that differs from the true derivative
  4. Which example can use an STE?
    • A continuous linear layer only
    • A quantization step that rounds activations during the forward pass
    • A dataset split with no model computation
    • A metric computed after training
  5. What does `x + stop_gradient(hard(x) - x)` achieve conceptually?
    • A hard forward value with gradients that behave like the identity on x
    • A smooth forward value with no gradient
    • A random category sampled from x
    • A hard value with exact derivative of the rounding function
  6. Why might a clipped pass-through gradient be used?
    • To make the forward value continuous
    • To turn the estimator unbiased
    • To limit gradient flow outside a chosen range for stability
    • To replace the model’s activation function
  7. Does an STE compute the true derivative of a hard threshold?
    • Yes, because thresholding is smooth
    • No; it substitutes a practical gradient approximation
    • Yes, if the model is quantized
    • It computes no forward value
  8. What is a sensible evaluation after training with an STE?
    • Evaluate only the surrogate gradient
    • Assume the estimator proves accuracy
    • Remove the hard operation at inference without testing
    • Measure task performance using the actual hard quantized or thresholded operation