Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Moja kwa Moja ya Kukadiria

Hujaribu thamani za kusonga mbele kwa bidii, viwango vya juu vya ziada, upendeleo, matumizi ya ukadiriaji, mantiki ya utekelezaji, tathmini iliyotumwa na upunguzaji wa upinde rangi.

Njia za mwongozo zinazohusianaMoja kwa Moja Kupitia Mkadiriaji
Swali 1 ya 8

How does the straight-through estimator handle a hard operation?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. How does the straight-through estimator handle a hard operation?
    • Replace the forward result with a probability vector
    • Stop all gradients before the operation
    • Use the hard result forward and approximate the backward gradient with an identity or smooth proxy
    • Compute an exact derivative for every discontinuity
  2. Why can rounding block ordinary gradient descent?
    • Its derivative is zero almost everywhere and undefined at jumps
    • Rounding always increases loss
    • The input becomes categorical
    • It removes the training labels
  3. What is the bias of a straight-through gradient estimate?
    • It always equals the exact gradient
    • It is unbiased for every hard function
    • It is a random label rather than a gradient
    • It uses a surrogate gradient that differs from the true derivative
  4. Which example can use an STE?
    • A continuous linear layer only
    • A quantization step that rounds activations during the forward pass
    • A dataset split with no model computation
    • A metric computed after training
  5. What does `x + stop_gradient(hard(x) - x)` achieve conceptually?
    • A hard forward value with gradients that behave like the identity on x
    • A smooth forward value with no gradient
    • A random category sampled from x
    • A hard value with exact derivative of the rounding function
  6. Why might a clipped pass-through gradient be used?
    • To make the forward value continuous
    • To turn the estimator unbiased
    • To limit gradient flow outside a chosen range for stability
    • To replace the model’s activation function
  7. Does an STE compute the true derivative of a hard threshold?
    • Yes, because thresholding is smooth
    • No; it substitutes a practical gradient approximation
    • Yes, if the model is quantized
    • It computes no forward value
  8. What is a sensible evaluation after training with an STE?
    • Evaluate only the surrogate gradient
    • Assume the estimator proves accuracy
    • Remove the hard operation at inference without testing
    • Measure task performance using the actual hard quantized or thresholded operation