গাইড লিঙ্ক ক্যুইজ · মাঝারি স্তর
স্ট্রেইট-থ্রু এস্টিমেটর কুইজ
হার্ড ফরওয়ার্ড মান, সারোগেট গ্রেডিয়েন্ট, পক্ষপাত, পরিমাপকরণ ব্যবহার, বাস্তবায়ন যুক্তি, স্থাপন করা মূল্যায়ন, এবং গ্রেডিয়েন্ট ক্লিপিং পরীক্ষা করে।
প্রশ্ন 1 এর 8
How does the straight-through estimator handle a hard operation?
প্রতিটি প্রশ্নের পূর্বরূপ দেখুন8 প্রশ্ন
আপনি কুইজ নেওয়ার সাথে সাথে উত্তর এবং ব্যাখ্যাগুলি উপস্থিত হয়।
- How does the straight-through estimator handle a hard operation?
- Replace the forward result with a probability vector
- Stop all gradients before the operation
- Use the hard result forward and approximate the backward gradient with an identity or smooth proxy
- Compute an exact derivative for every discontinuity
- Why can rounding block ordinary gradient descent?
- Its derivative is zero almost everywhere and undefined at jumps
- Rounding always increases loss
- The input becomes categorical
- It removes the training labels
- What is the bias of a straight-through gradient estimate?
- It always equals the exact gradient
- It is unbiased for every hard function
- It is a random label rather than a gradient
- It uses a surrogate gradient that differs from the true derivative
- Which example can use an STE?
- A continuous linear layer only
- A quantization step that rounds activations during the forward pass
- A dataset split with no model computation
- A metric computed after training
- What does `x + stop_gradient(hard(x) - x)` achieve conceptually?
- A hard forward value with gradients that behave like the identity on x
- A smooth forward value with no gradient
- A random category sampled from x
- A hard value with exact derivative of the rounding function
- Why might a clipped pass-through gradient be used?
- To make the forward value continuous
- To turn the estimator unbiased
- To limit gradient flow outside a chosen range for stability
- To replace the model’s activation function
- Does an STE compute the true derivative of a hard threshold?
- Yes, because thresholding is smooth
- No; it substitutes a practical gradient approximation
- Yes, if the model is quantized
- It computes no forward value
- What is a sensible evaluation after training with an STE?
- Evaluate only the surrogate gradient
- Assume the estimator proves accuracy
- Remove the hard operation at inference without testing
- Measure task performance using the actual hard quantized or thresholded operation