Gumbel-Softmax 和重新参数化测验

当您参加测验时，就会出现答案和解释。

1. 为什么通过普通反向传播对类别样本进行训练很困难？
   • Categorical distributions have no probabilities
   • The discrete choice is a non-differentiable operation
   • Sampling removes the input tensor
   • All categories have identical logits
2. 重参数化技巧旨在实现什么？
   • Remove all randomness from training
   • Replace gradients with labels
   • Make categorical outputs continuous at evaluation time
   • Express randomness through an external noise variable while keeping a differentiable path
3. Gumbel-Softmax 如何近似类别选择？
   • Use a hard class label as a gradient
   • Normalize the input pixels
   • Add Gumbel noise to logits and apply a temperature-controlled softmax
   • Average logits across unrelated examples
4. 当温度变得很大时，分布会发生什么变化？
   • It becomes smoother across categories
   • It becomes an exact one-hot sample
   • The logits are discarded
   • It becomes independent of the Gumbel noise
5. 当温度趋近于零时会发生什么？
   • The output becomes uniform
   • The noise becomes a label
   • The operation stops depending on logits
   • The output becomes sharper and closer to a discrete one-hot choice
6. What does the Gumbel-Max trick produce?
   • A differentiable mean over all categories
   • An exact categorical sample by adding independent Gumbel noise to logits and taking argmax
   • A uniform distribution regardless of logits
   • A gradient estimate from labels
7. Why is Gumbel-Softmax useful in a neural model?
   • It provides a differentiable relaxation for discrete choices during training
   • It makes every model deterministic
   • 它取代了所有隐藏层
   • It guarantees the learned category is correct
8. What should be remembered about a low-temperature soft sample?
   • It is always a ground-truth label
   • It no longer depends on probabilities
   • It is a near-discrete relaxation, not necessarily identical to a hard argmax in every training setup
   • It guarantees zero estimation bias