Gumbel-Softmax and Reparameterization Quiz

Answers and explanations appear as you take the quiz.

1. Why is a categorical sample difficult to train through with ordinary backpropagation?
   • Categorical distributions have no probabilities
   • The discrete choice is a non-differentiable operation
   • Sampling removes the input tensor
   • All categories have identical logits
2. What does the reparameterization trick aim to do?
   • Remove all randomness from training
   • Replace gradients with labels
   • Make categorical outputs continuous at evaluation time
   • Express randomness through an external noise variable while keeping a differentiable path
3. How does Gumbel-Softmax approximate a categorical choice?
   • Use a hard class label as a gradient
   • Normalize the input pixels
   • Add Gumbel noise to logits and apply a temperature-controlled softmax
   • Average logits across unrelated examples
4. What happens to the distribution as temperature becomes large?
   • It becomes smoother across categories
   • It becomes an exact one-hot sample
   • The logits are discarded
   • It becomes independent of the Gumbel noise
5. What happens as temperature approaches zero?
   • The output becomes uniform
   • The noise becomes a label
   • The operation stops depending on logits
   • The output becomes sharper and closer to a discrete one-hot choice
6. What does the Gumbel-Max trick produce?
   • A differentiable mean over all categories
   • An exact categorical sample by adding independent Gumbel noise to logits and taking argmax
   • A uniform distribution regardless of logits
   • A gradient estimate from labels
7. Why is Gumbel-Softmax useful in a neural model?
   • It provides a differentiable relaxation for discrete choices during training
   • It makes every model deterministic
   • It replaces all hidden layers
   • It guarantees the learned category is correct
8. What should be remembered about a low-temperature soft sample?
   • It is always a ground-truth label
   • It no longer depends on probabilities
   • It is a near-discrete relaxation, not necessarily identical to a hard argmax in every training setup
   • It guarantees zero estimation bias