Maswali yanayounganishwa na mwongozo · Ngumu Kiwango

Maswali ya Gumbel-Softmax na Reparameterization

Hujaribu sampuli za kategoria, utulivu unaoweza kutofautishwa, kelele ya Gumbel, halijoto, chaguo ngumu na vikomo vya matokeo laini.

Njia za mwongozo zinazohusianaGumbel Softmax Na Reparameterization
Swali 1 ya 8

Why is a categorical sample difficult to train through with ordinary backpropagation?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Why is a categorical sample difficult to train through with ordinary backpropagation?
    • Categorical distributions have no probabilities
    • The discrete choice is a non-differentiable operation
    • Sampling removes the input tensor
    • All categories have identical logits
  2. What does the reparameterization trick aim to do?
    • Remove all randomness from training
    • Replace gradients with labels
    • Make categorical outputs continuous at evaluation time
    • Express randomness through an external noise variable while keeping a differentiable path
  3. How does Gumbel-Softmax approximate a categorical choice?
    • Use a hard class label as a gradient
    • Normalize the input pixels
    • Add Gumbel noise to logits and apply a temperature-controlled softmax
    • Average logits across unrelated examples
  4. What happens to the distribution as temperature becomes large?
    • It becomes smoother across categories
    • It becomes an exact one-hot sample
    • The logits are discarded
    • It becomes independent of the Gumbel noise
  5. What happens as temperature approaches zero?
    • The output becomes uniform
    • The noise becomes a label
    • The operation stops depending on logits
    • The output becomes sharper and closer to a discrete one-hot choice
  6. What does the Gumbel-Max trick produce?
    • A differentiable mean over all categories
    • An exact categorical sample by adding independent Gumbel noise to logits and taking argmax
    • A uniform distribution regardless of logits
    • A gradient estimate from labels
  7. Why is Gumbel-Softmax useful in a neural model?
    • It provides a differentiable relaxation for discrete choices during training
    • It makes every model deterministic
    • Inachukua nafasi ya tabaka zote zilizofichwa
    • It guarantees the learned category is correct
  8. What should be remembered about a low-temperature soft sample?
    • It is always a ground-truth label
    • It no longer depends on probabilities
    • It is a near-discrete relaxation, not necessarily identical to a hard argmax in every training setup
    • It guarantees zero estimation bias