Quiz lié à un guide · Difficile Niveau

Gumbel-Softmax et Quiz de reparamétrage

Teste l'échantillonnage catégoriel, les relaxations différenciables, le bruit de Gumbel, la température, les choix difficiles et les limites des sorties douces.

Chemins de guidage associésGumbel Softmax et reparamétrage
Question 1 de 8

Why is a categorical sample difficult to train through with ordinary backpropagation?

Prévisualisez chaque question8 des questions

Les réponses et les explications apparaissent au fur et à mesure que vous répondez au quiz.

  1. Why is a categorical sample difficult to train through with ordinary backpropagation?
    • Categorical distributions have no probabilities
    • The discrete choice is a non-differentiable operation
    • Sampling removes the input tensor
    • All categories have identical logits
  2. What does the reparameterization trick aim to do?
    • Remove all randomness from training
    • Replace gradients with labels
    • Make categorical outputs continuous at evaluation time
    • Express randomness through an external noise variable while keeping a differentiable path
  3. How does Gumbel-Softmax approximate a categorical choice?
    • Use a hard class label as a gradient
    • Normalize the input pixels
    • Add Gumbel noise to logits and apply a temperature-controlled softmax
    • Average logits across unrelated examples
  4. What happens to the distribution as temperature becomes large?
    • It becomes smoother across categories
    • It becomes an exact one-hot sample
    • The logits are discarded
    • It becomes independent of the Gumbel noise
  5. What happens as temperature approaches zero?
    • The output becomes uniform
    • The noise becomes a label
    • The operation stops depending on logits
    • The output becomes sharper and closer to a discrete one-hot choice
  6. What does the Gumbel-Max trick produce?
    • A differentiable mean over all categories
    • An exact categorical sample by adding independent Gumbel noise to logits and taking argmax
    • A uniform distribution regardless of logits
    • A gradient estimate from labels
  7. Why is Gumbel-Softmax useful in a neural model?
    • It provides a differentiable relaxation for discrete choices during training
    • It makes every model deterministic
    • Il remplace toutes les couches cachées
    • It guarantees the learned category is correct
  8. What should be remembered about a low-temperature soft sample?
    • It is always a ground-truth label
    • It no longer depends on probabilities
    • It is a near-discrete relaxation, not necessarily identical to a hard argmax in every training setup
    • It guarantees zero estimation bias