Kvíz spojený s průvodcem · Těžko úroveň

Kvíz Z-Loss a trénink stability

Vypočítejte ztrátu z a odlište logitovou normalizaci od kvality predikce, včetně koeficientů, stabilního výpočtu a limitů ztráty směrovače.

Související vodicí cestyZ Ztráta a stabilita tréninku
Otázka 1 z 8

Which quantity does the common z-loss term penalize?

Náhled každé otázky8 otázky

Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.

  1. Which quantity does the common z-loss term penalize?
    • The squared logarithm of the softmax normalization constant
    • The number of training documents
    • Only the largest class probability
    • The file size of a checkpoint
  2. What happens to softmax probabilities when the same constant is added to every logit in exact arithmetic?
    • They all become zero
    • They remain unchanged
    • They become uniformly distributed for every input
    • The number of output classes doubles
  3. For logits [0, 0], what is the unweighted z-loss?
    • 0
    • 2
    • (ln 2)², approximately 0.48045
    • 1/2
  4. Which equal-logit pair gives Z = 1 and zero z-loss?
    • [0, 0]
    • [1, 1]
    • [ln 2, ln 2]
    • [−ln 2, −ln 2]
  5. Does zero z-loss show that a classifier predicts the right answer?
    • No; it describes normalization, not whether the predicted distribution matches the target
    • Yes; zero auxiliary loss means every class prediction is correct
    • Yes; all logits must then match the labels
    • Only if the model uses two classes
  6. Why should the calculation use a stable logsumexp implementation?
    • To force every probability to be equal
    • To avoid directly exponentiating very large logits
    • To remove the need for a main-task loss
    • To convert all inputs to integers
  7. In an MoE router, how does z-loss differ from a load-balancing auxiliary loss?
    • Both guarantee identical expert utilization
    • Z-loss always replaces the routing algorithm
    • Z-loss controls a log-normalizer quantity; load balancing addresses expert usage
    • Load balancing controls only the output vocabulary size
  8. Which evaluation best tests whether adding z-loss helped a training configuration?
    • Check only whether the auxiliary value fell
    • Assume a published coefficient guarantees stability
    • Stop measuring the main task once training runs
    • Compare stability and held-out task quality while tracking main and auxiliary losses separately