Guide-linked quiz · Hard Level

Z-Loss and Training Stability Quiz

Calculate z-loss and distinguish logit normalization from prediction quality, including coefficients, stable computation, and router-loss limits.

Question 1 of 8

Which quantity does the common z-loss term penalize?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Which quantity does the common z-loss term penalize?
    • The squared logarithm of the softmax normalization constant
    • The number of training documents
    • Only the largest class probability
    • The file size of a checkpoint
  2. What happens to softmax probabilities when the same constant is added to every logit in exact arithmetic?
    • They all become zero
    • They remain unchanged
    • They become uniformly distributed for every input
    • The number of output classes doubles
  3. For logits [0, 0], what is the unweighted z-loss?
    • 0
    • 2
    • (ln 2)², approximately 0.48045
    • 1/2
  4. Which equal-logit pair gives Z = 1 and zero z-loss?
    • [0, 0]
    • [1, 1]
    • [ln 2, ln 2]
    • [−ln 2, −ln 2]
  5. Does zero z-loss show that a classifier predicts the right answer?
    • No; it describes normalization, not whether the predicted distribution matches the target
    • Yes; zero auxiliary loss means every class prediction is correct
    • Yes; all logits must then match the labels
    • Only if the model uses two classes
  6. Why should the calculation use a stable logsumexp implementation?
    • To force every probability to be equal
    • To avoid directly exponentiating very large logits
    • To remove the need for a main-task loss
    • To convert all inputs to integers
  7. In an MoE router, how does z-loss differ from a load-balancing auxiliary loss?
    • Both guarantee identical expert utilization
    • Z-loss always replaces the routing algorithm
    • Z-loss controls a log-normalizer quantity; load balancing addresses expert usage
    • Load balancing controls only the output vocabulary size
  8. Which evaluation best tests whether adding z-loss helped a training configuration?
    • Check only whether the auxiliary value fell
    • Assume a published coefficient guarantees stability
    • Stop measuring the main task once training runs
    • Compare stability and held-out task quality while tracking main and auxiliary losses separately

Keep testing yourself

More quizzes picked for your level.

View all quizzes