Guide-linked quiz · Hard Level
Z-Loss and Training Stability Quiz
Calculate z-loss and distinguish logit normalization from prediction quality, including coefficients, stable computation, and router-loss limits.
Question 1 of 8
Which quantity does the common z-loss term penalize?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- Which quantity does the common z-loss term penalize?
- The squared logarithm of the softmax normalization constant
- The number of training documents
- Only the largest class probability
- The file size of a checkpoint
- What happens to softmax probabilities when the same constant is added to every logit in exact arithmetic?
- They all become zero
- They remain unchanged
- They become uniformly distributed for every input
- The number of output classes doubles
- For logits [0, 0], what is the unweighted z-loss?
- 0
- 2
- (ln 2)², approximately 0.48045
- 1/2
- Which equal-logit pair gives Z = 1 and zero z-loss?
- [0, 0]
- [1, 1]
- [ln 2, ln 2]
- [−ln 2, −ln 2]
- Does zero z-loss show that a classifier predicts the right answer?
- No; it describes normalization, not whether the predicted distribution matches the target
- Yes; zero auxiliary loss means every class prediction is correct
- Yes; all logits must then match the labels
- Only if the model uses two classes
- Why should the calculation use a stable logsumexp implementation?
- To force every probability to be equal
- To avoid directly exponentiating very large logits
- To remove the need for a main-task loss
- To convert all inputs to integers
- In an MoE router, how does z-loss differ from a load-balancing auxiliary loss?
- Both guarantee identical expert utilization
- Z-loss always replaces the routing algorithm
- Z-loss controls a log-normalizer quantity; load balancing addresses expert usage
- Load balancing controls only the output vocabulary size
- Which evaluation best tests whether adding z-loss helped a training configuration?
- Check only whether the auxiliary value fell
- Assume a published coefficient guarantees stability
- Stop measuring the main task once training runs
- Compare stability and held-out task quality while tracking main and auxiliary losses separately
Keep testing yourself
More quizzes picked for your level.