Kvíz spojený s průvodcem · Těžko úroveň
Kvíz Z-Loss a trénink stability
Vypočítejte ztrátu z a odlište logitovou normalizaci od kvality predikce, včetně koeficientů, stabilního výpočtu a limitů ztráty směrovače.
Otázka 1 z 8
Which quantity does the common z-loss term penalize?
Náhled každé otázky8 otázky
Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.
- Which quantity does the common z-loss term penalize?
- The squared logarithm of the softmax normalization constant
- The number of training documents
- Only the largest class probability
- The file size of a checkpoint
- What happens to softmax probabilities when the same constant is added to every logit in exact arithmetic?
- They all become zero
- They remain unchanged
- They become uniformly distributed for every input
- The number of output classes doubles
- For logits [0, 0], what is the unweighted z-loss?
- 0
- 2
- (ln 2)², approximately 0.48045
- 1/2
- Which equal-logit pair gives Z = 1 and zero z-loss?
- [0, 0]
- [1, 1]
- [ln 2, ln 2]
- [−ln 2, −ln 2]
- Does zero z-loss show that a classifier predicts the right answer?
- No; it describes normalization, not whether the predicted distribution matches the target
- Yes; zero auxiliary loss means every class prediction is correct
- Yes; all logits must then match the labels
- Only if the model uses two classes
- Why should the calculation use a stable logsumexp implementation?
- To force every probability to be equal
- To avoid directly exponentiating very large logits
- To remove the need for a main-task loss
- To convert all inputs to integers
- In an MoE router, how does z-loss differ from a load-balancing auxiliary loss?
- Both guarantee identical expert utilization
- Z-loss always replaces the routing algorithm
- Z-loss controls a log-normalizer quantity; load balancing addresses expert usage
- Load balancing controls only the output vocabulary size
- Which evaluation best tests whether adding z-loss helped a training configuration?
- Check only whether the auxiliary value fell
- Assume a published coefficient guarantees stability
- Stop measuring the main task once training runs
- Compare stability and held-out task quality while tracking main and auxiliary losses separately