Gradient Clipping Quiz

Answers and explanations appear as you take the quiz.

1. What does clip-by-norm gradient clipping primarily preserve while limiting the update?
   • The direction of the overall gradient update
   • The exact magnitude of every gradient
   • The number of model parameters
   • The learning rate schedule
2. In clip-by-norm with threshold c, what happens when the gradient norm g_norm exceeds c?
   • Gradients are set to zero
   • Each gradient is multiplied by c / g_norm
   • Training stops
   • The learning rate is doubled
3. Which problem is gradient clipping specifically designed to combat?
   • Vanishing gradients
   • Slow data loading
   • Exploding gradients
   • Overfitting to the validation set
4. How does clip-by-value differ from clip-by-norm?
   • It always preserves the update direction
   • It only works on convolutional layers
   • It removes gradients entirely
   • It clamps each gradient component independently into a range, which can change the direction
5. A minibatch produces an unusually large global gradient norm. What does clip-by-norm do before the optimizer update?
   • Scales the full gradient vector down to the configured threshold when the norm exceeds it
   • Changes the model’s labels
   • Drops every gradient component independently
   • Increases the learning rate until the loss recovers
6. In differentially private training (DP-SGD), what role does per-example gradient clipping play?
   • It encrypts the data
   • It bounds each example's influence so calibrated noise can guarantee privacy
   • It increases the batch size
   • It removes the need for a loss function
7. The global gradient norm is twice a clipping threshold c. By what factor is the gradient scaled?
   • Two
   • One quarter
   • One half (c divided by 2c)
   • It is left unchanged
8. Why does DP-SGD clip each example’s gradient before adding calibrated noise?
   • To make all examples have the same label
   • To remove the need for a privacy budget
   • To increase the norm of rare-class gradients
   • To bound an individual example’s influence on the released update