Weight Decay and L2 Regularization Quiz

Answers and explanations appear as you take the quiz.

1. What does L2 regularization add to the loss function?
   • The number of layers in the network
   • A penalty on the learning rate
   • The count of training examples
   • A penalty proportional to the sum of squared weights
2. What is the main problem that weight decay helps prevent?
   • Slow data loading
   • Overfitting, where the model memorizes noise in the training data
   • Hardware failures
   • Incorrect labels
3. In which direction does weight decay push the model's weights?
   • Toward infinity
   • Toward the largest weight
   • Toward zero
   • Toward random values
4. Why was AdamW introduced?
   • To decouple weight decay from the adaptive gradient update so decay behaves correctly
   • To remove the gradient entirely
   • To make training slower on purpose
   • To eliminate the need for a learning rate
5. For plain stochastic gradient descent, how do L2 regularization and weight decay relate?
   • They are always opposites
   • They are mathematically equivalent
   • Weight decay disables the gradient
   • L2 only works for SGD with momentum
6. What does the lambda coefficient control in L2 regularization?
   • The batch size
   • The number of epochs
   • The activation function
   • The strength of the penalty on large weights
7. Why can adding an L2 term to Adam’s loss differ from applying decoupled weight decay?
   • Adaptive per-parameter gradient scaling can distort the coupled penalty
   • Adam cannot train with any regularization
   • The two methods use different training labels
   • Decoupled decay removes the data loss
8. In a simple decoupled update, what changes the fractional shrinkage of a weight?
   • Only the number of classes
   • Only the model’s input dimension
   • The learning rate and decay coefficient together
   • Only the validation-set size