Adam and Adaptive Optimizers Quiz

Answers and explanations appear as you take the quiz.

1. What two quantities does Adam track for each parameter?
   • The running average of gradients and of squared gradients
   • The maximum and minimum gradient ever seen
   • The learning rate and the batch size
   • The weight value and its sign
2. Why does Adam apply bias correction to its moment estimates?
   • Because gradients are always negative early in training
   • Because the moving averages start at zero and are biased toward zero initially
   • To make the learning rate larger over time
   • To remove outlier data points from the batch
3. What is the main difference between Adam and AdamW?
   • AdamW uses no momentum at all
   • AdamW removes the second moment estimate
   • AdamW decouples weight decay from the gradient-based update
   • AdamW only works on convolutional networks
4. What role does the small epsilon term play in Adam's update rule?
   • It sets the momentum strength
   • It controls the number of training epochs
   • It increases the gradient magnitude
   • It prevents division by zero and stabilizes the update
5. Why is Adam often described as 'adaptive'?
   • It effectively gives each parameter its own scaled step size
   • It changes the network architecture during training
   • It adapts the dataset to the model
   • It switches between classification and regression automatically
6. Which Adam parameter controls the exponential decay rate of its squared-gradient estimate?
   • Beta1
   • Beta2
   • Epsilon
   • The weight-decay coefficient
7. A parameter repeatedly receives large gradients. Which Adam quantity helps scale down its effective step?
   • The number of model layers
   • The batch-normalization mean
   • The second-moment estimate of squared gradients
   • The AdamW weight-decay term alone
8. What happens when beta2 is set closer to one, all else equal?
   • The optimizer stops using gradients
   • The first-moment estimate is deleted
   • Weight decay becomes decoupled automatically
   • The squared-gradient estimate decays more slowly and retains a longer history