Learning Rate Scheduling Quiz

Answers and explanations appear as you take the quiz.

1. What is the purpose of a 'warmup' phase in a learning rate schedule?
   • To start with a small rate and ramp up, avoiding instability from early noisy gradients
   • To make the final accuracy lower
   • To skip the first few batches of data
   • To freeze all the weights at the start
2. Which schedule smoothly follows a half-cosine curve down toward a minimum rate?
   • Constant schedule
   • Cosine annealing
   • Random schedule
   • Linear increase
3. What happens if the learning rate is set far too high?
   • Training converges instantly with perfect accuracy
   • The model trains slower but always succeeds
   • The loss can diverge or oscillate instead of decreasing
   • Nothing changes; the rate has no effect
4. What does step decay do to the learning rate?
   • Increases it linearly every batch
   • Keeps it perfectly constant
   • Sets it randomly each step
   • Drops it by a fixed factor at predefined epochs
5. What does a warm restart do to a learning-rate schedule?
   • It raises the learning rate again to begin another schedule cycle
   • It resets the model parameters to random values
   • It permanently fixes the learning rate at zero
   • It deletes the optimizer’s moment estimates
6. In Goyal et al.’s large-minibatch ImageNet setup, what did their warmup scheme address?
   • It removed the need for a validation set
   • It eased early optimization difficulties after the learning-rate scale increased with batch size
   • It changed the model architecture after each epoch
   • It converted the learning-rate curve into a cosine wave
7. In the cosine formula, what does T denote?
   • The number of model layers
   • The batch size
   • The schedule horizon or total number of steps used for the decay
   • The optimizer’s momentum coefficient
8. If the cosine schedule uses lr_min greater than zero, where does the rate approach at the end of the horizon?
   • Exactly zero regardless of lr_min
   • The original warmup starting rate
   • An unbounded value
   • The specified minimum learning rate