Stochastic Gradient Descent with Momentum Quiz

Answers and explanations appear as you take the quiz.

1. What does the momentum term accumulate during training?
   • The total number of training epochs
   • A running (exponentially weighted) average of past gradients
   • The size of the dataset
   • The model's accuracy history
2. In a long, narrow ravine, what problem does plain SGD suffer that momentum helps fix?
   • It overflows numerically
   • It stops computing gradients
   • It increases the learning rate automatically
   • It zig-zags across the steep walls while progressing slowly along the floor
3. A common default value for the momentum coefficient is:
   • 0.9
   • 0.0
   • 0.5
   • 2.0
4. Which physical analogy is most often used to describe momentum?
   • A pendulum swinging forever
   • A spring snapping back
   • A heavy ball rolling downhill
   • Water freezing
5. Roughly how much does momentum amplify the effective step in a consistent direction when beta = 0.9?
   • About 2 times
   • About 0.1 times
   • No amplification
   • About 10 times
6. Which modern optimizer incorporates a momentum-style first-moment estimate?
   • Newton's method
   • Adam
   • K-means
   • Random search
7. In the update v = beta times v plus gradient, what happens when beta is zero?
   • The old velocity is doubled
   • Every gradient becomes zero
   • The previous velocity is discarded and the current gradient determines the step
   • The parameters stop depending on the learning rate
8. Why can repeated gradients with opposing sideways directions produce less zig-zagging under momentum?
   • Their opposing contributions can partially cancel in accumulated velocity
   • All past gradients are stored as separate parameters
   • The loss becomes convex automatically
   • Momentum ignores the current gradient entirely