Gradient Descent Quiz

Answers and explanations appear as you take the quiz.

1. In plain gradient descent, which direction is used relative to the objective's gradient?
   • The negative-gradient direction
   • The positive-gradient direction
   • An arbitrary perpendicular direction by definition
   • A direction chosen without evaluating any gradient
2. What does the learning rate control?
   • The number of layers in the model
   • How many classes the model can predict
   • The size of each step taken during optimization
   • The amount of training data used
3. How does a mini-batch gradient relate to the full selected training-data objective?
   • It uses no examples
   • It is computed on the batch and used as an estimate of the full-data gradient
   • It must equal the full-data gradient on every step
   • It only exists for images
4. What problem can a learning rate that is too large cause?
   • Training becomes extremely slow but stable
   • The model ignores the gradient
   • The dataset becomes corrupted
   • The model overshoots the minimum and may diverge
5. What information does momentum carry into later gradient-descent updates?
   • Only the model's vocabulary size
   • A permanent deletion of all earlier gradients
   • A running weighted combination of earlier gradients
   • An unrelated random reset of every weight
6. What is a key feature of the Adam optimizer compared to plain gradient descent?
   • It adapts the effective learning rate for each parameter
   • It uses a single fixed step for every parameter
   • It does not require gradients
   • It only trains the output layer
7. For plain gradient descent with old weight two, gradient three, and learning rate one-tenth, what is the new weight?
   • Two-point-three
   • One-point-nine-seven
   • Three
   • One-point-seven
8. What does a zero gradient fail to prove for a nonconvex training objective?
   • That the gradient value is zero at the evaluated point
   • That the current point is a global minimum
   • That the objective has been evaluated locally
   • That a stationary-point condition holds