Lookahead and Lion Optimizers Quiz

Answers and explanations appear as you take the quiz.

1. What is the defining computational step in the Lion optimizer's weight update?
   • Taking the sign of a momentum-based term
   • Dividing the gradient by its squared running average
   • Computing a second-order Hessian approximation
   • Averaging weights across multiple checkpoints
2. How does Lookahead structure its optimization?
   • It uses a single set of weights updated twice per step
   • It maintains 'fast' weights updated by a base optimizer and 'slow' weights that periodically interpolate toward them
   • It removes momentum entirely to save memory
   • It trains two networks and averages their outputs
3. How was the Lion optimizer originally discovered?
   • By hand-deriving it from convex optimization theory
   • By averaging the updates of SGD and Adam
   • Through automated symbolic program search over optimizer algorithms
   • From a physics simulation of particle motion
4. Compared with Adam, what is a key memory advantage of Lion?
   • It stores no state at all
   • It uses half-precision weights automatically
   • It shares state across all layers
   • It keeps only one momentum buffer instead of Adam's two state tensors
5. Why does Lion typically require a smaller learning rate than Adam?
   • Because the sign operation makes every update's magnitude uniform, effectively scaling the step
   • Because it uses no momentum
   • Because it processes larger batches
   • Because it disables weight decay
6. A practical benefit of Lookahead is that it tends to make training:
   • More sensitive to the choice of hyperparameters
   • More stable and robust, with reduced oscillation
   • Slower per step by a factor of ten
   • Incompatible with Adam or SGD
7. Lookahead slow weights are 0 and the fast weights end an inner loop at 8. If alpha is 0.25, where do the slow weights move?
   • To 8, regardless of alpha
   • To −2, away from the fast weights
   • To 2, one quarter of the distance toward the fast weights
   • They remain at 0 because alpha only changes Lion
8. After Lookahead updates its slow weights at synchronization, where do the fast weights restart?
   • At the original random initialization
   • At the final validation predictions
   • At the previous slow weights before interpolation
   • At the updated slow-weight values for the next inner loop