Multi-Head Latent Attention Quiz

Answers and explanations appear as you take the quiz.

1. What is the primary problem Multi-Head Latent Attention is designed to reduce?
   • The memory cost of the key-value cache during inference
   • Training data scarcity
   • The number of model parameters
   • Tokenizer vocabulary size
2. How does MLA shrink the KV cache?
   • It deletes older tokens from the cache
   • It stores a small low-rank latent vector instead of full per-head keys and values
   • It uses 1-bit weights for everything
   • It skips attention entirely on some tokens
3. Which model first introduced Multi-Head Latent Attention?
   • GPT-2
   • BERT
   • DeepSeek-V2
   • LLaMA 1
4. Why does MLA need a separate 'decoupled' path for rotary position embeddings?
   • Because positions are irrelevant to attention
   • Because RoPE doubles the vocabulary
   • Because it speeds up the optimizer
   • Because rotation cannot be folded into the absorbed up-projection like the rest of the key can
5. Roughly how much KV-cache memory reduction did DeepSeek-V2 report from MLA versus standard multi-head attention?
   • Over 90%
   • About 5%
   • Exactly 50%
   • No reduction, only speed gains
6. What practical benefit follows directly from MLA's smaller per-token cache?
   • Elimination of the need for any GPU
   • Larger inference batch sizes and longer contexts on the same hardware
   • Automatic translation into all languages
   • Removal of the softmax operation
7. What is the purpose of absorbing MLA up-projection weights into other projections?
   • Remove positional information from every token
   • Increase the size of the cached latent
   • Avoid materializing full per-head keys and values during inference
   • Replace attention with a recurrent network
8. What positional information does MLA handle on a separate decoupled key path?
   • Class labels from the training data
   • Image segmentation maps
   • The output vocabulary probabilities
   • Rotary position embeddings that cannot be absorbed like the other key projections