KV Cache Optimization Quiz

Answers and explanations appear as you take the quiz.

1. What does the KV cache store and why?
   • Previously computed attention keys and values so they aren't recomputed each step
   • The model's training data for retraining
   • User passwords for authentication
   • The final text output only
2. Why can the KV cache become a memory problem?
   • It corrupts the GPU permanently
   • It grows with sequence length, batch size, and layers, sometimes exceeding the model weights
   • It only works on text shorter than 10 words
   • It duplicates the operating system
3. What operating-system concept does PagedAttention borrow?
   • Disk defragmentation utilities
   • Antivirus scanning
   • Virtual-memory paging with fixed-size blocks
   • Screen resolution scaling
4. How do Grouped-Query and Multi-Query Attention reduce KV cache size?
   • By deleting the attention mechanism
   • By storing the cache as images
   • By training on less data
   • By letting many query heads share fewer key/value heads
5. What is one benefit of prefix sharing in the KV cache?
   • Requests with the same starting prompt can reuse the same cached blocks
   • It encrypts the model
   • It slows down every request equally
   • It removes the need for attention
6. Compared with storing the same KV entries in 16-bit, what does 8-bit storage do to raw cache-value bytes?
   • It doubles the raw bytes
   • It roughly halves the raw bytes, before metadata and other runtime memory
   • It removes the key and value tensors entirely
   • It guarantees twice the inference speed
7. With full-context attention, no prefix sharing, and a fixed model and cache dtype, what happens if sequence length and batch size both double?
   • It stays constant because model weights are fixed
   • It roughly doubles
   • Raw KV storage grows to roughly four times its previous size
   • It becomes one quarter as large
8. What must generally match for two requests to reuse the same cached prefix blocks?
   • Only the user account name
   • The requests must have different model weights
   • The prompts must have the same character count but different tokens
   • Their tokenized prefix and relevant model/cache configuration must be compatible