KV Cache Quiz

Answers and explanations appear as you take the quiz.

1. What does the KV cache store?
   • The key and value vectors computed for previous tokens
   • The model's full weight matrices
   • The user's chat history as plain text
   • The final output probabilities
2. What problem does the KV cache primarily solve?
   • It improves the grammar of generated text
   • It avoids recomputing keys and values for all prior tokens at every step
   • It increases the model's vocabulary
   • It encrypts the conversation
3. What is the main downside of the KV cache?
   • It makes generation slower
   • It reduces output quality
   • Its memory usage grows linearly with context length
   • It only works on CPUs
4. Which technique reduces KV cache size by sharing keys and values across attention heads?
   • Dropout
   • Beam search
   • Tokenization
   • Grouped-query or multi-query attention
5. What are the two phases of transformer inference relative to the KV cache?
   • Prefill (process the prompt) and decode (generate one token at a time)
   • Training and testing
   • Encode and compress
   • Upload and download
6. What does vLLM's PagedAttention do for the KV cache?
   • Deletes the cache after each token
   • Allocates the cache in non-contiguous blocks to reduce memory fragmentation
   • Converts the cache into text
   • Doubles the model's parameter count
7. Why can KV-cache memory limit how many requests a GPU serves concurrently?
   • The cache stores all training examples on the GPU
   • Each request creates a new model copy
   • Each active sequence retains key and value tensors that grow with its generated context
   • The cache stores only final text on disk
8. What is one benefit of sharing KV-cache blocks among requests with a common prefix?
   • It makes unrelated prefixes identical
   • It removes the need to compute attention
   • It guarantees lower latency in every workload
   • It can avoid storing duplicate prefix state and reduce memory use