Guide-linked quiz · Hard Level

Estimating GPU Memory Needed for LLMs Quiz

Estimate LLM memory from weights, training state, activations and KV cache, and identify how context, batch and precision change the budget.

Question 1 of 8

Which formula gives a first-order estimate of stored weight memory?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Which formula gives a first-order estimate of stored weight memory?
    • Number of layers multiplied by context length alone
    • Parameter count without accounting for numeric precision
    • Number of parameters multiplied by bytes per parameter
    • Batch size multiplied by the output vocabulary only
  2. Why does full fine-tuning typically need more memory than inference?
    • Training avoids loading the model weights
    • It stores gradients, optimizer state, and training activations
    • Forward activations are never needed for backward computation
    • Gradients and optimizer state are absent during training
  3. What does the factor of two represent in a standard KV-cache estimate?
    • Two bytes are assumed per cache value even when another precision is used
    • Separate forward and backward model copies are cached
    • The number of KV heads is doubled to include query heads
    • Separate key and value tensors are stored
  4. Which variables increase standard KV-cache size?
    • Batch size, cached tokens, KV heads, head dimension, and bytes per value
    • Only the layer count in the model configuration
    • The model parameter count and the number of training examples
    • Only context length; active batch size cannot affect cache memory
  5. How can grouped-query attention affect KV-cache memory?
    • It uses more KV heads than query heads by definition
    • It requires all cached values to use float32
    • It can use fewer key-value heads than query heads
    • It removes the KV cache from autoregressive inference
  6. Why should memory be tested at maximum context and concurrency?
    • The cache is fixed once weights are loaded, regardless of active tokens
    • The cache and runtime state grow with active tokens and sequences
    • Concurrency changes output quality but not memory
    • The weight file is rewritten for each simultaneous request
  7. Which statement about optimizer memory estimates is sound?
    • All optimizers use the same state size per parameter
    • FP16 weights require every optimizer tensor to use FP16
    • Training memory equals weight size plus input tokens only
    • Bytes per parameter depend on optimizer and precision choices
  8. What should follow a formula-based GPU memory estimate?
    • Measure peak memory with the actual model, runtime, and workload
    • Assume all advertised device memory is usable for weights
    • Ignore allocator and temporary workspace buffers
    • Test only an empty prompt with no concurrent requests

Keep testing yourself

More quizzes picked for your level.

View all quizzes