Kvíz spojený s průvodcem · Těžko úroveň

Odhad paměti GPU potřebné pro kvíz LLM

Odhadněte paměť LLM z vah, stavu tréninku, aktivací a mezipaměti KV a zjistěte, jak kontext, dávka a přesnost mění rozpočet.

Související vodicí cestyOdhad paměti GPU pro LLM
Otázka 1 z 8

Which formula gives a first-order estimate of stored weight memory?

Náhled každé otázky8 otázky

Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.

  1. Which formula gives a first-order estimate of stored weight memory?
    • Number of layers multiplied by context length alone
    • Parameter count without accounting for numeric precision
    • Number of parameters multiplied by bytes per parameter
    • Batch size multiplied by the output vocabulary only
  2. Why does full fine-tuning typically need more memory than inference?
    • Training avoids loading the model weights
    • It stores gradients, optimizer state, and training activations
    • Forward activations are never needed for backward computation
    • Gradients and optimizer state are absent during training
  3. What does the factor of two represent in a standard KV-cache estimate?
    • Two bytes are assumed per cache value even when another precision is used
    • Separate forward and backward model copies are cached
    • The number of KV heads is doubled to include query heads
    • Separate key and value tensors are stored
  4. Which variables increase standard KV-cache size?
    • Batch size, cached tokens, KV heads, head dimension, and bytes per value
    • Only the layer count in the model configuration
    • The model parameter count and the number of training examples
    • Only context length; active batch size cannot affect cache memory
  5. How can grouped-query attention affect KV-cache memory?
    • It uses more KV heads than query heads by definition
    • It requires all cached values to use float32
    • It can use fewer key-value heads than query heads
    • It removes the KV cache from autoregressive inference
  6. Why should memory be tested at maximum context and concurrency?
    • The cache is fixed once weights are loaded, regardless of active tokens
    • The cache and runtime state grow with active tokens and sequences
    • Concurrency changes output quality but not memory
    • The weight file is rewritten for each simultaneous request
  7. Which statement about optimizer memory estimates is sound?
    • All optimizers use the same state size per parameter
    • FP16 weights require every optimizer tensor to use FP16
    • Training memory equals weight size plus input tokens only
    • Bytes per parameter depend on optimizer and precision choices
  8. What should follow a formula-based GPU memory estimate?
    • Measure peak memory with the actual model, runtime, and workload
    • Assume all advertised device memory is usable for weights
    • Ignore allocator and temporary workspace buffers
    • Test only an empty prompt with no concurrent requests