Kvíz spojený s průvodcem · Těžko úroveň
Odhad paměti GPU potřebné pro kvíz LLM
Odhadněte paměť LLM z vah, stavu tréninku, aktivací a mezipaměti KV a zjistěte, jak kontext, dávka a přesnost mění rozpočet.
Otázka 1 z 8
Which formula gives a first-order estimate of stored weight memory?
Náhled každé otázky8 otázky
Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.
- Which formula gives a first-order estimate of stored weight memory?
- Number of layers multiplied by context length alone
- Parameter count without accounting for numeric precision
- Number of parameters multiplied by bytes per parameter
- Batch size multiplied by the output vocabulary only
- Why does full fine-tuning typically need more memory than inference?
- Training avoids loading the model weights
- It stores gradients, optimizer state, and training activations
- Forward activations are never needed for backward computation
- Gradients and optimizer state are absent during training
- What does the factor of two represent in a standard KV-cache estimate?
- Two bytes are assumed per cache value even when another precision is used
- Separate forward and backward model copies are cached
- The number of KV heads is doubled to include query heads
- Separate key and value tensors are stored
- Which variables increase standard KV-cache size?
- Batch size, cached tokens, KV heads, head dimension, and bytes per value
- Only the layer count in the model configuration
- The model parameter count and the number of training examples
- Only context length; active batch size cannot affect cache memory
- How can grouped-query attention affect KV-cache memory?
- It uses more KV heads than query heads by definition
- It requires all cached values to use float32
- It can use fewer key-value heads than query heads
- It removes the KV cache from autoregressive inference
- Why should memory be tested at maximum context and concurrency?
- The cache is fixed once weights are loaded, regardless of active tokens
- The cache and runtime state grow with active tokens and sequences
- Concurrency changes output quality but not memory
- The weight file is rewritten for each simultaneous request
- Which statement about optimizer memory estimates is sound?
- All optimizers use the same state size per parameter
- FP16 weights require every optimizer tensor to use FP16
- Training memory equals weight size plus input tokens only
- Bytes per parameter depend on optimizer and precision choices
- What should follow a formula-based GPU memory estimate?
- Measure peak memory with the actual model, runtime, and workload
- Assume all advertised device memory is usable for weights
- Ignore allocator and temporary workspace buffers
- Test only an empty prompt with no concurrent requests