quiz bu lëkkale ak tegtal · Dëgër Niveau
Xayma mémoire GPU bi war ngir LLMs Quiz
Xaymal LLM mémoire ci poid yi, tolluwaayu tàggat yaram, aktivasioŋ ak cache KV, nga xam naka la contexte, lots ak njub di soppi budget bi.
Laaj 1 de 8
Which formula gives a first-order estimate of stored weight memory?
Xoolaatal laaj bu nekk8 laaj
Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.
- Which formula gives a first-order estimate of stored weight memory?
- Number of layers multiplied by context length alone
- Parameter count without accounting for numeric precision
- Number of parameters multiplied by bytes per parameter
- Batch size multiplied by the output vocabulary only
- Why does full fine-tuning typically need more memory than inference?
- Training avoids loading the model weights
- It stores gradients, optimizer state, and training activations
- Forward activations are never needed for backward computation
- Gradients and optimizer state are absent during training
- What does the factor of two represent in a standard KV-cache estimate?
- Two bytes are assumed per cache value even when another precision is used
- Separate forward and backward model copies are cached
- The number of KV heads is doubled to include query heads
- Separate key and value tensors are stored
- Which variables increase standard KV-cache size?
- Batch size, cached tokens, KV heads, head dimension, and bytes per value
- Only the layer count in the model configuration
- The model parameter count and the number of training examples
- Only context length; active batch size cannot affect cache memory
- How can grouped-query attention affect KV-cache memory?
- It uses more KV heads than query heads by definition
- It requires all cached values to use float32
- It can use fewer key-value heads than query heads
- It removes the KV cache from autoregressive inference
- Why should memory be tested at maximum context and concurrency?
- The cache is fixed once weights are loaded, regardless of active tokens
- The cache and runtime state grow with active tokens and sequences
- Concurrency changes output quality but not memory
- The weight file is rewritten for each simultaneous request
- Which statement about optimizer memory estimates is sound?
- All optimizers use the same state size per parameter
- FP16 weights require every optimizer tensor to use FP16
- Training memory equals weight size plus input tokens only
- Bytes per parameter depend on optimizer and precision choices
- What should follow a formula-based GPU memory estimate?
- Measure peak memory with the actual model, runtime, and workload
- Assume all advertised device memory is usable for weights
- Ignore allocator and temporary workspace buffers
- Test only an empty prompt with no concurrent requests