Guide-linked quiz · Medium Level

Choosing a GPU for Machine Learning Quiz

Match GPU memory, bandwidth, compute, interconnect and software support to a defined training or inference workload. for the target deployment setting.

Question 1 of 8

What determines whether a model's working state fits on a GPU?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What determines whether a model's working state fits on a GPU?
    • Weights, gradients, and activations alone regardless of inference cache
    • Peak FLOPs relative to the model parameter count
    • Available memory relative to weights, activations, optimizer state, and cache
    • Memory bandwidth relative to the model input rate
  2. How does memory bandwidth differ from memory capacity?
    • Both terms describe the number of bytes installed
    • Bandwidth measures data transfer rate; capacity measures how much can fit
    • Capacity measures transfer rate; bandwidth measures storage size
    • Bandwidth and capacity are both theoretical FLOPs
  3. When can interconnect become important in multi-GPU training?
    • When one GPU processes a workload without exchanging data with peer GPUs
    • Only when data is first read from host storage before training
    • When each model layer fits on one device and no peer transfer occurs
    • When workers exchange gradients, activations, or model partitions
  4. Why verify framework and driver support before selecting hardware?
    • A device may lack compatible or optimized kernels in the current software stack
    • Peak FLOPs guarantee every model operator will use the fastest path
    • Framework support is independent of driver and runtime versions
    • A GPU makes every precision format equally fast
  5. Which benchmark comparison is most informative?
    • Use the same model but different precision and batch sizes on each GPU
    • Compare product-sheet peak values for different arithmetic formats
    • Benchmark the same model, precision, workload, and latency target on each candidate
    • Compare the same model at different sequence lengths and request rates
  6. What can constrain batch-one autoregressive LLM decoding after weights are loaded?
    • Training optimizer state and full-dataset size after weights are loaded
    • Memory bandwidth and KV-cache capacity can constrain batch-one decoding
    • The number of concurrent requests has no impact once model weights fit
    • The inference engine removes all memory traffic after model loading
  7. Why can a high-peak GPU scale poorly across several devices?
    • Adding GPUs automatically raises linear scaling efficiency
    • Peak FLOPs specifications include cross-GPU communication costs
    • Training never synchronizes model or gradient state
    • Communication or synchronization can dominate useful computation
  8. What should guide final hardware selection?
    • Use measured end-to-end workload performance and total cost
    • Choose by peak throughput for a precision the workload cannot use
    • Choose by nominal capacity without runtime memory overhead
    • Choose by throughput measured at a different latency target

Keep testing yourself

More quizzes picked for your level.

View all quizzes