引導式測驗 · 中等 等級

為機器學習選擇 GPU 測驗

將 GPU 記憶體、頻寬、運算、互連和軟體支援與定義的訓練或推理工作負載相符。用於目標部署設定。

相關引導路徑為機器學習選擇 GPU
問題 1 的 8

What determines whether a model's working state fits on a GPU?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What determines whether a model's working state fits on a GPU?
    • Weights, gradients, and activations alone regardless of inference cache
    • Peak FLOPs relative to the model parameter count
    • Available memory relative to weights, activations, optimizer state, and cache
    • Memory bandwidth relative to the model input rate
  2. How does memory bandwidth differ from memory capacity?
    • Both terms describe the number of bytes installed
    • Bandwidth measures data transfer rate; capacity measures how much can fit
    • Capacity measures transfer rate; bandwidth measures storage size
    • Bandwidth and capacity are both theoretical FLOPs
  3. When can interconnect become important in multi-GPU training?
    • When one GPU processes a workload without exchanging data with peer GPUs
    • Only when data is first read from host storage before training
    • When each model layer fits on one device and no peer transfer occurs
    • When workers exchange gradients, activations, or model partitions
  4. Why verify framework and driver support before selecting hardware?
    • A device may lack compatible or optimized kernels in the current software stack
    • Peak FLOPs guarantee every model operator will use the fastest path
    • Framework support is independent of driver and runtime versions
    • A GPU makes every precision format equally fast
  5. Which benchmark comparison is most informative?
    • Use the same model but different precision and batch sizes on each GPU
    • Compare product-sheet peak values for different arithmetic formats
    • Benchmark the same model, precision, workload, and latency target on each candidate
    • Compare the same model at different sequence lengths and request rates
  6. What can constrain batch-one autoregressive LLM decoding after weights are loaded?
    • Training optimizer state and full-dataset size after weights are loaded
    • Memory bandwidth and KV-cache capacity can constrain batch-one decoding
    • The number of concurrent requests has no impact once model weights fit
    • The inference engine removes all memory traffic after model loading
  7. Why can a high-peak GPU scale poorly across several devices?
    • Adding GPUs automatically raises linear scaling efficiency
    • Peak FLOPs specifications include cross-GPU communication costs
    • Training never synchronizes model or gradient state
    • Communication or synchronization can dominate useful computation
  8. What should guide final hardware selection?
    • Use measured end-to-end workload performance and total cost
    • Choose by peak throughput for a precision the workload cannot use
    • Choose by nominal capacity without runtime memory overhead
    • Choose by throughput measured at a different latency target