引導式測驗 · 中等 等級

混合精準度訓練測驗

涵蓋 FP16 範圍和損失縮放、bfloat16、主權重、自動投射、Tensor Core 累積以及測量的硬體相關增益。

相關引導路徑混合精準訓練
問題 1 的 8

In an FP16 training recipe that keeps an FP32 master weight, why retain it?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. In an FP16 training recipe that keeps an FP32 master weight, why retain it?
    • 為了保持 16 位元可以四捨五入的微小權重更新的精度
    • 使磁碟上的模型檔案更小
    • 允許在沒有 GPU 的情況下進行訓練
    • 加密模型參數
  2. FP16 訓練中損失縮放解決了什麼問題?
    • 模型過度擬合數據
    • 由於 FP16 的範圍很窄,梯度下溢到零
    • GPU 磁碟空間不足
    • 從磁碟載入資料緩慢
  3. bfloat16相對於FP16有什麼優勢?
    • 它使用更少的總位數
    • 它存儲文本而不是數字
    • 它具有與 FP32 相同的寬指數範圍,減少了損失縮放的需要
    • 它僅在 CPU 上運行
  4. For many supported Tensor Core matrix operations, how are partial sums commonly accumulated?
    • In integer arithmetic with no rounding
    • In text before converting to FP16
    • In FP16 only, which prevents every numerical error
    • In FP32, which reduces rounding error compared with low-precision accumulation
  5. What is a common benefit of using 16-bit values for supported training operations?
    • They can reduce tensor memory and improve throughput, with gains depending on hardware and workload
    • They guarantee higher model accuracy
    • They remove the need to compute gradients
    • They always double end-to-end training speed
  6. Which current PyTorch APIs are commonly used for automatic mixed-precision training?
    • torch.cuda.amp only, which is the recommended new namespace
    • torch.amp.autocast and, for FP16 gradient scaling, torch.amp.GradScaler
    • torch.load and torch.save
    • torch.nn.DataParallel and torch.manual_seed
  7. Why does bfloat16 often need less loss scaling than FP16?
    • It has more mantissa bits than FP32
    • It stores gradients as integers
    • It has an exponent range like FP32, so very small gradients are less likely to underflow
    • It never rounds values
  8. What does autocast do in automatic mixed precision?
    • Converts every tensor and operation to FP16 unconditionally
    • Replaces the optimizer with a different algorithm
    • Guarantees identical bitwise results to FP32
    • Selects operation dtypes so supported operations can use lower precision while others retain safer precision