混合精度训练测验

当您参加测验时，就会出现答案和解释。

1. In an FP16 training recipe that keeps an FP32 master weight, why retain it?
   • 为了保持 16 位可以四舍五入的微小权重更新的精度
   • 使磁盘上的模型文件更小
   • 允许在没有 GPU 的情况下进行训练
   • 加密模型参数
2. FP16 训练中损失缩放解决了什么问题？
   • 模型过度拟合数据
   • 由于 FP16 的范围很窄，梯度下溢到零
   • GPU 磁盘空间不足
   • 从磁盘加载数据缓慢
3. bfloat16相对于FP16有什么优势？
   • 它使用更少的总位数
   • 它存储文本而不是数字
   • 它具有与 FP32 相同的宽指数范围，减少了损失缩放的需要
   • 它仅在 CPU 上运行
4. For many supported Tensor Core matrix operations, how are partial sums commonly accumulated?
   • In integer arithmetic with no rounding
   • In text before converting to FP16
   • In FP16 only, which prevents every numerical error
   • In FP32, which reduces rounding error compared with low-precision accumulation
5. What is a common benefit of using 16-bit values for supported training operations?
   • They can reduce tensor memory and improve throughput, with gains depending on hardware and workload
   • They guarantee higher model accuracy
   • They remove the need to compute gradients
   • They always double end-to-end training speed
6. Which current PyTorch APIs are commonly used for automatic mixed-precision training?
   • torch.cuda.amp only, which is the recommended new namespace
   • torch.amp.autocast and, for FP16 gradient scaling, torch.amp.GradScaler
   • torch.load and torch.save
   • torch.nn.DataParallel and torch.manual_seed
7. Why does bfloat16 often need less loss scaling than FP16?
   • It has more mantissa bits than FP32
   • It stores gradients as integers
   • It has an exponent range like FP32, so very small gradients are less likely to underflow
   • It never rounds values
8. What does autocast do in automatic mixed precision?
   • Converts every tensor and operation to FP16 unconditionally
   • Replaces the optimizer with a different algorithm
   • Guarantees identical bitwise results to FP32
   • Selects operation dtypes so supported operations can use lower precision while others retain safer precision