Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo Cyamahugurwa Yibanze

Gupfukirana urwego rwa FP16 no gupima igihombo, bfloat16, uburemere bukomeye, autocast, kwirundanya kwa Tensor, no gupima inyungu zishingiye ku byuma.

Inzira zijyanye no kuyoboraAmahugurwa avanze neza
Ikibazo 1 Bya 8

In an FP16 training recipe that keeps an FP32 master weight, why retain it?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. In an FP16 training recipe that keeps an FP32 master weight, why retain it?
    • Kugirango ubungabunge neza muburemere buke buremereye 16-bit ishobora kuzenguruka
    • Gukora dosiye yicyitegererezo ntoya kuri disiki
    • Kwemerera imyitozo idafite GPU
    • Kugirango uhishe icyitegererezo
  2. Ni ikihe kibazo cyo gupima igihombo gikemura mumahugurwa ya FP16?
    • Icyitegererezo kirenze amakuru
    • Gradients itemba kuri zeru kuko intera ya FP16 iragufi
    • GPU ibura umwanya wa disiki
    • Buhoro buhoro amakuru yikuramo kuri disiki
  3. Ni izihe nyungu bfloat16 ifite kuri FP16?
    • Ikoresha bike
    • Irabika inyandiko aho kuba imibare
    • Ifite intera nini yerekana FP32, igabanya gukenera gupima igihombo
    • Ikora kuri CPU gusa
  4. For many supported Tensor Core matrix operations, how are partial sums commonly accumulated?
    • In integer arithmetic with no rounding
    • In text before converting to FP16
    • In FP16 only, which prevents every numerical error
    • In FP32, which reduces rounding error compared with low-precision accumulation
  5. What is a common benefit of using 16-bit values for supported training operations?
    • They can reduce tensor memory and improve throughput, with gains depending on hardware and workload
    • They guarantee higher model accuracy
    • They remove the need to compute gradients
    • They always double end-to-end training speed
  6. Which current PyTorch APIs are commonly used for automatic mixed-precision training?
    • torch.cuda.amp only, which is the recommended new namespace
    • torch.amp.autocast and, for FP16 gradient scaling, torch.amp.GradScaler
    • torch.load and torch.save
    • torch.nn.DataParallel and torch.manual_seed
  7. Why does bfloat16 often need less loss scaling than FP16?
    • It has more mantissa bits than FP32
    • It stores gradients as integers
    • It has an exponent range like FP32, so very small gradients are less likely to underflow
    • It never rounds values
  8. What does autocast do in automatic mixed precision?
    • Converts every tensor and operation to FP16 unconditionally
    • Replaces the optimizer with a different algorithm
    • Guarantees identical bitwise results to FP32
    • Selects operation dtypes so supported operations can use lower precision while others retain safer precision