quiz bu lëkkale ak tegtal · Dëgër Niveau

ZeRO ak Quiz buy gëna xéewale

Dafay muur etape ZeRO, mémoire stade Adam, dajale parametre, wàññi gradient, tànneefi decharge, ak njëgu jokkoo.

Yooni njiit yu jëm ciZero ak Optimisatër yu Sharded
Laaj 1 de 8

Ban redondance la ZeRO di dindi buñu ko méngale ak parallelism done bu leer?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Ban redondance la ZeRO di dindi buñu ko méngale ak parallelism done bu leer?
    • Ñaari kopi yu mat sëkk ci staadu optimisatër bi, degrade yi ak parametre yi ci GPU bu nekk
    • ñaari kopi done tàggat ci GPU bu nekk
    • Fiil reso redondant ci digganté node yi
    • Baamtu jëm kanam romb ci benn batch
  2. Lan moo waral optimiser nekk mbaam xuux bi gëna mag ci mémoire ak Adam?
    • Adam amul lenn luñu denc
    • Adam dafay denc yeneen valeur ci paramet bu nekk (lu melni momentum ak variance), lu bari yoon lu tollu ci dayo model bi
    • Adam dafay ñaari yoon done yépp
    • Adam bëgg na denc bépp gradient bu weesu ba fàww
  3. Lan la ZeRO Stage 3 xaaj te Stage 1 ak 2 duñu def?
    • Done yiñ joxe ci tàggat yaram
    • Fonction perte
    • parametru model bi ci boppam
    • Jadwalu tolluwaayu jàng
  4. Ci ZeRO Stage 3, naka la GPU di def ba mëna am paramet yi mu soxla ngir mëna dem ca kanam?
    • Du soxla parametre yu mat
    • Daf leen di yebbi ci net bi
    • Saa yu nekk daf leen di xayma ci noonu
    • Dafay def all-gather ngir dajale daggitu yi ñàkk, ba noppi sànni leen ginaaw ga
  5. Ban man-manu PyTorch mooy jëfandikoo sharding bu nuroo ak ZeRO?
    • FSDP (Done yuñ xaaj lépp te wuute)
    • Sargeeru done
    • TorchSkript
    • Otograd
  6. Lan mooy kompromis bu mag bi ZeRO def ngir sakkanal mémoire bi?
    • Dafay bàyyi model bu jaar yoon ngir gaawaay
    • Dafay yokk jokkoo ngir dajale ak tasaare ay piyees yuñ soxla
    • Dina dindi ay couche ci model bi
    • Dafay dox ludul ci model yu ndaw
  7. Which state is partitioned in ZeRO Stage 1?
    • Only parameters, with optimizer states replicated
    • Parameters, gradients, and optimizer states all sharded
    • Optimizer states, while parameters and gradients remain replicated
    • Only input activations and labels
  8. What is a practical cost of ZeRO-style sharding?
    • It removes the need to calculate gradients
    • It makes every optimizer update local with no data exchange
    • It requires the model to be retrained from scratch for every step
    • Additional communication to gather parameters and distribute gradients