Laajteb dajale degrade

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

1. Lan lay njëkka may nga def dajale gradient?
   • Tàggat te amul benn GPU
   • Teg ci paas biy dellu ginaaw
   • Simule ab dayo lote bu gëna mag ci memory bu néew
   • dindil optimisatër bi lëmm
2. Ak dayo micro-lote 4 ak 8 jéego yuñ dajale, dayo lote bi gën mooy:
   • 32
   • 4
   • 8
   • 12
3. With an accumulation factor N, when should optimizer.step() and zero_grad() occur?
   • Ginaaw bépp micro-lote
   • Musul
   • Laata ñuy jàll ci kanam
   • Call optimizer.step() after N micro-batches, then clear gradients before the next accumulation window
4. Ngir mëna wéy di jàng bu baax, perte micro-batch bu nekk dafay faral di nekk:
   • yokk ci N
   • Séddale ko ci limu jéego yiñ def ngir dajale N
   • Def ci zero
   • ñu wecci ko ak njub
5. Ban couche de normalisation moo gëna set ak dajalu gradient bi?
   • Normalisation layer wala groupe, ndax nekkul ci lim yiñ def ci lots
   • Normalisasioŋ ci lots, ndax dafa bëgg lots yu ndaw
   • Amul benn normalisasioŋ buy dox ak moom
   • Dafay bàyyi jàng
6. Njëg li gëna mag ci jëfandikoo dajale gradient mooy:
   • Mémoire GPU bu bari lañuy jëfandikoo ci kaw
   • Ñàkk bépp degrade
   • Wall-clock bu gëna gudd ci yeesal bu am solo
   • Mënul jëfandikoo ab optimisatër
7. A training loop clears gradients after every micro-batch but steps the optimizer every fourth batch. What is lost?
   • The model weights are deleted
   • The earlier micro-batch contributions to the accumulated update
   • The effective batch becomes four times larger
   • The loss automatically becomes zero
8. Why may four micro-batches with BatchNorm behave differently from one physical batch of the same total size?
   • The optimizer cannot accept four backward passes
   • Gradient addition is forbidden in BatchNorm
   • The number of training examples changes
   • BatchNorm uses statistics from each micro-batch rather than the combined batch