Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Mkusanyiko wa Gradient

Jaribu hesabu za viwango vya bechi ndogo, saizi inayofaa ya bechi, kuongeza hasara, muda wa kiboreshaji, vikomo vya kuhalalisha, na hitilafu za kuweka upya mapema.

Njia za mwongozo zinazohusianaMkusanyiko wa Gradient
Swali 1 ya 8

Mkusanyiko wa gradient hukuruhusu kufanya nini kimsingi?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Mkusanyiko wa gradient hukuruhusu kufanya nini kimsingi?
    • Treni bila GPU yoyote
    • Ruka pasi ya nyuma
    • Iga ukubwa bora wa bechi kwenye kumbukumbu ndogo
    • Ondoa kiboreshaji kabisa
  2. Kwa ukubwa wa kundi ndogo 4 na hatua 8 za mkusanyiko, saizi inayofaa ya bechi ni:
    • 32
    • 4
    • 8
    • 12
  3. With an accumulation factor N, when should optimizer.step() and zero_grad() occur?
    • Baada ya kila kundi ndogo
    • Kamwe
    • Kabla ya kupita mbele
    • Call optimizer.step() after N micro-batches, then clear gradients before the next accumulation window
  4. Ili kuweka kiwango bora cha ujifunzaji kuwa sawa, upotezaji wa kila bechi ndogo ni kawaida:
    • Imezidishwa na N
    • Imegawanywa na idadi ya hatua za mkusanyiko N
    • Weka kwa sifuri
    • Imebadilishwa kwa usahihi
  5. Ni safu gani ya urekebishaji inayooanisha kwa usafi zaidi na mkusanyiko wa gradient?
    • Urekebishaji wa safu au kikundi, kwa kuwa hautegemei takwimu za kundi
    • Kurekebisha kwa kundi, kwa sababu inapenda vikundi vidogo
    • Hakuna urekebishaji unaofanya kazi nayo
    • Kuacha tu
  6. Gharama kuu ya kutumia mkusanyiko wa gradient ni:
    • Kumbukumbu zaidi ya GPU inayotumika katika kilele
    • Kupoteza kwa gradients zote
    • Muda mrefu wa saa ya ukutani kwa kila sasisho linalofaa
    • Kutokuwa na uwezo wa kutumia kiboreshaji
  7. A training loop clears gradients after every micro-batch but steps the optimizer every fourth batch. What is lost?
    • The model weights are deleted
    • The earlier micro-batch contributions to the accumulated update
    • The effective batch becomes four times larger
    • The loss automatically becomes zero
  8. Why may four micro-batches with BatchNorm behave differently from one physical batch of the same total size?
    • The optimizer cannot accept four backward passes
    • Gradient addition is forbidden in BatchNorm
    • The number of training examples changes
    • BatchNorm uses statistics from each micro-batch rather than the combined batch