Guide-yakabatana mibvunzo · Pakati Nhanho

Gradient Accumulation Quiz

Edzai micro-batch gradient sums, inoshanda batch size, kurasikirwa kuyera, optimizer nguva, normalization miganho, uye nguva isati yakwana zvikanganiso.

Nzira dzekutungamira dzinoenderanaGradient Accumulation
Mubvunzo 1 ye 8

Chii chinoita kuti kuunganidza gradient kukuregedze iwe kuti uite?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Chii chinoita kuti kuunganidza gradient kukuregedze iwe kuti uite?
    • Chitima pasina chero GPU
    • Svetuka pass yekumashure
    • Tevedzera saizi yakakura inoshanda yebhechi pane shoma ndangariro
    • Bvisa optimizer zvachose
  2. Iine micro-batch saizi 4 uye 8 kuunganidza matanho, iyo inoshanda saizi yebatch ndeiyi:
    • 32
    • 4
    • 8
    • 12
  3. With an accumulation factor N, when should optimizer.step() and zero_grad() occur?
    • Mushure mese micro-batch
    • Never
    • Kusati kwapfuura
    • Call optimizer.step() after N micro-batches, then clear gradients before the next accumulation window
  4. Kuchengeta chiyero chekudzidza chinoshanda chakarurama, iyo per-micro-batch kurasikirwa kunowanzo:
    • Yakawedzerwa naN
    • Yakakamurwa nehuwandu hwematanho ekuunganidza N
    • Gadzirisa ku zero
    • Yakatsiviwa nekururama
  5. Ndeipi normalization layer pairi zvakachena zvakanyanya ne gradient kuunganidza?
    • Layer kana boka normalization, sezvo ivo vasingaenderane nebatch statistics
    • Batch normalization, nekuti inoda mabheti madiki
    • Hapana normalization inoshanda nayo
    • Kungosiya chete
  6. Mutengo mukuru wekushandisa gradient kuunganidza ndewe:
    • Yakawanda GPU ndangariro inoshandiswa pakakwirira
    • Kurasika kwese magradients
    • Yakareba nguva yewachi-wachi pakugadzirisa kunoshanda
    • Kusagona kushandisa optimizer
  7. A training loop clears gradients after every micro-batch but steps the optimizer every fourth batch. What is lost?
    • The model weights are deleted
    • The earlier micro-batch contributions to the accumulated update
    • The effective batch becomes four times larger
    • The loss automatically becomes zero
  8. Why may four micro-batches with BatchNorm behave differently from one physical batch of the same total size?
    • The optimizer cannot accept four backward passes
    • Gradient addition is forbidden in BatchNorm
    • The number of training examples changes
    • BatchNorm uses statistics from each micro-batch rather than the combined batch