Idanwo ikojọpọ Gradient

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

1. Kini ikojọpọ gradient nipataki jẹ ki o ṣe?
   • Reluwe laisi GPU eyikeyi
   • Rekọja iwọle sẹhin
   • Ṣe afiwe iwọn ipele ti o munadoko ti o tobi julọ lori iranti to lopin
   • Imukuro ohun ti o dara ju patapata
2. Pẹlu iwọn kekere 4 ati awọn igbesẹ ikojọpọ 8, iwọn ipele ti o munadoko jẹ:
   • 32
   • 4
   • 8
   • 12
3. With an accumulation factor N, when should optimizer.step() and zero_grad() occur?
   • Lẹhin gbogbo bulọọgi-ipele
   • Kò
   • Ṣaaju ki o to kọja siwaju
   • Call optimizer.step() after N micro-batches, then clear gradients before the next accumulation window
4. Lati jẹ ki oṣuwọn ẹkọ ti o munadoko jẹ deede, pipadanu fun-micro-batch jẹ igbagbogbo:
   • Pipọsi nipasẹ N
   • Pipin nipasẹ nọmba awọn igbesẹ ikojọpọ N
   • Ṣeto si odo
   • Rọpo pẹlu išedede
5. Eyi ti normalization Layer orisii diẹ mọ pẹlu gradient ikojọpọ?
   • Layer tabi deede ẹgbẹ, niwon wọn ko dale lori awọn iṣiro ipele
   • Iṣe deede ipele, nitori pe o nifẹ awọn ipele kekere
   • Ko si normalization ṣiṣẹ pẹlu rẹ
   • Iyọkuro nikan
6. Iye idiyele akọkọ ti lilo ikojọpọ gradient jẹ:
   • Diẹ sii iranti GPU ti a lo ni tente oke
   • Isonu ti gbogbo gradients
   • Akoko aago odi gigun fun imudojuiwọn to munadoko
   • Ailagbara lati lo olupilẹṣẹ
7. A training loop clears gradients after every micro-batch but steps the optimizer every fourth batch. What is lost?
   • The model weights are deleted
   • The earlier micro-batch contributions to the accumulated update
   • The effective batch becomes four times larger
   • The loss automatically becomes zero
8. Why may four micro-batches with BatchNorm behave differently from one physical batch of the same total size?
   • The optimizer cannot accept four backward passes
   • Gradient addition is forbidden in BatchNorm
   • The number of training examples changes
   • BatchNorm uses statistics from each micro-batch rather than the combined batch