Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Adam ati Adaptive Optimizers adanwo

Checks Adam’s gradient moments, bias correction, adaptive scaling, epsilon and AdamW’s decoupled weight-decay update. in the standard parameter update.

Jẹmọ itọsọna awọn ọnaAdam Ati Adaptive Optimizers
Ibeere 1 ti 8

Awọn iwọn meji wo ni Adam tọpinpin fun paramita kọọkan?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Awọn iwọn meji wo ni Adam tọpinpin fun paramita kọọkan?
    • Iwọn ti nṣiṣẹ ti awọn gradients ati ti awọn gradients onigun mẹrin
    • Iwọn ti o pọju ati iwọn kekere ti a ti ri tẹlẹ
    • Oṣuwọn ẹkọ ati iwọn ipele
    • Iwọn iwuwo ati ami rẹ
  2. Kini idi ti Adam fi lo atunṣe aibikita si awọn iṣiro akoko rẹ?
    • Nitori awọn gradients nigbagbogbo jẹ odi ni kutukutu ikẹkọ
    • Nitoripe awọn iwọn gbigbe bẹrẹ ni odo ati pe o jẹ abosi si odo lakoko
    • Lati jẹ ki oṣuwọn ikẹkọ tobi ju akoko lọ
    • Lati yọ awọn aaye data ita kuro ninu ipele naa
  3. Kini iyatọ akọkọ laarin Adam ati AdamW?
    • AdamW ko lo ipa kankan rara
    • AdamW yọ awọn keji akoko ti siro
    • AdamW decouples idibajẹ iwuwo lati imudojuiwọn-orisun gradient
    • AdamW ṣiṣẹ nikan lori awọn nẹtiwọọki iṣọtẹ
  4. Kini ipa wo ni ọrọ epsilon kekere ṣe ninu ofin imudojuiwọn Adam?
    • O ṣeto agbara ipa
    • O nṣakoso nọmba awọn akoko ikẹkọ
    • O mu iwọn iwọn didun pọ si
    • O ṣe idilọwọ pipin nipasẹ odo ati ṣe imudojuiwọn imudojuiwọn
  5. Kí nìdí tí a fi sábà máa ń ṣàpèjúwe Ádámù gẹ́gẹ́ bí ‘amúparadà’?
    • O fun ni imunadoko ni paramita kọọkan iwọn igbesẹ ti iwọn tirẹ
    • O yipada faaji nẹtiwọọki lakoko ikẹkọ
    • O adapts awọn dataset to awọn awoṣe
    • O yipada laarin isọdi ati ipadasẹhin laifọwọyi
  6. Which Adam parameter controls the exponential decay rate of its squared-gradient estimate?
    • Beta1
    • Beta2
    • Epsilon
    • The weight-decay coefficient
  7. A parameter repeatedly receives large gradients. Which Adam quantity helps scale down its effective step?
    • The number of model layers
    • The batch-normalization mean
    • The second-moment estimate of squared gradients
    • The AdamW weight-decay term alone
  8. What happens when beta2 is set closer to one, all else equal?
    • The optimizer stops using gradients
    • The first-moment estimate is deleted
    • Weight decay becomes decoupled automatically
    • The squared-gradient estimate decays more slowly and retains a longer history