Idanwo ti o sopọ mọ itọsọna · Alabọde Ipele

Idanwo Iṣeto Iṣeto Ikẹkọ

Checks warmup, cosine and step schedules, restart behavior, early instability and the common warmup-then-decay training recipe.

Jẹmọ itọsọna awọn ọnaIṣeto Oṣuwọn Ẹkọ
Ibeere 1 ti 8

Kini idi ti alakoso 'gbona' ni iṣeto oṣuwọn ẹkọ?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini idi ti alakoso 'gbona' ni iṣeto oṣuwọn ẹkọ?
    • Lati bẹrẹ pẹlu iwọn kekere ati rampu soke, yago fun aisedeede lati awọn gradients alariwo ni kutukutu
    • Lati jẹ ki išedede ikẹhin dinku
    • Lati fo awọn ipele akọkọ ti data
    • Lati di gbogbo awọn iwuwo ni ibẹrẹ
  2. Iṣeto wo ni laisiyonu tẹle ọna-idaji-cosine si isalẹ si iwọn ti o kere ju?
    • Ibakan iṣeto
    • Cosine annealing
    • ID iṣeto
    • Ilọsoke laini
  3. Kini yoo ṣẹlẹ ti oṣuwọn ikẹkọ ba ti ṣeto ga julọ?
    • Ikẹkọ kojọpọ lẹsẹkẹsẹ pẹlu deede pipe
    • Awọn awoṣe reluwe losokepupo sugbon nigbagbogbo aseyori
    • Pipadanu le yato tabi oscillate dipo idinku
    • Ko si ohun ti o yipada; oṣuwọn ko ni ipa
  4. Kini ibajẹ igbesẹ ṣe si oṣuwọn ẹkọ?
    • Ṣe alekun rẹ laini gbogbo ipele
    • Ntọju o ni pipe ibakan
    • Ṣeto rẹ laileto ni igbesẹ kọọkan
    • Ju silẹ nipasẹ ifosiwewe ti o wa titi ni awọn akoko ti a ti sọ tẹlẹ
  5. What does a warm restart do to a learning-rate schedule?
    • It raises the learning rate again to begin another schedule cycle
    • It resets the model parameters to random values
    • It permanently fixes the learning rate at zero
    • It deletes the optimizer’s moment estimates
  6. In Goyal et al.’s large-minibatch ImageNet setup, what did their warmup scheme address?
    • It removed the need for a validation set
    • It eased early optimization difficulties after the learning-rate scale increased with batch size
    • It changed the model architecture after each epoch
    • It converted the learning-rate curve into a cosine wave
  7. In the cosine formula, what does T denote?
    • The number of model layers
    • Iwọn ipele naa
    • The schedule horizon or total number of steps used for the decay
    • The optimizer’s momentum coefficient
  8. If the cosine schedule uses lr_min greater than zero, where does the rate approach at the end of the horizon?
    • Exactly zero regardless of lr_min
    • The original warmup starting rate
    • An unbounded value
    • The specified minimum learning rate