Ikibazo gihuza ibibazo · Biragoye Urwego

Lookahead na Ntare Optimizers Ikibazo

Kugenzura Lookahead yihuta kandi itinda, Intare yerekana ibimenyetso bishya byerekana imbaraga, hamwe nibisobanuro byo kwibuka no guhitamo igipimo.

Inzira zijyanye no kuyoboraReba neza na Ntare
Ikibazo 1 Bya 8

Niyihe ntambwe yo kubara intambwe yo kuvugurura ibiro byintare?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Niyihe ntambwe yo kubara intambwe yo kuvugurura ibiro byintare?
    • Gufata ikimenyetso cyijambo rishingiye ku mbaraga
    • Kugabanya icyiciro cya kare cyagereranijwe
    • Kubara icyiciro cya kabiri Hessian igereranya
    • Impuzandengo y'ibipimo hejuru ya bariyeri nyinshi
  2. Nigute Lookahead itunganya uburyo bwiza?
    • Ikoresha umurongo umwe wibiro byavuguruwe kabiri kuri buri ntambwe
    • Igumana uburemere 'bwihuse' buvugururwa nubushakashatsi bwibanze hamwe nuburemere 'buhoro' bugenda buhuza nabo.
    • Ikuraho imbaraga rwose kugirango ubike ububiko
    • Itoza imiyoboro ibiri kandi igereranya ibisubizo byabo
  3. Nigute optimizer ya Ntare yavumbuwe?
    • Mugukoresha intoki bivuye muri convex optimizasiyo
    • Mugereranije ibishya bya SGD na Adam
    • Binyuze mu buryo bwikora bwa porogaramu ishakisha hejuru ya optimizer algorithms
    • Uhereye kuri fiziki yigana ibice bigenda
  4. Ugereranije na Adamu, niyihe nyungu yibanze yo kwibuka Intare?
    • Ntabwo ibika leta na gato
    • Ikoresha kimwe cya kabiri cyibipimo byikora
    • Iragabana leta murwego rwose
    • Igumana umuvuduko umwe gusa aho kuba Adamu ebyiri za leta
  5. Kuki Intare isaba igipimo gito cyo kwiga kurusha Adamu?
    • Kuberako ibikorwa byibimenyetso bituma buri update yubunini buringaniye, igapima neza intambwe
    • Kuberako idakoresha imbaraga
    • Kuberako itunganya ibyiciro binini
    • Kuberako ibuza kubora
  6. Inyungu ifatika ya Lookahead nuko ikunda gukora imyitozo:
    • Kumva cyane guhitamo hyperparameter
    • Birenzeho kandi bikomeye, hamwe no kugabanuka kunyeganyega
    • Buhoro kuri buri ntambwe kubintu icumi
    • Ntibishobora kubangikanya na Adam cyangwa SGD
  7. Lookahead slow weights are 0 and the fast weights end an inner loop at 8. If alpha is 0.25, where do the slow weights move?
    • To 8, regardless of alpha
    • To −2, away from the fast weights
    • To 2, one quarter of the distance toward the fast weights
    • They remain at 0 because alpha only changes Lion
  8. After Lookahead updates its slow weights at synchronization, where do the fast weights restart?
    • At the original random initialization
    • At the final validation predictions
    • At the previous slow weights before interpolation
    • At the updated slow-weight values for the next inner loop