Ikibazo gihuza ibibazo · Biragoye Urwego

Itondere Rollout hamwe n'ikibazo cyo gutema umutwe

Gerageza uburyo umuzingo uhuza ibitekerezo hamwe n'inzira zisigaye, icyo gutema umutwe ugereranya, nicyo ayo manota ashobora kandi adashobora gusobanura.

Inzira zijyanye no kuyoboraItondere Kuzunguruka no Gukata Umutwe
Ikibazo 1 Bya 8

Niyihe ntego yo kwitabwaho?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Niyihe ntego yo kwitabwaho?
    • Kugirango ukurikirane amakuru yuzuye aturuka kumurongo winjiza kugeza ibisubizo kurwego
    • Kongera ibice byinshi kuri Transformer
    • Kongera igipimo cyo kwiga
    • Guhindura ibitekerezo mubyemezo
  2. Ni ukubera iki ikarita imwe yo kwitondera ikarita idahagije kubisobanuro?
    • Icyitonderwa ntigishobora kubarwa
    • Amakuru avanga mubice byinshi numutwe, ikarita imwe rero ibura inzira yuzuye
    • Ikarita ihora ari imwe
    • Abahindura bafite umutwe umwe gusa
  3. Ni ubuhe buryo bwo kwitondera bwiyongera kuri buri matrike yo kwitondera kugirango uhuze ibisigisigi?
    • Kubogama kwa zeru
    • Ijambo ry'urusaku rudasanzwe
    • Indangamuntu ya matrix mbere yo gukora ibisanzwe
    • Porogaramu yongeyeho
  4. Ni ubuhe bushakashatsi bwakozwe ku mitwe myinshi bwagaragaje iki ku bijyanye no kugabanuka k'umutwe?
    • Umutwe wose ni ngombwa
    • Umutwe ntushobora kuvaho
    • Gukata buri gihe gukuba kabiri
    • Imitwe myinshi irashobora gutemwa hamwe no gutakaza neza
  5. Nigute akamaro k'umutwe gakunze kugereranywa no gutema?
    • Binyuze kuri gradient-ishingiye kuri sensibilité yo gutakaza umutwe wa mask
    • Ukurikije imyandikire yacyo
    • Kubara ibipimo byayo gusa
    • Ku manywa
  6. What different jobs do attention rollout and head pruning each do?
    • Nubuhanga bumwe
    • Rollout ifasha gusobanura mugihe gutema ikoresha ubudahangarwa kugirango ugabanye moderi
    • Byombi ongeraho ibipimo gusa
    • Nta nubwo bikora ku mutima
  7. In an attention-rollout calculation, why are residual paths included with layer attention?
    • To make every head have the same weight
    • To remove the value vectors from the Transformer
    • To approximate information that can pass along the identity path when layer transitions are composed
    • To prevent the attention matrix from being normalized
  8. How should an attention-rollout score be interpreted?
    • As proof that a token caused the output
    • As the exact value vector used at every layer
    • As a measure of how many heads the model contains
    • As a post-hoc approximation of relative token influence, not a causal explanation