Idanwo ti o sopọ mọ itọsọna · Lile Ipele

Yiyi Ifarabalẹ ati adanwo Pruning Ori

Tests how rollout combines layered attention with residual paths, what head pruning estimates, and what those scores can and cannot explain.

Jẹmọ itọsọna awọn ọnaIfojusi Rollout Ati Head Pruning
Ibeere 1 ti 8

Kini ibi-afẹde ti yiyọkuro akiyesi?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Kini ibi-afẹde ti yiyọkuro akiyesi?
    • Lati ṣawari ṣiṣan alaye akopọ lati awọn ami titẹ sii si awọn abajade kọja awọn fẹlẹfẹlẹ
    • Lati fi awọn fẹlẹfẹlẹ diẹ sii si Amunawa
    • Lati mu iwọn ẹkọ pọ si
    • Lati se iyipada akiyesi sinu convolution
  2. Kí nìdí ni kan nikan Layer ká akiyesi map igba insufficient fun alaye?
    • Ifarabalẹ kii ṣe iṣiro
    • Alaye dapọ kọja ọpọlọpọ awọn fẹlẹfẹlẹ ati awọn ori, nitorinaa maapu kan padanu ọna kikun
    • Awọn maapu nigbagbogbo jẹ aṣọ
    • Awọn iyipada ni ori kan ṣoṣo
  3. Kini yiyọ ifarabalẹ ṣe afikun si matrix akiyesi kọọkan si akọọlẹ fun awọn isopọ to ku?
    • Iyatọ ti odo
    • A ID ariwo igba
    • Apakan matrix idanimọ kan ṣaaju ṣiṣe deede
    • Ohun afikun softmax
  4. Kini iwadi lori akiyesi ori-pupọ ṣe afihan nipa apọju ori?
    • Gbogbo ori jẹ pataki
    • Awọn ori ko le yọ kuro
    • Pruning nigbagbogbo ilọpo meji deede
    • Ọpọlọpọ awọn ori le ti wa ni pruned pẹlu kekere iyege pipadanu
  5. Bawo ni a ṣe ṣe ifoju pataki ori kan fun gige gige?
    • Nipasẹ ifamọ-orisun gradient ti pipadanu si iboju-ori
    • Nipa ipo ti alfabeti rẹ
    • Nipa kika awọn paramita rẹ nikan
    • Nipa akoko ti ọjọ
  6. What different jobs do attention rollout and head pruning each do?
    • Wọn jẹ awọn ilana kanna
    • Yiyi ṣe iranlọwọ itumọ lakoko ti gige gige lo nilokulo apọju lati rọpọ awọn awoṣe
    • Mejeeji nikan fi paramita
    • Bẹni ko kan akiyesi
  7. In an attention-rollout calculation, why are residual paths included with layer attention?
    • To make every head have the same weight
    • To remove the value vectors from the Transformer
    • To approximate information that can pass along the identity path when layer transitions are composed
    • To prevent the attention matrix from being normalized
  8. How should an attention-rollout score be interpreted?
    • As proof that a token caused the output
    • As the exact value vector used at every layer
    • As a measure of how many heads the model contains
    • As a post-hoc approximation of relative token influence, not a causal explanation