Ikibazo gihuza ibibazo · Hagati Urwego

Ikibazo Cyitonderwa Cyibibazo Ikibazo

Gereranya hafi, kwaguka, gutondekanya, no kwisi yose guhuza ibitekerezo hamwe nuburyo intungamubiri zasimbuwe zisimbuka ibyakoreshejwe mugihe kirekire-cyo gutunganya.

Inzira zijyanye no kuyoboraNtibisanzwe
Ikibazo 1 Bya 8

Kuki kwiyitaho byuzuye bihenze kurikurikirane?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Kuki kwiyitaho byuzuye bihenze kurikurikirane?
    • Igiciro cyacyo gikura hamwe na kare yuburebure bwikurikiranya
    • Ntishobora gukoresha GPU
    • Ikora gusa kumagambo magufi
    • Ntabwo bisaba kwibuka na gato
  2. Niki kunyerera-idirishya (hafi) kwitondera?
    • Buri kimenyetso cyitabira kubikubiyemo
    • Buri kimenyetso cyitabira gusa abaturanyi begeranye
    • Tokens ntiyigera yitabira ikintu na kimwe
    • Buri kimenyetso cyitabira ibimenyetso bibiri kabiri
  3. Niyihe ntego ya 'global tokens' mubitekerezo bidakunze kubaho?
    • Kugabanya icyitegererezo hasi
    • Kubika amashusho
    • Gukora nka hubs yitabira ibintu byose kandi ko byose byitabira
    • Gusimbuza amagambo
  4. Ni ubuhe buryo bwagaragaje ko guhuza ibintu bidasanzwe, idirishya, hamwe nisi yose bikomeza kwitabwaho byuzuye?
    • ResNet
    • Ijambo2Vec
    • AlexNet
    • BigBird
  5. Hafi yukuntu umubare wibihuza wapima muburyo bwateguwe neza?
    • Hafi yumurongo hamwe nuburebure bwikurikiranya
    • Bikurikiranye hamwe n'uburebure bukurikirana
    • Ntabwo biterwa n'uburebure
    • Byerekanwe n'uburebure bukurikirana
  6. What do strided or dilated sparse-attention patterns allow?
    • Kubara hanyuma ukabishyira kuri zeru
    • Skip ahead to reach distant context with fewer attention links
    • Kohereza kubindi seriveri
    • Ubibike burundu
  7. How does Longformer combine local and global attention?
    • Every token attends to every other token
    • Only random edges are allowed
    • Most tokens use a sliding local window, while selected global tokens connect across the sequence
    • Each token attends only to the final token
  8. What is the goal of learned, content-based sparse attention?
    • A full dense connection to every token
    • A rule for removing all future tokens
    • A learned vocabulary for each attention head
    • Avoid spending attention compute and memory on content unrelated to the current query