Maswali yanayounganishwa na mwongozo · Kati Kiwango

Maswali ya Miundo ya Umakini ya Sparse

Linganisha miunganisho ya kawaida, iliyopanuliwa, nasibu, na ya kimataifa ya tahadhari na jinsi punje zilizoboreshwa zinavyoruka maingizo ambayo hayajatumika katika kuchakata pembejeo kwa muda mrefu.

Njia za mwongozo zinazohusianaMiundo ya Usikivu Sparse
Swali 1 ya 8

Kwa nini umakini kamili wa kibinafsi ni ghali kwenye mlolongo mrefu?

Hakiki kila swali8 maswali

Majibu na maelezo huonekana unapojibu maswali.

  1. Kwa nini umakini kamili wa kibinafsi ni ghali kwenye mlolongo mrefu?
    • Gharama yake inakua na mraba wa urefu wa mlolongo
    • Haiwezi kutumia GPU
    • Inafanya kazi kwa maneno mafupi tu
    • Haihitaji kumbukumbu hata kidogo
  2. Uangalifu wa dirisha la kuteleza (ndani) ni nini?
    • Kila ishara huhudhuria hifadhidata isiyo ya kawaida
    • Kila ishara huhudhuria tu majirani zake wa karibu
    • Ishara hazihudhurii chochote
    • Kila ishara inashughulikia tokeni zote mara mbili
  3. Ni nini madhumuni ya 'ishara za kimataifa' kwa uangalifu mdogo?
    • Ili kupunguza kasi ya mfano
    • Ili kuhifadhi picha
    • Kufanya kama vitovu vinavyoshughulikia kila kitu na ambacho kila kitu kinashughulikiwa
    • Ili kuchukua nafasi ya msamiati
  4. Ni kielelezo gani kilithibitisha kuwa kuchanganya usikivu wa nasibu, dirisha, na kimataifa hudumisha umakini kamili?
    • ResNet
    • Neno2Vec
    • AlexNet
    • BigBird
  5. Takriban ni jinsi gani idadi ya miunganisho inakua katika umakini mdogo ulioundwa vizuri?
    • Takriban mstari na urefu wa mfuatano
    • Mara nne na urefu wa mlolongo
    • Haitegemei urefu
    • Kwa kielelezo na urefu wa mlolongo
  6. What do strided or dilated sparse-attention patterns allow?
    • Huzihesabu na kisha kuziweka hadi sifuri
    • Skip ahead to reach distant context with fewer attention links
    • Huzituma kwa seva nyingine
    • Huzihifadhi kwa kudumu
  7. How does Longformer combine local and global attention?
    • Every token attends to every other token
    • Only random edges are allowed
    • Most tokens use a sliding local window, while selected global tokens connect across the sequence
    • Each token attends only to the final token
  8. What is the goal of learned, content-based sparse attention?
    • A full dense connection to every token
    • A rule for removing all future tokens
    • A learned vocabulary for each attention head
    • Avoid spending attention compute and memory on content unrelated to the current query