Guide-yakabatana mibvunzo · Pakati Nhanho

Sparse Attention Patterns Quiz

Enzanisa hukama hwemunharaunda, hwakanyungudutswa, husina tsarukano, uye hwepasirese hushoma hwekutarisisa uye kuti kernels dzakakwenenzverwa dzinosvetuka sei zvinyorwa zvisingashandiswe mukugadziriswa-refu-yekuisa.

Nzira dzekutungamira dzinoenderanaSparse Attention Patterns
Mubvunzo 1 ye 8

Sei kuzvitarisisa kuzere kuchidhura pakutevedzana kwakareba?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Sei kuzvitarisisa kuzere kuchidhura pakutevedzana kwakareba?
    • Mutengo wayo unokura neskweya yekutevedzana kureba
    • Haikwanise kushandisa maGPU
    • Inoshanda chete pamashoko mapfupi
    • Zvinoda kusarangarira zvachose
  2. Chii chinonzi sliding-window (yenzvimbo) kutarisa?
    • Chiratidzo chega chega chinotarisa kune isina kurongeka dhatabhesi
    • Tokeni imwe neimwe inongobatsira vavakidzani vayo vari pedyo chete
    • Tokeni haamboiti chero chinhu
    • Chiratidzo chega chega chinochengeta tokeni dzese kaviri
  3. Chii chinangwa che 'global tokens' mukutarisisa kushoma?
    • Kuderedza modhi pasi
    • Kuchengeta mifananidzo
    • Kuita senzvimbo dzinotarisira kune zvese uye izvo zvese zvinoteera
    • Kutsiva mazwi
  4. Ndeipi modhi yakaratidza kuti kusanganisa zvisingaite, hwindo, uye kutarisisa kwepasirese kunochengeta kuzere kutarisisa kutaura?
    • ResNet
    • Word2Vec
    • AlexNet
    • BigBird
  5. Zvichida kuti nhamba yekubatanidza inokwira sei mukutarisisa kwakagadzirwa zvakanaka?
    • Zvinenge mutsara nehurefu hwekutevedzana
    • Quadratically nehurefu hwekutevedzana
    • Hazvinei nekureba
    • Exponentially nehurefu hwekutevedzana
  6. What do strided or dilated sparse-attention patterns allow?
    • Anoaverenga obva aaisa ku zero
    • Skip ahead to reach distant context with fewer attention links
    • Zvitumire kune imwe sevha
    • Anozvichengeta zvachose
  7. How does Longformer combine local and global attention?
    • Every token attends to every other token
    • Only random edges are allowed
    • Most tokens use a sliding local window, while selected global tokens connect across the sequence
    • Each token attends only to the final token
  8. What is the goal of learned, content-based sparse attention?
    • A full dense connection to every token
    • A rule for removing all future tokens
    • A learned vocabulary for each attention head
    • Avoid spending attention compute and memory on content unrelated to the current query