quiz bu lëkkale ak tegtal · Dëgër Niveau

Quiz ngir bàyyi xel ci Flash

Test FlashAttention's IO-aware tiling, softmax ci net bi, bàyyi xel bu jaar yoon, escalier bu toppalante, ak wuute gi nekk ci diggante trafik mémoire ak calcul.

Yooni njiit yu jëm ciFlash Fexe
Laaj 1 de 8

Lan mooy jafe-jafe bi gëna mag bi Flash Attention di wut?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan mooy jafe-jafe bi gëna mag bi Flash Attention di wut?
    • Liggéey ak bind matrix bàyyi xel bu rëy ngir yeexal mémoire GPU
    • Limu bariwaayu poñ flottër yi ci bàyyi xel
    • Njëg li ñuy jëfandikoo ngir yab model poids ci disk bi
    • Latency reso ci digganté GPU yu bari
  2. naka la Flash Attention di moytu denc matrix N-by-N bu mat sëkk?
    • Dafay xayma bàyyi xel ci ay projection yu bari
    • Dafay liggéey ci laaj yi, caabi yi ak valeur yi ci ay carreaux yu ndaw ba noppi boole ay resultaa yu néew
    • Dafay salte ñaari token yu bari
    • Dafay komprime matrix bi ak kantite
  3. Ban pexe mooy may Flash Attention mu xayma softmax ci anam wu jaar yoon te du gis liiñ bi yépp benn yoon?
    • Tablo seetlu buñ njëkka xayma ci exponentiel yi
    • Dakkal softmax bi te jëfandikoo ap xayma ligneer
    • Softmax buy daw (ci net bi) di topp max ak sum buy daw
    • Tànn poñ yi balaa ñuy normalise
  4. Buñu ko méngale ak bàyyi xel buñ miin, li Flash Attention di génne mooy:
    • Xayma bu gàtt buy wecci njub ak gaawaay
    • Dafa gëna dëggu ndax gëna dëggu
    • Du am solo ci sekans yu gàtt
    • ci wàllu math dañuy nuru, waaye dañu gëna xarañ ci xayma
  5. Lan moo waral Flash Attention di gëna am njariñ ci toppalante yu gudd yi?
    • Ndax fàttaliku bàyyi xel dafay màgg quadratiquement ak guddaayu toppalante
    • Ndax sequence yu gëna gudd yi ñoo gëna néew ay couche
    • Ndax softmax nekkul lu am njariñ ci dugal yu gudd
    • Ndax GPU yi ñoo gëna yeex ci toppalante yu gàtt
  6. Lan mooy 'IO-xam' jëmmal Flash Attention njëkka wàññi?
    • Limu paramet model yi
    • Memoire bi dafay jàng ak bind diggante SRAM ak HBM
    • xóotaayu reso neuronal bi
    • Dayo vocabulaire bu tokenizer bi
  7. What remains computationally quadratic for dense exact self-attention even when FlashAttention reduces memory traffic?
    • The number of model layers must double
    • Every token is compared with only one other token
    • The number of query-key pair interactions grows with the square of sequence length
    • The number of model parameters becomes zero
  8. Why can FlashAttention’s results differ slightly from a reference implementation despite computing the same mathematical attention?
    • It intentionally drops all long-distance tokens
    • It changes the model weights during inference
    • It replaces softmax with a random sample
    • Floating-point operations can occur in a different order during tiled accumulation