quiz bu lëkkale ak tegtal · Diggante Niveau

Laaj ak tontu Flash

Saytu ni FlashAttention di wàññi dem bi ak dikk bi ci mémoire bu bari bi ngay xayma bàyyi xel bu dëggu bi, ak ban aparey mooy soppi mébetu FlashAttention-3.

Yooni njiit yu jëm ciFlash focus
Laaj 1 de 8

Lan mooy jafe-jafe bi gëna mag ci FlashAttention ci bàyyi xel buñ miin?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan mooy jafe-jafe bi gëna mag ci FlashAttention ci bàyyi xel buñ miin?
    • Liggéey ak bind matrix bàyyi xel bu rëy ngir yeexal mémoire GPU
    • Limu bariwaayu point flottant
    • Njëg li ñuy fay ngir tokenize mbindu dugal
    • Latency reso ci digganté GPU yi
  2. Ak naka lay méngoo ak génnug FlashAttention ak bàyyi xel buñ miin?
    • Xayma la buy wecci njub ak gaawaay
    • Dafay nuru ci wàllu math (ci dëgg)
    • Du dox ludul ci ay toppalante yu gàtt
    • Dafay soppi diisaayu model bi
  3. Ban mémoire GPU la FlashAttention di jëfandikoo ngir defar ay done ci biir ay carreaux?
    • Memoire bu yaatu waaye yeex (HBM)
    • RAM bu CPU bi
    • SRAM bu ndaw waaye gaaw lool
    • Dencukaay disk
  4. Ban pexe mooy may FlashAttention mu xayma softmax te du gis poñ yépp benn yoon?
    • Dañuy sànni softmax bi yépp
    • Round poñ yépp ci nul
    • Softmax biy waajal ordinatër ci CPU bi
    • Softmax ci net bi (di yokk) ak eskalaasioŋ bu bees
  5. Ndax matrix attention bi yépp musu ñu ko denc ci HBM, jëfandikoo mémoire bi dafay gëna yokk:
    • Lineer ak guddaayu toppalante
    • Quadratiquement ak guddaayu toppalante
    • Exponentiellement ak guddaayu toppalante
    • Guddaayi fi ñuy toppalante mënul aju ci
  6. Which numeric format does FlashAttention-3 use in its Hopper-focused experiments to accelerate attention?
    • Only 64-bit floating point
    • FP8, with techniques to manage its numerical error
    • Integer text tokens
    • No reduced-precision format
  7. What work-partitioning change does FlashAttention-2 introduce to improve GPU parallelism?
    • It stores the complete attention matrix in HBM
    • It replaces attention with convolution
    • It reduces non-matrix-multiplication work and improves parallelization across thread blocks and warps
    • It uses only CPU execution
  8. In FlashAttention-3, what lets data movement overlap with computation?
    • Replacing attention with a lookup table
    • Computing all scores in host memory
    • Disabling the softmax step
    • Asynchronous execution on Hopper hardware