quiz bu lëkkale ak tegtal · Dëgër Niveau

Lineer ak Quiz Kernels Performer

Saytu njëgu toppalante softmax, yamalewaat bu boole, jegewaale kernel FAVOR +, man-mani yu bari ak jafe-jafe ligneer.

Yooni njiit yu jëm ciLinear ak Kernels Performer
Laaj 1 de 8

Lan moo waral softmax biñ miin di bàyyi xel ci guddaayig toppalante bi?

Xoolaatal laaj bu nekk8 laaj

Sooy def quiz bi, tontu yi ak leeral yi dañuy feeñ.

  1. Lan moo waral softmax biñ miin di bàyyi xel ci guddaayig toppalante bi?
    • Jamonoy ak fàttaliku dañuy màgg ak kaare guddaayu toppalante bi
    • Dafay laaj nga tànnal jeton yépp ci alfabe
    • Benn token kese lay mëna jëfandikoo benn yoon
    • Dafa soxla reso neuronal bu wuute ci jeton bu nekk
  2. Ban màndarga math mooy may bàyyi xel ci ligneer mu moytu matrix n-by-n?
    • Kommutatifu yokk
    • Associativité bu matrix
    • Yoon séddale bu softmax
    • Dekomposisioŋu valeur boppam
  3. Lan mooy mekanismu FAVOR+ bu Performer bi?
    • coppite ci wàcci degrade
    • Baatu tokenisatër bi
    • Kernel biy bàyyi xel ci softmax
    • lim ak xayma ci normalisasioŋ
  4. Lan moo waral Performer di jëfandikoo màndarga yu baax yu bari te baña jëfandikoo màndarga trigonometrik yu njëkk ya?
    • Dañu gëna gaaw ci CPU yi kese
    • Dañu dindi bépp njariñ bu bari
    • Dañuy tax ñu mëna tekki xeetu model bi
    • Dañuy garanti poids attention bu amul negatif te gëna yombal stabilite numérique
  5. Lan mooy xayma jafe-jafe yi ci stil Performer ci guddaayig toppalante n?
    • O(n)
    • O(n^2)
    • O(n log n)
    • O(2^n)
  6. Lan mooy yam ci bàyyi xel ci lineer bu sell buñu ko méngale ak bàyyi xel ci softmax bu mat sëkk?
    • Mënul dox ci GPU yi
    • Dafay faral di am kalite modeling bu gëna néew tuuti
    • Nataal kese lay dox
    • Dafay laaj done yuñ etikete ngir token bu nekk
  7. For attention written as phi(Q)(phi(K)^T V), which multiplication can be done first to avoid creating an n-by-n matrix?
    • Compute QK^T and store every pairwise score
    • Apply softmax to the full n-by-n matrix first
    • Compute phi(K)^T V before multiplying by phi(Q)
    • Multiply the final output by the sequence length
  8. What benefit does using orthogonal random features provide in Performer’s FAVOR+ approximation?
    • It guarantees exact softmax for every sequence
    • It removes the need for query and key vectors
    • It makes attention quadratic in sequence length
    • It reduces variance in the random-feature estimate