Guide-yakabatana mibvunzo · Zvakaoma Nhanho

Linear Kuteerera uye Muiti Kernels Quiz

Inotarisa kuteedzera mutengo weSoftmax, kurongekazve, FAVOR + kernel fungidziro, zvimiro zvisina kurongeka uye kuomarara kwemutsara.

Nzira dzekutungamira dzinoenderanaLinear Attention Uye Muiti Kernels
Mubvunzo 1 ye 8

Nei yakajairwa softmax kutarisisa ichikwira zvisina kunaka nehurefu hwekutevedzana?

Tarisa mubvunzo wese8 mibvunzo

Mhinduro uye tsananguro zvinoonekwa paunenge uchitora bvunzo.

  1. Nei yakajairwa softmax kutarisisa ichikwira zvisina kunaka nehurefu hwekutevedzana?
    • Nguva yaro uye ndangariro zvinokura nechikwere chehurefu hwekutevedzana
    • Zvinoda kurongedza tokeni zvese nealfabheti
    • Inogona chete kugadzirisa chiratidzo chimwe panguva
    • Inoda yakaparadzana neural network pane tokeni
  2. Ndeipi pfuma yemasvomhu inoita kuti mutsara utarise kudzivirira iyo n-by-n matrix?
    • Commutativity yekuwedzera
    • Kubatanidzwa kwematrix kuwanda
    • Mutemo wekugovera weSoftmax
    • Eigenvalue decomposition
  3. Chii chinonzi Performer's FAVOR+ maitiro anokwana?
    • Gradient descent inogadziridza
    • Izwi rekuti tokenizer
    • Iyo softmax yekutarisisa kernel
    • Layer normalization statistics
  4. Sei Muiti anoshandisa zvakanaka zvisina mwero maficha kwete ekutanga trigonometric?
    • Vanomhanya nekukurumidza paCPUs chete
    • Ivo kubvisa kudiwa chero randomness
    • Vanoita kuti muenzaniso wacho unyatsodudzirwa
    • Vanovimbisa huremu husina-negative uye vanovandudza kugadzikana kwenhamba
  5. Ndeipi fungidziro yakaoma yePerformer-style mutsara kutarisa mukutevedzana kureba n?
    • O(n)
    • O(n^2)
    • O(n log n)
    • O(2^n)
  6. Ndechipi chidziviso chakajairwa chekutarisa kwakachena kwemutsara uchienzaniswa neakazara softmax kutarisisa?
    • Haikwanise kumhanya paGPUs
    • Iyo inowanzoita yakaderera zvishoma yekuenzanisira mhando
    • Inoshanda chete kumifananidzo
    • Inoda data rakanyorwa kune yega tokeni
  7. For attention written as phi(Q)(phi(K)^T V), which multiplication can be done first to avoid creating an n-by-n matrix?
    • Compute QK^T and store every pairwise score
    • Apply softmax to the full n-by-n matrix first
    • Compute phi(K)^T V before multiplying by phi(Q)
    • Multiply the final output by the sequence length
  8. What benefit does using orthogonal random features provide in Performer’s FAVOR+ approximation?
    • It guarantees exact softmax for every sequence
    • It removes the need for query and key vectors
    • It makes attention quadratic in sequence length
    • It reduces variance in the random-feature estimate