Ikibazo gihuza ibibazo · Biragoye Urwego

Umurongo Witondewe hamwe nuwukora Kernels Ikibazo

Kugenzura softmax kwitondera ibiciro bikurikirana, guhuza gahunda, guhuza FAVOR + kugereranya, ibintu bidasanzwe hamwe numurongo utoroshye.

Inzira zijyanye no kuyoboraImirongo Yitondewe Numukora
Ikibazo 1 Bya 8

Ni ukubera iki softmax isanzwe yitabwaho cyane hamwe n'uburebure bukurikirana?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Ni ukubera iki softmax isanzwe yitabwaho cyane hamwe n'uburebure bukurikirana?
    • Igihe cyacyo nibuka bikura hamwe na kare yuburebure bwikurikiranya
    • Birasaba gutondeka ibimenyetso byose mubisobanuro
    • Irashobora gutunganya ikimenyetso kimwe icyarimwe
    • Irakeneye imiyoboro itandukanye ya neural kuri buri kimenyetso
  2. Ni ubuhe buryo bw'imibare butuma umurongo witondera kwirinda n-by-n matrix?
    • Commutativite yo kwiyongera
    • Kwishyira hamwe kwa kugwiza matrix
    • Amategeko yo gukwirakwiza software
    • Kwangirika kwa Eigenvalue
  3. Ni ubuhe buryo bwa FAVOR + bukora neza?
    • Ivugurura rya Gradient
    • Ikimenyetso cyamagambo
    • Softmax kwitondera intangiriro
    • Imibare isanzwe
  4. Kuki Performer ikoresha ibintu byiza bidasanzwe aho gukoresha trigonometric yambere?
    • Ziruka vuba kuri CPU gusa
    • Bakuraho ibikenewe byose
    • Bituma icyitegererezo gisobanurwa neza
    • Bemeza uburemere butari bwiza kandi butezimbere imibare
  5. Ni ubuhe buryo bugereranijwe bwa Performer-stil umurongo utondekanya muburyo bukurikiranye n?
    • O (n)
    • O (n ^ 2)
    • O (n log n)
    • O (2 ^ n)
  6. Ni ubuhe butumwa busanzwe bwo kwitonda kumurongo ugereranije na softmax yuzuye?
    • Ntishobora gukora kuri GPUs
    • Akenshi igera ku gipimo gito cyo kwerekana imiterere
    • Ikora gusa kumashusho
    • Irasaba amakuru yanditse kuri buri kimenyetso
  7. For attention written as phi(Q)(phi(K)^T V), which multiplication can be done first to avoid creating an n-by-n matrix?
    • Compute QK^T and store every pairwise score
    • Apply softmax to the full n-by-n matrix first
    • Compute phi(K)^T V before multiplying by phi(Q)
    • Multiply the final output by the sequence length
  8. What benefit does using orthogonal random features provide in Performer’s FAVOR+ approximation?
    • It guarantees exact softmax for every sequence
    • It removes the need for query and key vectors
    • It makes attention quadratic in sequence length
    • It reduces variance in the random-feature estimate