Ikibazo gihuza ibibazo · Biragoye Urwego
Umurongo Witondewe hamwe nuwukora Kernels Ikibazo
Kugenzura softmax kwitondera ibiciro bikurikirana, guhuza gahunda, guhuza FAVOR + kugereranya, ibintu bidasanzwe hamwe numurongo utoroshye.
Ikibazo 1 Bya 8
Ni ukubera iki softmax isanzwe yitabwaho cyane hamwe n'uburebure bukurikirana?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Ni ukubera iki softmax isanzwe yitabwaho cyane hamwe n'uburebure bukurikirana?
- Igihe cyacyo nibuka bikura hamwe na kare yuburebure bwikurikiranya
- Birasaba gutondeka ibimenyetso byose mubisobanuro
- Irashobora gutunganya ikimenyetso kimwe icyarimwe
- Irakeneye imiyoboro itandukanye ya neural kuri buri kimenyetso
- Ni ubuhe buryo bw'imibare butuma umurongo witondera kwirinda n-by-n matrix?
- Commutativite yo kwiyongera
- Kwishyira hamwe kwa kugwiza matrix
- Amategeko yo gukwirakwiza software
- Kwangirika kwa Eigenvalue
- Ni ubuhe buryo bwa FAVOR + bukora neza?
- Ivugurura rya Gradient
- Ikimenyetso cyamagambo
- Softmax kwitondera intangiriro
- Imibare isanzwe
- Kuki Performer ikoresha ibintu byiza bidasanzwe aho gukoresha trigonometric yambere?
- Ziruka vuba kuri CPU gusa
- Bakuraho ibikenewe byose
- Bituma icyitegererezo gisobanurwa neza
- Bemeza uburemere butari bwiza kandi butezimbere imibare
- Ni ubuhe buryo bugereranijwe bwa Performer-stil umurongo utondekanya muburyo bukurikiranye n?
- O (n)
- O (n ^ 2)
- O (n log n)
- O (2 ^ n)
- Ni ubuhe butumwa busanzwe bwo kwitonda kumurongo ugereranije na softmax yuzuye?
- Ntishobora gukora kuri GPUs
- Akenshi igera ku gipimo gito cyo kwerekana imiterere
- Ikora gusa kumashusho
- Irasaba amakuru yanditse kuri buri kimenyetso
- For attention written as phi(Q)(phi(K)^T V), which multiplication can be done first to avoid creating an n-by-n matrix?
- Compute QK^T and store every pairwise score
- Apply softmax to the full n-by-n matrix first
- Compute phi(K)^T V before multiplying by phi(Q)
- Multiply the final output by the sequence length
- What benefit does using orthogonal random features provide in Performer’s FAVOR+ approximation?
- It guarantees exact softmax for every sequence
- It removes the need for query and key vectors
- It makes attention quadratic in sequence length
- It reduces variance in the random-feature estimate