Ikibazo gihuza ibibazo · Biragoye Urwego
Icyitegererezo cya Leta Umwanya na Mamba Ikibazo
Gerageza leta-umwanya ukurikirana bigoye, guhitamo Mamba, ibyuma-bimenyesha scan, gusubiramo inshuro nyinshi, hamwe na moderi ya Hybrid.
Ikibazo 1 Bya 8
Nigute leta yerekana umwanya munini hamwe nuburebure bwikurikiranya, ugereranije no kwiyitaho bisanzwe?
Reba buri kibazo8 ibibazo
Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.
- Nigute leta yerekana umwanya munini hamwe nuburebure bwikurikiranya, ugereranije no kwiyitaho bisanzwe?
- Mu buryo bugaragara, hamwe no kwitondera kwadratike
- Kwikurikiranya, hamwe no kwitondera umurongo ugereranije
- Ibipimo byombi
- Biragaragara kuri byombi
- Ni ubuhe buryo bushya Mamba yongeyeho kuri SSM zubatswe mbere nka S4?
- Gukoresha softmax kwitondera imbere
- Gukora leta-ivugurura ibipimo biterwa ninjiza (guhitamo)
- Kuraho leta yihishe rwose
- Guhindura kuri quadratic-time computing
- Ni ukubera iki Mamba yari akeneye ibyuma byabigenewe-bimenyerewe kubisikana?
- Kuberako GPU idashobora gukora kugwiza matrix
- Kugabanya umubare wibice
- Kuberako gukora ibipimo byinjira-biterwa na tekinike yoroshye yo kwemeza
- Kuberako idakoresha ibipimo byize
- Mugihe cyo gufata umwanzuro, inyungu nyamukuru ya SSM nka Mamba nuko ikoresha:
- Ububiko bukura hamwe nuburebure bwuzuye, nkububiko bwa KV
- Nta kwibuka na gato
- Kwibuka birenze Transformer yubunini bungana
- Hafi yibuka hafi kuri buri kimenyetso cyakozwe
- Muri SSM isanzwe yubatswe (mbere yo guhitamo), isubiramo rishobora kubarwa neza kuko rishobora kwandikwa nka a:
- Kwemeza birebire
- Igikorwa cya Softmax
- Igiti
- Ishyamba risanzwe
- Which language-modeling corpus did the Mamba paper use in its standard autoregressive scaling experiments?
- ImageNet-1K
- Ikirundo
- LibriSpeech only
- D4RL
- In the Mamba block, what operation combines the selective-SSM output with the other projected branch?
- A nearest-neighbor lookup over audio frames
- A cross-attention layer over image patches
- An elementwise multiplicative interaction with a gated branch
- A graph-degree normalization step
- What language-modeling result does the Mamba paper report for its 3B model?
- It requires a Transformer cache twice as large
- It replaces the training corpus with audio only
- It can be trained without any data
- It matches Transformers twice its size in pretraining and downstream evaluation