Ikibazo gihuza ibibazo · Biragoye Urwego

Icyitegererezo cya Leta Umwanya na Mamba Ikibazo

Gerageza leta-umwanya ukurikirana bigoye, guhitamo Mamba, ibyuma-bimenyesha scan, gusubiramo inshuro nyinshi, hamwe na moderi ya Hybrid.

Inzira zijyanye no kuyoboraIkirere cya Leta Icyitegererezo na Mamba
Ikibazo 1 Bya 8

Nigute leta yerekana umwanya munini hamwe nuburebure bwikurikiranya, ugereranije no kwiyitaho bisanzwe?

Reba buri kibazo8 ibibazo

Ibisubizo nibisobanuro bigaragara mugihe ufata ikibazo.

  1. Nigute leta yerekana umwanya munini hamwe nuburebure bwikurikiranya, ugereranije no kwiyitaho bisanzwe?
    • Mu buryo bugaragara, hamwe no kwitondera kwadratike
    • Kwikurikiranya, hamwe no kwitondera umurongo ugereranije
    • Ibipimo byombi
    • Biragaragara kuri byombi
  2. Ni ubuhe buryo bushya Mamba yongeyeho kuri SSM zubatswe mbere nka S4?
    • Gukoresha softmax kwitondera imbere
    • Gukora leta-ivugurura ibipimo biterwa ninjiza (guhitamo)
    • Kuraho leta yihishe rwose
    • Guhindura kuri quadratic-time computing
  3. Ni ukubera iki Mamba yari akeneye ibyuma byabigenewe-bimenyerewe kubisikana?
    • Kuberako GPU idashobora gukora kugwiza matrix
    • Kugabanya umubare wibice
    • Kuberako gukora ibipimo byinjira-biterwa na tekinike yoroshye yo kwemeza
    • Kuberako idakoresha ibipimo byize
  4. Mugihe cyo gufata umwanzuro, inyungu nyamukuru ya SSM nka Mamba nuko ikoresha:
    • Ububiko bukura hamwe nuburebure bwuzuye, nkububiko bwa KV
    • Nta kwibuka na gato
    • Kwibuka birenze Transformer yubunini bungana
    • Hafi yibuka hafi kuri buri kimenyetso cyakozwe
  5. Muri SSM isanzwe yubatswe (mbere yo guhitamo), isubiramo rishobora kubarwa neza kuko rishobora kwandikwa nka a:
    • Kwemeza birebire
    • Igikorwa cya Softmax
    • Igiti
    • Ishyamba risanzwe
  6. Which language-modeling corpus did the Mamba paper use in its standard autoregressive scaling experiments?
    • ImageNet-1K
    • Ikirundo
    • LibriSpeech only
    • D4RL
  7. In the Mamba block, what operation combines the selective-SSM output with the other projected branch?
    • A nearest-neighbor lookup over audio frames
    • A cross-attention layer over image patches
    • An elementwise multiplicative interaction with a gated branch
    • A graph-degree normalization step
  8. What language-modeling result does the Mamba paper report for its 3B model?
    • It requires a Transformer cache twice as large
    • It replaces the training corpus with audio only
    • It can be trained without any data
    • It matches Transformers twice its size in pretraining and downstream evaluation