Idanwo ti o sopọ mọ itọsọna · Lile Ipele

State Space Models ati Mamba adanwo

Test state-space sequence complexity, Mamba selectivity, hardware-aware scan, recurrent inference state, and hybrid models.

Jẹmọ itọsọna awọn ọnaState Space Models Ati Mamba
Ibeere 1 ti 8

Bawo ni iwọn awoṣe aaye aaye kan ṣe pẹlu gigun ọkọọkan, ni akawe si akiyesi ara ẹni boṣewa?

Awotẹlẹ gbogbo ibeere8 ibeere

Awọn idahun ati awọn alaye han bi o ṣe n mu ibeere naa.

  1. Bawo ni iwọn awoṣe aaye aaye kan ṣe pẹlu gigun ọkọọkan, ni akawe si akiyesi ara ẹni boṣewa?
    • Laini ila, dipo akiyesi igbelowọn kuadiratiki
    • Ni ẹẹmẹrin, dipo igbewọn laini akiyesi
    • Mejeeji asekale identically
    • Exponentally fun awọn mejeeji
  2. Kini isọdọtun aarin ti Mamba ṣafikun si awọn SSM ti iṣeto tẹlẹ bii S4?
    • Lilo softmax akiyesi inu
    • Ṣiṣe awọn paramita imudojuiwọn-ipinlẹ da lori titẹ sii (aṣayan)
    • Yiyọ awọn farasin ipinle patapata
    • Yipada si iṣiro-akoko kuadiratiki
  3. Kini idi ti Mamba nilo ọlọjẹ ohun elo ti o mọ ni afiwe?
    • Nitori awọn GPU ko le ṣe isodipupo matrix
    • Lati din awọn nọmba ti fẹlẹfẹlẹ
    • Nitori ṣiṣe awọn igbewọle-igbẹkẹle awọn paramita fi opin si ẹtan convolution ti o rọrun
    • Nitoripe ko lo awọn aye ti a kọ ẹkọ
  4. Lakoko itọkasi, anfani ṣiṣe bọtini kan ti SSM bii Mamba ni pe o nlo:
    • Iranti ti o dagba pẹlu ipari gigun ni kikun, bii kaṣe KV kan
    • Ko si iranti rara
    • Iranti diẹ sii ju Amunawa ti iwọn dogba
    • Aijọju ibakan iranti fun ipilẹṣẹ àmi
  5. Ni awọn SSMs ti a ṣe agbekalẹ aṣaaju (ṣaaju yiyan), atunwi naa le ṣe iṣiro daradara nitori pe o le tun kọ bi:
    • Long convolution
    • Softmax isẹ
    • Igi ipinnu
    • igbo ID
  6. Which language-modeling corpus did the Mamba paper use in its standard autoregressive scaling experiments?
    • ImageNet-1K
    • Pile naa
    • LibriSpeech only
    • D4RL
  7. In the Mamba block, what operation combines the selective-SSM output with the other projected branch?
    • A nearest-neighbor lookup over audio frames
    • A cross-attention layer over image patches
    • An elementwise multiplicative interaction with a gated branch
    • A graph-degree normalization step
  8. What language-modeling result does the Mamba paper report for its 3B model?
    • It requires a Transformer cache twice as large
    • It replaces the training corpus with audio only
    • It can be trained without any data
    • It matches Transformers twice its size in pretraining and downstream evaluation