State Space Models and Mamba Quiz

Answers and explanations appear as you take the quiz.

1. How does a state space model scale with sequence length, compared to standard self-attention?
   • Linearly, versus attention's quadratic scaling
   • Quadratically, versus attention's linear scaling
   • Both scale identically
   • Exponentially for both
2. What is the central innovation that Mamba added to prior structured SSMs like S4?
   • Using softmax attention internally
   • Making the state-update parameters depend on the input (selectivity)
   • Removing the hidden state entirely
   • Switching to quadratic-time computation
3. Why did Mamba need a custom hardware-aware parallel scan?
   • Because GPUs cannot do matrix multiplication
   • To reduce the number of layers
   • Because making parameters input-dependent breaks the simple convolution trick
   • Because it uses no learned parameters
4. During inference, a key efficiency advantage of an SSM like Mamba is that it uses:
   • Memory that grows with the full sequence length, like a KV cache
   • No memory at all
   • More memory than a Transformer of equal size
   • Roughly constant memory per generated token
5. In classic structured SSMs (before selectivity), the recurrence can be computed efficiently because it can be rewritten as a:
   • Long convolution
   • Softmax operation
   • Decision tree
   • Random forest
6. Which language-modeling corpus did the Mamba paper use in its standard autoregressive scaling experiments?
   • ImageNet-1K
   • The Pile
   • LibriSpeech only
   • D4RL
7. In the Mamba block, what operation combines the selective-SSM output with the other projected branch?
   • A nearest-neighbor lookup over audio frames
   • A cross-attention layer over image patches
   • An elementwise multiplicative interaction with a gated branch
   • A graph-degree normalization step
8. What language-modeling result does the Mamba paper report for its 3B model?
   • It requires a Transformer cache twice as large
   • It replaces the training corpus with audio only
   • It can be trained without any data
   • It matches Transformers twice its size in pretraining and downstream evaluation