Long-Context Modeling Quiz

Answers and explanations appear as you take the quiz.

1. What does a model's 'context window' refer to?
   • The maximum number of tokens it can attend to in one pass
   • The screen size of the chat interface
   • The number of users allowed at once
   • How long the model has been trained
2. Why does standard self-attention become expensive for long inputs?
   • It grows linearly and is always cheap
   • Its compute and memory grow quadratically with sequence length
   • It cannot read more than one token
   • It only works on images
3. What is the 'lost in the middle' problem?
   • Models forget their training entirely
   • Models can only read the middle of a document
   • Models recall information at the start and end of a long input better than facts buried in the middle
   • Models translate the middle into another language
4. What does FlashAttention primarily improve?
   • It makes the model output longer text
   • It adds more training data automatically
   • It removes the need for any attention
   • It reduces the memory bottleneck with an IO-aware, tiled computation
5. What role do rotary position embeddings (RoPE) play in long-context models?
   • They encode token positions and, with interpolation, help generalize to longer sequences
   • They delete tokens from the input
   • They translate text between languages
   • They store the model's weights
6. What does prompt-cache research reuse across compatible requests?
   • The final answer regardless of the new query
   • Computed attention state for reusable prompt segments
   • A fixed set of token IDs without model computation
   • The model’s training data
7. What does the RULER benchmark add beyond locating one isolated “needle” fact?
   • Only image classification
   • Audio transcription from waveforms
   • Tasks such as multi-hop tracing and aggregation over long contexts
   • GPU memory allocation benchmarks
8. Why test several task types when measuring a model’s effective context length?
   • Every task uses the same computation
   • The advertised window guarantees all tasks work
   • Task type never affects context use
   • A model may handle simple retrieval yet fail to combine or track information across a long input