YaRN and Context Length Extension Quiz

Answers and explanations appear as you take the quiz.

1. What position-encoding method does YaRN modify to extend context?
   • Rotary Position Embeddings (RoPE)
   • Absolute sinusoidal embeddings
   • Learned positional lookup tables
   • Relative bias matrices only
2. What goes wrong when a RoPE model sees sequences longer than its training length?
   • The tokenizer crashes
   • The rotation angles enter unseen ranges and performance collapses
   • The model outputs only zeros
   • Gradients become exactly one
3. How does YaRN treat different RoPE frequency dimensions?
   • It interpolates all dimensions equally
   • It deletes high-frequency dimensions
   • It interpolates low-frequency dimensions while largely preserving high-frequency ones
   • It randomizes every dimension
4. Besides rescaling frequencies, what extra adjustment does YaRN apply?
   • A new tokenizer
   • Doubling the number of layers
   • Removing layer normalization
   • An attention temperature (softmax scaling) adjustment
5. What is a key practical advantage of YaRN over naive interpolation?
   • It reaches long context with far less fine-tuning data and steps
   • It needs no model at all
   • It only works below 512 tokens
   • It eliminates the need for GPUs
6. Why is naive linear interpolation of all RoPE frequencies harmful?
   • It only affects the bias terms
   • It compresses high-frequency dimensions that encode local detail
   • It makes the model larger
   • It changes the vocabulary size
7. In RoPE, which vectors receive position-dependent rotations?
   • Only the value vectors
   • Only the model’s token IDs
   • The query and key vectors
   • The optimizer’s gradient and momentum
8. What does YaRN’s attention-temperature adjustment address when context length is extended?
   • The number of GPU layers
   • The dimensions of the token vocabulary
   • The weight decay coefficient
   • Changes in attention entropy or softmax sharpness caused by the longer context