Positional Interpolation for Long Context Quiz

Answers and explanations appear as you take the quiz.

1. What is the core idea behind Positional Interpolation?
   • Squeezing new position indices into the range the model was already trained on
   • Training a brand-new model from scratch for each length
   • Removing positional information entirely
   • Sorting tokens by importance
2. Why does naive extrapolation to longer positions fail?
   • It uses too little memory
   • It can blow up attention scores to extremely large, unstable values
   • It always returns the same token
   • It requires a different tokenizer
3. Roughly how much fine-tuning did the original PI work require to extend LLaMA to 32K?
   • No fine-tuning at all
   • About 1 million steps
   • About 1,000 steps
   • A full pretraining run
4. If you extend context by a factor s, how does PI transform a position m?
   • It maps m to m times s
   • It maps m to m squared
   • It leaves m unchanged
   • It maps m to m divided by s
5. How did PI influence later methods like YaRN?
   • It became the baseline that frequency-aware methods improved upon
   • It was abandoned and never cited
   • It replaced the Transformer architecture
   • It only applies to image models
6. Which position encoding does Positional Interpolation primarily target?
   • One-hot encodings
   • RoPE (Rotary Position Embeddings)
   • Word2Vec vectors
   • Bag-of-words counts
7. A model trained to position 2K tokens is extended to 32K using PI. What is the extension factor s?
   • 2
   • 8
   • 16
   • 32
8. With extension factor s = 16, where does PI map original position m = 16,000?
   • Position 16,000
   • Position 256,000
   • Position 32,000
   • Position 1,000