引导式测验 · 中等 等级

上下文扩展测验的位置插值

测试 RoPE 位置压缩、外推、扩展因子、频率感知方法、模型自适应和长上下文资源成本。

相关引导路径用于上下文扩展的位置插值
问题 1 的 8

位置插值的核心思想是什么？

预览每个问题8 问题

当您参加测验时，就会出现答案和解释。

  1. 位置插值的核心思想是什么？
    • 添加更多关注头
    • 重新调整（缩小）位置索引，使它们保持在训练范围内，而不是推断
    • 完全删除位置嵌入
    • 在长序列上从头开始重新训练模型
  2. 原始位置插值工作通过位置索引重新缩放扩展了哪种位置方案？
    • A fixed output-vocabulary lookup table only
    • No positional information of any kind
    • Rotary positional embeddings, or RoPE
    • An image compression format rather than model positions
  3. 为什么未经修改的位置外推法会在更长的输入上降低 RoPE 模型的性能？
    • The sequence introduces positional and attention conditions beyond the model's training experience
    • A longer context automatically retrains every parameter
    • RoPE removes all attention from the model
    • The tokenizer necessarily loses its vocabulary at a specific length
  4. 如果将上下文长度从 L 扩展到 L'，基本 PI 将什么重新缩放因子应用于位置 m？
    • m 乘以 L'/L
    • 将 L' 添加到 m
    • 将 m 设置为零
    • m 除以 L'/L（即 m / (L'/L)）
  5. YaRN 的频率感知方法与统一位置插值有何不同？
    • It removes every query and key from attention
    • It changes only the output vocabulary size
    • It treats frequency bands differently to preserve local detail while extending positional range
    • It guarantees that all long-context tasks have identical accuracy
  6. What makes the original Position Interpolation workflow different from pretraining a new long-context model from scratch?
    • It eliminates every need for computation
    • It adapts an existing pretrained model toward the tested longer window with rescaling and fine-tuning
    • It requires replacing all learned parameters with random values
    • It proves all models can extend indefinitely without evaluation
  7. Does basic Position Interpolation inherently add new attention heads or enlarge the trainable architecture?
    • Yes; every doubled context requires twice as many attention heads
    • Yes; the vocabulary must expand by the context-length factor
    • Only the model's filename changes, with no effect on positions
    • No; it changes the position coordinates used by the existing model
  8. Why must a longer-context PI deployment still assess memory and computation requirements?
    • It processes more tokens and stores relevant state; compressing position coordinates does not remove that work
    • The number of actual input tokens shrinks automatically with their positions
    • All KV-cache storage becomes zero after rescaling
    • Every extended model has unchanged latency regardless of input length