旋转位置嵌入测验

当您参加测验时，就会出现答案和解释。

1. RoPE 如何将位置信息注入变压器？
   • 通过将查询向量和键向量中的维度对旋转位置相关的角度
   • 通过向输入嵌入添加固定向量
   • 通过随机重新排序令牌
   • 通过将位置编号附加为文本
2. 应用RoPE后，两个token之间的注意力分数主要取决于：
   • 仅他们的绝对位置
   • 他们的相对距离（m减n）
   • 词汇量大小
   • 批量大小
3. What training-efficiency comparison did the YaRN paper report for context extension?
   • Exactly the same data and steps as all prior methods
   • No fine-tuning or training data of any kind
   • About 10x fewer tokens and 2.5x fewer training steps than prior methods
   • A fixed 100x increase in output tokens
4. 为什么不同维度对在 RoPE 中以不同频率旋转？
   • 通过跳过某些维度来节省内存
   • 随机化模型的输出
   • 因为所有对必须使用相同的角度
   • 捕捉精细的局部秩序和远程位置
5. What is the goal of techniques such as Position Interpolation and YaRN for RoPE-based models?
   • Extend the usable context length beyond the original training window using additional scaling/fine-tuning methods
   • Guarantee accurate recall at any sequence length with no evaluation
   • Remove position information from attention
   • Replace all token embeddings with absolute learned vectors
6. Does standard RoPE require learned absolute-position vectors to inject position into query and key representations?
   • Yes; it adds one learned vector for every possible sequence position
   • No; it applies position-dependent rotations to paired query/key dimensions
   • It removes query and key vectors entirely
   • It only adds position to value vectors
7. What does standard RoPE alone guarantee about a model’s context length?
   • Unlimited accurate context at every position
   • A fixed context of exactly 128 tokens
   • It does not by itself guarantee reliable use beyond the trained context length
   • Removal of the attention mechanism
8. What extended context length did the Position Interpolation paper report for its LLaMA evaluations?
   • Exactly 1,000 tokens with no position changes
   • Unlimited context with no fine-tuning
   • Up to 2 million tokens for all models
   • Up to 32,768 tokens with minimal fine-tuning