引導式測驗 · 硬 等級
YaRN 上下文視窗縮放測驗
查看 YaRN 頻段、過渡混合、注意力溫度、適配成本以及支援的長度和性能之間的差異。
問題 1 的 8
YaRN 修改了什麼位置編碼方案來擴展上下文長度?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- YaRN 修改了什麼位置編碼方案來擴展上下文長度?
- 旋轉位置嵌入 (RoPE)
- 絕對是正弦嵌入
- 學習位置查找表
- ALiBi 線性偏差
- YaRN 如何處理高頻與低頻 RoPE 維度?
- 它相等地插值兩者
- 它完全消除了高頻維度
- 它只修改高頻維度
- 它使高頻維度基本保持不變並插入低頻維度
- How should YaRN's reported data and fine-tuning efficiency be interpreted?
- As proof that no YaRN model ever needs training
- As results for its evaluated model-extension setups, rather than a fixed guarantee for every model
- As a universal promise that every dataset size works equally well
- As removal of the need for GPUs or inference computation
- 除了重新調整頻率之外,YaRN 還應用哪些額外調整來維持遠端注意力穩定?
- 輟學率增加
- 詞彙量更大
- 注意溫度(softmax 縮放)調整
- 漸層裁切
- What motivates evaluating a RoPE model before using a context far beyond its original training length?
- The model necessarily becomes perfectly accurate at longer lengths
- The context extension automatically reduces memory to zero
- A larger window proves every passage is equally easy to retrieve
- Positional and attention behavior may degrade outside the conditions it learned
- 樸素的線性位置插值會損害 YaRN 旨在保留的哪些功能?
- 附近標記之間良好的局部關係
- 模型的詞彙量大小
- 分詞器的速度
- 層數
- What happens in the transition region between YaRN's high- and low-frequency treatments?
- Every dimension is assigned a random vocabulary token
- All transition dimensions are deleted from attention
- The scheme blends original and interpolated frequency treatment across the relevant dimensions
- One identical scaling rule ignores frequency differences entirely
- What does a checkpoint's advertised longer YaRN context limit fail to establish by itself?
- Whether the positional scheme can accept any coordinates at all
- Reliable performance for every intended task and position within that window
- The spelling of the model's name
- A requirement that every inference request retrain all weights