引導式測驗 · 中等 等級

推測性解碼測驗

涵蓋草稿和驗證解碼、精確目標分佈保存、接受率限制以及加速的實際測量。

相關引導路徑推測性解碼
問題 1 的 8

推測性解碼如何加速大型模型的產生?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. 推測性解碼如何加速大型模型的產生?
    • 小型草稿模型猜測大型模型並行驗證的幾個標記
    • 將模型永久壓縮到一半大小
    • 移除注意力機制
    • 使用更多資料訓練模型
  2. For a large model at low batch size, why can one-token autoregressive decoding be slow?
    • Each token requires internet access
    • Repeated reads of large model weights can make decoding memory-bandwidth bound
    • The model permanently stores a full copy of every possible output
    • The tokenizer performs the model’s matrix multiplications
  3. In greedy speculative decoding, what happens at the first token where draft and target predictions disagree?
    • The whole conversation is thrown away and restarted from scratch
    • 草稿模型立即重新訓練
    • The matching prefix is kept, the target token is used at the mismatch, and later draft tokens are discarded
    • 輸出品質永久下降
  4. Under exact speculative sampling, how does the output distribution compare with sampling from the target model alone?
    • It is always lower quality but faster
    • It is identical only when the draft model is larger
    • It becomes deterministic regardless of target sampling
    • It is identical in distribution
  5. 最直接決定推測解碼加速的因素是什麼?
    • 模型草案猜測的接受率
    • GPU的顏色
    • 型號名稱的長度
    • 一天中的時間
  6. 美杜莎和 EAGLE 在方法上有什麼共同點?
    • 他們需要量子硬體
    • 它們消除了對單獨的草稿模型的需要
    • 他們刪除目標模型
    • 他們只處理圖像
  7. What should a serving team measure before claiming speculative decoding will speed up its workload?
    • Only the target model’s parameter count
    • Only whether the draft produces fluent text
    • Draft cost, target verification cost, accepted tokens and end-to-end latency under representative traffic
    • The number of tokenizer files
  8. In exact speculative sampling, what happens when a proposed token is rejected?
    • The whole conversation is discarded
    • The target model is permanently changed
    • The rejected token is kept without verification
    • A correction step samples from the distribution needed to preserve the target model’s output distribution