引導式測驗 · 硬 等級

連續批次測驗

應用迭代級 LLM 批次、容量限制和預填權衡,包括為什麼連續接納請求並不能保證零等待。

相關引導路徑連續配料
問題 1 的 8

LLM 服務的靜態(固定)批次的主要弱點是什麼?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. LLM 服務的靜態(固定)批次的主要弱點是什麼?
    • 它只能在CPU上運行
    • 它需要先標記所有數據
    • 短請求提前完成,但它們的槽空閒,而批次等待最長的請求
    • 它使模型不太準確
  2. 連續批次以什麼粒度新增和刪除請求?
    • 每天一次
    • 每個解碼步驟(一次一個令牌)
    • 僅根據完整請求
    • 每個訓練時期
  3. 一個持續批次的請求完成並釋放容量後,排程器接下來可以做什麼?
    • Restart every remaining request from its first token
    • Guarantee every queued request runs immediately
    • Discard the finished answer without returning it
    • Admit eligible queued work if its resource budgets allow
  4. 哪種技術與連續批次自然地結合在一起,可以使插入/刪除序列變得便宜?
    • PagedAttention的基於區塊的KV快取
    • FP32 雙精度
    • 磁碟交換
    • 手動數據標記
  5. 哪些研究論文普遍認為推廣了迭代級(連續)批次?
    • 最初的 Transformer 論文
    • 虎鯨紙
    • 20 世紀 80 年代的資料庫論文
    • ImageNet 論文
  6. What utilization problem can continuous batching reduce compared with a fixed request batch?
    • Every model accuracy error
    • The need to store any KV cache
    • All queueing at any traffic rate
    • Waiting for the longest request before admitting new eligible work
  7. A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
    • No; admission can wait until suitable capacity is available
    • Yes; the name guarantees unlimited request admission
    • Yes; the server removes all active outputs automatically
    • Only if the request has no punctuation
  8. A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
    • Measure only the weight-file name
    • Assume more admitted prompts always improve every latency
    • Measure time between decode tokens alongside throughput
    • Count only the number of model layers