引導式測驗 · 硬 等級
連續批次測驗
應用迭代級 LLM 批次、容量限制和預填權衡,包括為什麼連續接納請求並不能保證零等待。
問題 1 的 8
LLM 服務的靜態(固定)批次的主要弱點是什麼?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- LLM 服務的靜態(固定)批次的主要弱點是什麼?
- 它只能在CPU上運行
- 它需要先標記所有數據
- 短請求提前完成,但它們的槽空閒,而批次等待最長的請求
- 它使模型不太準確
- 連續批次以什麼粒度新增和刪除請求?
- 每天一次
- 每個解碼步驟(一次一個令牌)
- 僅根據完整請求
- 每個訓練時期
- 一個持續批次的請求完成並釋放容量後,排程器接下來可以做什麼?
- Restart every remaining request from its first token
- Guarantee every queued request runs immediately
- Discard the finished answer without returning it
- Admit eligible queued work if its resource budgets allow
- 哪種技術與連續批次自然地結合在一起,可以使插入/刪除序列變得便宜?
- PagedAttention的基於區塊的KV快取
- FP32 雙精度
- 磁碟交換
- 手動數據標記
- 哪些研究論文普遍認為推廣了迭代級(連續)批次?
- 最初的 Transformer 論文
- 虎鯨紙
- 20 世紀 80 年代的資料庫論文
- ImageNet 論文
- What utilization problem can continuous batching reduce compared with a fixed request batch?
- Every model accuracy error
- The need to store any KV cache
- All queueing at any traffic rate
- Waiting for the longest request before admitting new eligible work
- A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
- No; admission can wait until suitable capacity is available
- Yes; the name guarantees unlimited request admission
- Yes; the server removes all active outputs automatically
- Only if the request has no punctuation
- A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
- Measure only the weight-file name
- Assume more admitted prompts always improve every latency
- Measure time between decode tokens alongside throughput
- Count only the number of model layers