引導式測驗 · 中等 等級
第一個令牌的時間和延遲指標測驗
區分 TTFT、令牌間延遲、端對端延遲、吞吐量和尾部百分位數,然後比較類似工作負載下的服務效能。
問題 1 的 8
What does TTFT measure in a streaming LLM response?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- What does TTFT measure in a streaming LLM response?
- The time between the last two tokens only
- The number of tokens processed per hour
- The time from request start until the first generated token arrives
- The time to load a model into a registry
- What does inter-token latency describe?
- The time gaps between generated output tokens
- The delay before a request is submitted
- The total number of concurrent requests
- The model’s training time
- How does aggregate throughput differ from per-request latency?
- They are identical metrics
- Throughput is always a percentile
- Latency measures only token count
- Throughput counts work handled over time; latency measures delay for a request
- Why inspect p95 or p99 latency in addition to the average?
- They remove network delay from all requests
- Tail percentiles reveal slow requests hidden by a faster average
- They measure model quality
- They guarantee a maximum response time
- Which factor can contribute to high TTFT?
- Only the final response length
- Only the model’s training corpus
- Queueing or processing a long prompt before generation begins
- The count of cached database rows unrelated to the request
- Why must benchmark comparisons use similar prompts, output lengths, and load?
- They make all models produce identical answers
- Those workload differences can change latency and throughput
- A benchmark cannot include output tokens
- Only hardware affects results
- How does the cited vLLM benchmark define TPOT?
- A time-per-output-token metric derived from end-to-end latency and TTFT
- The time to download model weights
- A throughput quota
- A percentile of training loss
- Does a low average TTFT alone prove an application feels fast?
- Yes, for every application
- Yes, if throughput is not measured
- No, because TTFT is unrelated to response delay
- No; task needs, tail latency, token cadence, errors, and quality also matter