引導式測驗 · 中等 等級

第一個令牌的時間和延遲指標測驗

區分 TTFT、令牌間延遲、端對端延遲、吞吐量和尾部百分位數,然後比較類似工作負載下的服務效能。

問題 1 的 8

What does TTFT measure in a streaming LLM response?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What does TTFT measure in a streaming LLM response?
    • The time between the last two tokens only
    • The number of tokens processed per hour
    • The time from request start until the first generated token arrives
    • The time to load a model into a registry
  2. What does inter-token latency describe?
    • The time gaps between generated output tokens
    • The delay before a request is submitted
    • The total number of concurrent requests
    • The model’s training time
  3. How does aggregate throughput differ from per-request latency?
    • They are identical metrics
    • Throughput is always a percentile
    • Latency measures only token count
    • Throughput counts work handled over time; latency measures delay for a request
  4. Why inspect p95 or p99 latency in addition to the average?
    • They remove network delay from all requests
    • Tail percentiles reveal slow requests hidden by a faster average
    • They measure model quality
    • They guarantee a maximum response time
  5. Which factor can contribute to high TTFT?
    • Only the final response length
    • Only the model’s training corpus
    • Queueing or processing a long prompt before generation begins
    • The count of cached database rows unrelated to the request
  6. Why must benchmark comparisons use similar prompts, output lengths, and load?
    • They make all models produce identical answers
    • Those workload differences can change latency and throughput
    • A benchmark cannot include output tokens
    • Only hardware affects results
  7. How does the cited vLLM benchmark define TPOT?
    • A time-per-output-token metric derived from end-to-end latency and TTFT
    • The time to download model weights
    • A throughput quota
    • A percentile of training loss
  8. Does a low average TTFT alone prove an application feels fast?
    • Yes, for every application
    • Yes, if throughput is not measured
    • No, because TTFT is unrelated to response delay
    • No; task needs, tail latency, token cadence, errors, and quality also matter