Bài kiểm tra có liên kết hướng dẫn · Trung bình Cấp độ
Bài kiểm tra về số liệu về thời gian đến mã thông báo đầu tiên và độ trễ
Phân biệt TTFT, độ trễ giữa các mã thông báo, độ trễ từ đầu đến cuối, thông lượng và phần trăm đuôi, sau đó so sánh hiệu suất phân phát trong các khối lượng công việc tương tự.
Câu hỏi 1 của 8
What does TTFT measure in a streaming LLM response?
Xem trước mọi câu hỏi8 câu hỏi
Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.
- What does TTFT measure in a streaming LLM response?
- The time between the last two tokens only
- The number of tokens processed per hour
- The time from request start until the first generated token arrives
- The time to load a model into a registry
- What does inter-token latency describe?
- The time gaps between generated output tokens
- The delay before a request is submitted
- The total number of concurrent requests
- The model’s training time
- How does aggregate throughput differ from per-request latency?
- They are identical metrics
- Throughput is always a percentile
- Latency measures only token count
- Throughput counts work handled over time; latency measures delay for a request
- Why inspect p95 or p99 latency in addition to the average?
- They remove network delay from all requests
- Tail percentiles reveal slow requests hidden by a faster average
- They measure model quality
- They guarantee a maximum response time
- Which factor can contribute to high TTFT?
- Only the final response length
- Only the model’s training corpus
- Queueing or processing a long prompt before generation begins
- The count of cached database rows unrelated to the request
- Why must benchmark comparisons use similar prompts, output lengths, and load?
- They make all models produce identical answers
- Those workload differences can change latency and throughput
- A benchmark cannot include output tokens
- Only hardware affects results
- How does the cited vLLM benchmark define TPOT?
- A time-per-output-token metric derived from end-to-end latency and TTFT
- The time to download model weights
- A throughput quota
- A percentile of training loss
- Does a low average TTFT alone prove an application feels fast?
- Yes, for every application
- Yes, if throughput is not measured
- No, because TTFT is unrelated to response delay
- No; task needs, tail latency, token cadence, errors, and quality also matter