Kuiz berkaitan panduan · Sederhana Tahap
Time to First Token and Latency Metrics Quiz
Distinguish TTFT, inter-token latency, end-to-end latency, throughput, and tail percentiles, then compare serving performance under similar workloads.
soalan 1 daripada 8
What does TTFT measure in a streaming LLM response?
Pratonton setiap soalan8 soalan
Jawapan dan penjelasan muncul semasa anda mengambil kuiz.
- What does TTFT measure in a streaming LLM response?
- The time between the last two tokens only
- The number of tokens processed per hour
- The time from request start until the first generated token arrives
- The time to load a model into a registry
- What does inter-token latency describe?
- The time gaps between generated output tokens
- The delay before a request is submitted
- The total number of concurrent requests
- The model’s training time
- Bagaimana throughput agregat berbeza daripada kelewatan per permintaan?
- They are identical metrics
- Throughput is always a percentile
- Latency measures only token count
- Throughput counts work handled over time; latency measures delay for a request
- Mengapa memeriksa kelewatan p95 atau p99 selain purata?
- They remove network delay from all requests
- Tail percentiles reveal slow requests hidden by a faster average
- They measure model quality
- They guarantee a maximum response time
- Which factor can contribute to high TTFT?
- Only the final response length
- Only the model’s training corpus
- Queueing or processing a long prompt before generation begins
- The count of cached database rows unrelated to the request
- Mengapa perbandingan penanda aras mesti menggunakan prompt, panjang output, dan beban yang serupa?
- They make all models produce identical answers
- Those workload differences can change latency and throughput
- A benchmark cannot include output tokens
- Only hardware affects results
- How does the cited vLLM benchmark define TPOT?
- A time-per-output-token metric derived from end-to-end latency and TTFT
- The time to download model weights
- A throughput quota
- A percentile of training loss
- Does a low average TTFT alone prove an application feels fast?
- Yes, for every application
- Yes, if throughput is not measured
- No, because TTFT is unrelated to response delay
- No; task needs, tail latency, token cadence, errors, and quality also matter