Útmutatóhoz kapcsolódó kvíz · Közepes Szint

Reducing LLM App Latency Quiz

Map latency techniques to the stages they affect and check tradeoffs in quality, freshness, reliability, privacy, and cost.

Kapcsolódó útmutató utakReducing LLM App Latency
kérdés 1 -ból 8

What does streaming usually improve for a user?

Tekintse meg az összes kérdést8 kérdéseket

A válaszok és magyarázatok megjelennek a kvíz kitöltésekor.

  1. What does streaming usually improve for a user?
    • The model’s factual accuracy automatically
    • How soon partial output becomes visible, not necessarily total completion time
    • The time required to train the model
    • The freshness of cached data
  2. When is parallel execution most appropriate?
    • When each call depends on the previous result
    • Whenever a request uses a model
    • When failures must be hidden
    • When calls are independent and their results can be combined safely
  3. What can provider prompt caching reduce?
    • Repeated processing for eligible shared prompt prefixes
    • Every application response’s generation time to zero
    • All output tokens billed by every provider
    • The need for authorization checks
  4. What risk comes with application-level response caching?
    • Increasing model accuracy automatically
    • Making every cache miss faster
    • Returning stale or user-specific content to the wrong request
    • Removing the need for network security
  5. What should be done before routing a task to a smaller model?
    • Assume smaller models always perform identically
    • Evaluate task-specific quality as well as latency and cost
    • Disable all monitoring
    • Use only training accuracy
  6. Why are batch APIs generally unsuitable for an interactive reply?
    • They are designed for asynchronous processing windows rather than immediate response
    • They return outputs before receiving requests
    • They always have lower final accuracy
    • They cannot process multiple requests
  7. How should an application reduce tool-call latency?
    • Run all calls concurrently regardless of dependency
    • Skip permission checks
    • Cache all results indefinitely
    • Remove unnecessary round trips and parallelize only independent steps
  8. What should a latency optimization experiment monitor?
    • Only average TTFT
    • Only the number of source files
    • Tail latency, failures, quality, and cost on representative traffic
    • Csak a modell mérete