Kvíz spojený s průvodcem · Střední úroveň

Reducing LLM App Latency Quiz

Map latency techniques to the stages they affect and check tradeoffs in quality, freshness, reliability, privacy, and cost.

Související vodicí cestySnížení latence aplikací s LLM
Otázka 1 z 8

What does streaming usually improve for a user?

Náhled každé otázky8 otázky

Odpovědi a vysvětlení se zobrazují, jak budete v kvízu.

  1. What does streaming usually improve for a user?
    • The model’s factual accuracy automatically
    • How soon partial output becomes visible, not necessarily total completion time
    • The time required to train the model
    • The freshness of cached data
  2. When is parallel execution most appropriate?
    • When each call depends on the previous result
    • Whenever a request uses a model
    • When failures must be hidden
    • When calls are independent and their results can be combined safely
  3. What can provider prompt caching reduce?
    • Repeated processing for eligible shared prompt prefixes
    • Every application response’s generation time to zero
    • All output tokens billed by every provider
    • The need for authorization checks
  4. What risk comes with application-level response caching?
    • Increasing model accuracy automatically
    • Making every cache miss faster
    • Returning stale or user-specific content to the wrong request
    • Removing the need for network security
  5. What should be done before routing a task to a smaller model?
    • Assume smaller models always perform identically
    • Evaluate task-specific quality as well as latency and cost
    • Disable all monitoring
    • Use only training accuracy
  6. Why are batch APIs generally unsuitable for an interactive reply?
    • They are designed for asynchronous processing windows rather than immediate response
    • They return outputs before receiving requests
    • They always have lower final accuracy
    • They cannot process multiple requests
  7. How should an application reduce tool-call latency?
    • Run all calls concurrently regardless of dependency
    • Skip permission checks
    • Cache all results indefinitely
    • Remove unnecessary round trips and parallelize only independent steps
  8. What should a latency optimization experiment monitor?
    • Only average TTFT
    • Only the number of source files
    • Tail latency, failures, quality, and cost on representative traffic
    • Only model size