Guide-linked quiz · Medium Level

Reducing LLM App Latency Quiz

Map latency techniques to the stages they affect and check tradeoffs in quality, freshness, reliability, privacy, and cost.

Related guide pathsReducing LLM App Latency
Question 1 of 8

What does streaming usually improve for a user?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What does streaming usually improve for a user?
    • The model’s factual accuracy automatically
    • How soon partial output becomes visible, not necessarily total completion time
    • The time required to train the model
    • The freshness of cached data
  2. When is parallel execution most appropriate?
    • When each call depends on the previous result
    • Whenever a request uses a model
    • When failures must be hidden
    • When calls are independent and their results can be combined safely
  3. What can provider prompt caching reduce?
    • Repeated processing for eligible shared prompt prefixes
    • Every application response’s generation time to zero
    • All output tokens billed by every provider
    • The need for authorization checks
  4. What risk comes with application-level response caching?
    • Increasing model accuracy automatically
    • Making every cache miss faster
    • Returning stale or user-specific content to the wrong request
    • Removing the need for network security
  5. What should be done before routing a task to a smaller model?
    • Assume smaller models always perform identically
    • Evaluate task-specific quality as well as latency and cost
    • Disable all monitoring
    • Use only training accuracy
  6. Why are batch APIs generally unsuitable for an interactive reply?
    • They are designed for asynchronous processing windows rather than immediate response
    • They return outputs before receiving requests
    • They always have lower final accuracy
    • They cannot process multiple requests
  7. How should an application reduce tool-call latency?
    • Run all calls concurrently regardless of dependency
    • Skip permission checks
    • Cache all results indefinitely
    • Remove unnecessary round trips and parallelize only independent steps
  8. What should a latency optimization experiment monitor?
    • Only average TTFT
    • Only the number of source files
    • Tail latency, failures, quality, and cost on representative traffic
    • Only model size

Keep testing yourself

More quizzes picked for your level.

View all quizzes