Guide-linked quiz · Medium Level
Reducing LLM App Latency Quiz
Map latency techniques to the stages they affect and check tradeoffs in quality, freshness, reliability, privacy, and cost.
Question 1 of 8
What does streaming usually improve for a user?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- What does streaming usually improve for a user?
- The model’s factual accuracy automatically
- How soon partial output becomes visible, not necessarily total completion time
- The time required to train the model
- The freshness of cached data
- When is parallel execution most appropriate?
- When each call depends on the previous result
- Whenever a request uses a model
- When failures must be hidden
- When calls are independent and their results can be combined safely
- What can provider prompt caching reduce?
- Repeated processing for eligible shared prompt prefixes
- Every application response’s generation time to zero
- All output tokens billed by every provider
- The need for authorization checks
- What risk comes with application-level response caching?
- Increasing model accuracy automatically
- Making every cache miss faster
- Returning stale or user-specific content to the wrong request
- Removing the need for network security
- What should be done before routing a task to a smaller model?
- Assume smaller models always perform identically
- Evaluate task-specific quality as well as latency and cost
- Disable all monitoring
- Use only training accuracy
- Why are batch APIs generally unsuitable for an interactive reply?
- They are designed for asynchronous processing windows rather than immediate response
- They return outputs before receiving requests
- They always have lower final accuracy
- They cannot process multiple requests
- How should an application reduce tool-call latency?
- Run all calls concurrently regardless of dependency
- Skip permission checks
- Cache all results indefinitely
- Remove unnecessary round trips and parallelize only independent steps
- What should a latency optimization experiment monitor?
- Only average TTFT
- Only the number of source files
- Tail latency, failures, quality, and cost on representative traffic
- Only model size
Keep testing yourself
More quizzes picked for your level.