引導式測驗 · 中等 等級

減少 LLM 應用程式延遲測驗

將延遲技術映射到它們影響的階段,並檢查品質、新鮮度、可靠性、隱私和成本方面的權衡。

問題 1 的 8

What does streaming usually improve for a user?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What does streaming usually improve for a user?
    • The model’s factual accuracy automatically
    • How soon partial output becomes visible, not necessarily total completion time
    • The time required to train the model
    • The freshness of cached data
  2. When is parallel execution most appropriate?
    • When each call depends on the previous result
    • Whenever a request uses a model
    • When failures must be hidden
    • When calls are independent and their results can be combined safely
  3. What can provider prompt caching reduce?
    • Repeated processing for eligible shared prompt prefixes
    • Every application response’s generation time to zero
    • All output tokens billed by every provider
    • The need for authorization checks
  4. What risk comes with application-level response caching?
    • Increasing model accuracy automatically
    • Making every cache miss faster
    • Returning stale or user-specific content to the wrong request
    • Removing the need for network security
  5. What should be done before routing a task to a smaller model?
    • Assume smaller models always perform identically
    • Evaluate task-specific quality as well as latency and cost
    • Disable all monitoring
    • Use only training accuracy
  6. Why are batch APIs generally unsuitable for an interactive reply?
    • They are designed for asynchronous processing windows rather than immediate response
    • They return outputs before receiving requests
    • They always have lower final accuracy
    • They cannot process multiple requests
  7. How should an application reduce tool-call latency?
    • Run all calls concurrently regardless of dependency
    • Skip permission checks
    • Cache all results indefinitely
    • Remove unnecessary round trips and parallelize only independent steps
  8. What should a latency optimization experiment monitor?
    • Only average TTFT
    • Only the number of source files
    • Tail latency, failures, quality, and cost on representative traffic
    • 僅型號尺寸