AI Inference Optimization Quiz

Answers and explanations appear as you take the quiz.

1. Why measure end-to-end performance before optimizing a model?
   • Every workload is model-compute bound
   • Optimization always improves task quality
   • A short warm benchmark represents production use
   • The bottleneck may be retrieval, tokenization, transfer, or output handling rather than inference
2. What can batching improve, and what can it worsen?
   • Throughput may improve while individual latency rises as requests wait for a batch
   • Both always improve together
   • Memory always decreases
   • Task accuracy is guaranteed to increase
3. When is a cached model response safe to reuse?
   • Whenever the request text looks similar
   • When the key accounts for relevant input, model version, permissions, and context
   • When the cache is stored privately
   • Only when the answer is short
4. What should be checked after reducing numerical precision?
   • Only whether inference became faster
   • Only the model’s file size
   • Task quality on the same representative evaluation set as the original configuration
   • Whether the GPU supports the format
5. A benchmark uses one warmed-up short request. Why may it mislead?
   • It contains too many user cases
   • A warm request is never measurable
   • It evaluates the wrong model weights by definition
   • It omits cold starts, concurrency, long requests, and cancellation behavior
6. What does time to first token measure compared with total completion time?
   • The initial streaming delay versus the time to finish the response
   • Two names for exactly the same metric
   • Model accuracy versus calibration
   • Queue depth versus cache hit rate
7. Why tune one meaningful factor at a time?
   • It guarantees no regression
   • It helps attribute improvements and regressions to the change made
   • It removes the need for a baseline
   • It makes different workloads comparable automatically
8. What is the optimization objective in the guide?
   • The highest isolated throughput number
   • The lowest latency on a single test
   • A better completed task within the required quality and resource constraints
   • The fewest model parameters regardless of quality