Continuous Batching Quiz

Answers and explanations appear as you take the quiz.

1. What is the main weakness of static (fixed) batching for LLM serving?
   • It can only run on CPUs
   • It requires labeling all the data first
   • Short requests finish early but their slots idle while the batch waits for the longest one
   • It makes the model less accurate
2. At what granularity does continuous batching add and remove requests?
   • Once per day
   • Per decoding step (one token at a time)
   • Per full request only
   • Per training epoch
3. A continuous-batching request finishes and frees capacity. What can the scheduler do next?
   • Restart every remaining request from its first token
   • Guarantee every queued request runs immediately
   • Discard the finished answer without returning it
   • Admit eligible queued work if its resource budgets allow
4. Which technique pairs naturally with continuous batching to make inserting/removing sequences cheap?
   • PagedAttention's block-based KV cache
   • FP32 double precision
   • Disk swapping
   • Manual data labeling
5. What research paper is commonly credited with popularizing iteration-level (continuous) batching?
   • The original Transformer paper
   • The Orca paper
   • A 1980s database paper
   • The ImageNet paper
6. What utilization problem can continuous batching reduce compared with a fixed request batch?
   • Every model accuracy error
   • The need to store any KV cache
   • All queueing at any traffic rate
   • Waiting for the longest request before admitting new eligible work
7. A continuous-batching server has reached its memory budget. Must a newly arrived request enter the next iteration?
   • No; admission can wait until suitable capacity is available
   • Yes; the name guarantees unlimited request admission
   • Yes; the server removes all active outputs automatically
   • Only if the request has no punctuation
8. A long prompt’s prefill delays tokens for already decoding requests. Which serving objective exposes that tradeoff?
   • Measure only the weight-file name
   • Assume more admitted prompts always improve every latency
   • Measure time between decode tokens alongside throughput
   • Count only the number of model layers