AI Inference Quiz

Answers and explanations appear as you take the quiz.

1. What is the difference between training and inference?
   • Training adjusts learned parameters; inference uses a trained model to process an input
   • They are two names for the same parameter-update step
   • Training only formats outputs; inference labels the training data
   • Inference always retrains the model before responding
2. When a text service generates a response token by token, what is happening?
   • The entire response is written before the request arrives
   • The model is run repeatedly using the context so far to generate more tokens
   • The model’s weights are necessarily updated after every token
   • A person must be typing each token behind the interface
3. In an inference service, what does latency measure?
   • How many requests finish over a period
   • How many parameters the model contains
   • How long a request takes to receive a response
   • How accurate the training labels are
4. In the guide’s constructed sequential request, validation takes 20 ms, retrieval 180 ms, generation 900 ms and formatting 30 ms. What is the total?
   • 1,030 ms
   • 1,100 ms
   • 930 ms
   • 1,130 ms
5. Why might batching improve throughput but increase the wait for one request?
   • The service processes requests together, so an individual request may wait for the batch
   • Batching removes all model computation
   • Every request is completed before it enters a batch
   • Throughput and latency always move in the same direction
6. What should a serving memory budget include besides model weights?
   • Only the disk space used by the model file
   • Working buffers, concurrent requests and cached representations
   • Only the text returned to the user
   • The training labels but no runtime state
7. What should an inference deployment provide when the model cannot answer or exceeds its limit?
   • An infinite wait for more tokens
   • A silent failure with no user indication
   • A timeout and a usable fallback response
   • A fabricated answer to avoid an error
8. After changing precision, batching or preprocessing, how should a team judge the optimization?
   • Compare only speed on one short prompt
   • Assume fewer bits always preserve quality
   • Use the same test set repeatedly to select every tuning choice
   • Retest outputs on a versioned evaluation set and confirm required quality remains