Guide-linked quiz · Medium Level
Nondeterminism in LLM Outputs Quiz
Understand why repeated LLM calls vary and which controls improve reproducibility without guaranteeing identical outputs across systems.
Question 1 of 8
Why might an LLM return different text for the same prompt on two runs?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- Why might an LLM return different text for the same prompt on two runs?
- A prompt can only ever have one possible continuation
- Sampling, backend changes, numerical execution, or different tool results can affect generation
- Temperature zero guarantees identical output everywhere
- A model never uses probabilities
- What does a seed parameter generally provide in a supported API?
- A guarantee of identical output across providers
- A guarantee that an answer is correct
- A way to pin tool results automatically
- Best-effort repeatability under matching request and backend conditions
- What can a system fingerprint help a developer detect?
- A change in backend configuration that may affect reproducibility
- Whether the generated answer is true
- Whether users like the answer
- The prompt’s character count
- Why might temperature zero still fail to guarantee identical hosted outputs?
- Temperature zero disables the model
- The API ignores every prompt
- Infrastructure and numerical execution can vary despite deterministic sampling settings
- Tokens are not probabilistic
- What should an experiment log to help reproduce an output?
- Only the response text
- Model identifier, prompt, sampling settings, seed, fingerprint, and tool state
- Only the user’s account name
- Only the final score
- When is exact-string matching most appropriate in an evaluation?
- When the task explicitly requires exact formatting or a fixed output
- For every open-ended response
- When model versions change often
- When no reference output exists
- How can teams evaluate variation when exact matching is too brittle?
- Ignore all failures
- Compare only response length
- Assume every paraphrase is wrong
- Use structured fields, semantic criteria, or task outcomes
- Does recording a seed make a system correct or secure?
- Yes, it guarantees factual answers
- Yes, it encrypts prompts
- No; it only helps control generation variation in supported conditions
- No, because seeds have no use
Keep testing yourself
More quizzes picked for your level.