Guide-linked quiz · Medium Level
Eval-Driven Development Quiz
Use representative cases, baselines, failure analysis, regression tests, and versioned prompts to guide changes to an LLM application.
Question 1 of 8
What established software practice does eval-driven development borrow its structure from?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- What established software practice does eval-driven development borrow its structure from?
- Test-driven development, where a test is written before the code that satisfies it
- Waterfall project management
- Agile daily standups
- Object-oriented design patterns
- Why does the deep dive say prompt changes can be risky without an eval suite?
- Because prompt changes always cost more in API fees
- Because rewording one instruction to fix one failure can silently break a different one, and this might not surface until production
- Because prompts cannot be version-controlled
- Because models ignore prompt changes entirely
- In the model-switch example, what did the eval suite reveal about the newer, faster model?
- It was cheaper but had worse latency
- It failed to run at all
- It scored lower on factual accuracy for financial documents despite being faster
- It required a different programming language
- How should a team treat a confirmed real-world failure in its evaluation suite?
- Keep every reported output forever without review
- Add only failures that are easy to automate
- Delete the case after one passing run
- Add a reproducible case when it reflects intended behavior, and revise or retire it if requirements change
- Why might an aggregate pass rate be misleading when comparing a baseline and candidate prompt?
- It can hide a specific case where the candidate got worse even while overall scores improved
- Aggregate numbers are always calculated incorrectly by eval tools
- Aggregate scores cannot be computed for LLM outputs
- Because only judge models produce aggregate scores
- Does eval-driven development require a large test suite before a prototype can be evaluated?
- Yes; every possible input must be listed
- No; start with a small set of important representative cases and grow it as evidence accumulates
- Yes, but only for models with tools
- No; manual demos replace all repeatable checks
- According to the technical insight, what three things does a working eval-driven workflow typically separate?
- The model provider, the API key, and the billing account
- The frontend, backend, and database layers
- The test cases, the scoring method, and a comparison report between baseline and candidate
- The prompt language, model size, and temperature setting
- Why can version-controlling prompts and eval suites together help teams investigate a regression?
- It reduces cloud storage costs
- Because version control systems require it for prompts specifically
- Because it makes prompts run faster at inference time
- It can link a changed prompt to the corresponding eval results in version history
Keep testing yourself
More quizzes picked for your level.