Guide-linked quiz · Medium Level

Eval-Driven Development Quiz

Use representative cases, baselines, failure analysis, regression tests, and versioned prompts to guide changes to an LLM application.

Related guide pathsEval Driven Development
Question 1 of 8

What established software practice does eval-driven development borrow its structure from?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What established software practice does eval-driven development borrow its structure from?
    • Test-driven development, where a test is written before the code that satisfies it
    • Waterfall project management
    • Agile daily standups
    • Object-oriented design patterns
  2. Why does the deep dive say prompt changes can be risky without an eval suite?
    • Because prompt changes always cost more in API fees
    • Because rewording one instruction to fix one failure can silently break a different one, and this might not surface until production
    • Because prompts cannot be version-controlled
    • Because models ignore prompt changes entirely
  3. In the model-switch example, what did the eval suite reveal about the newer, faster model?
    • It was cheaper but had worse latency
    • It failed to run at all
    • It scored lower on factual accuracy for financial documents despite being faster
    • It required a different programming language
  4. How should a team treat a confirmed real-world failure in its evaluation suite?
    • Keep every reported output forever without review
    • Add only failures that are easy to automate
    • Delete the case after one passing run
    • Add a reproducible case when it reflects intended behavior, and revise or retire it if requirements change
  5. Why might an aggregate pass rate be misleading when comparing a baseline and candidate prompt?
    • It can hide a specific case where the candidate got worse even while overall scores improved
    • Aggregate numbers are always calculated incorrectly by eval tools
    • Aggregate scores cannot be computed for LLM outputs
    • Because only judge models produce aggregate scores
  6. Does eval-driven development require a large test suite before a prototype can be evaluated?
    • Yes; every possible input must be listed
    • No; start with a small set of important representative cases and grow it as evidence accumulates
    • Yes, but only for models with tools
    • No; manual demos replace all repeatable checks
  7. According to the technical insight, what three things does a working eval-driven workflow typically separate?
    • The model provider, the API key, and the billing account
    • The frontend, backend, and database layers
    • The test cases, the scoring method, and a comparison report between baseline and candidate
    • The prompt language, model size, and temperature setting
  8. Why can version-controlling prompts and eval suites together help teams investigate a regression?
    • It reduces cloud storage costs
    • Because version control systems require it for prompts specifically
    • Because it makes prompts run faster at inference time
    • It can link a changed prompt to the corresponding eval results in version history

Keep testing yourself

More quizzes picked for your level.

View all quizzes