Eval-Driven Development

Eval-driven development borrows its name and structure from test-driven development in traditional software: instead of writing production code and then hoping it works, a developer writes a failing test first, then writes code until the test passes. Applied to LLM systems, this means writing eval test cases - representative inputs paired with a way to judge correctness, whether that's an exact-match check, a rule-based assertion, or an LLM-as-judge rubric - before changing a prompt, swapping a model, or adjusting a retrieval pipeline. The evals then serve as the yardstick for whether a proposed change is actually an improvement, rather than relying on a developer's subjective impression from trying a few examples. This matters especially for LLM systems because prompt changes have famously non-local effects: rewording one instruction to fix one failure mode can silently break a different one, and without a broad eval suite that regression may not surface until a user hits it in production. In practice, eval suites tend to grow organically - a common pattern is that every reported real-world failure becomes a new permanent eval case, so that a reproduced regression can be caught if it returns under the covered test conditions. A frequent misconception is that eval-driven development requires a large formal test suite from day one; most teams start with a handful of cases covering their highest-value or most failure-prone scenarios and expand from there. Another misconception is that eval scores alone are sufficient without periodic human review, since a rubric or judge model can itself drift out of alignment with what users actually consider correct.