Production Readiness Checklist for LLM Apps

Unlike traditional software, LLM applications don't fail in a fixed way — the same input can produce different outputs across runs, and a prompt or model change can silently degrade quality on some inputs while improving it on others. A structured evaluation suite is a useful baseline before changing a prompt or model. It scores outputs against defined criteria rather than requiring an exact string match, and should include the task’s real failure cases. Input and output checks can help flag harmful requests, policy violations, and sensitive-data exposure, but they do not guarantee that every unsafe or false answer will be caught. Test them against representative and adversarial cases, and keep a human escalation path for high-impact uses. Operational monitoring may track latency, token use, errors, and evaluation trends. If prompts or completions are logged, minimize personal data, restrict access, set retention limits, and redact sensitive fields where appropriate. Rate limits protect both the app's budget and the shared quota with the model provider; without them, a bug (like an infinite retry loop or a scraper hitting an endpoint) can burn through a monthly budget in hours. Fallback behavior matters because model providers do have outages and degraded performance windows — a production app needs a defined behavior (a cached response, a secondary provider, or a clear error message) rather than an unhandled exception reaching the user. Cost caps, ideally enforced both in application logic and in the provider's own billing controls, protect against runaway bills from bugs or abuse. A common misconception is that passing a demo or a handful of manual tests is equivalent to production readiness; LLM failure modes (hallucination, prompt injection via user content, inconsistent formatting) often only surface at scale or under adversarial input, which is exactly what evals and guardrails are designed to catch systematically.