Guide-linked quiz · Medium Level

LLM Eval Frameworks: Promptfoo and DeepEval Quiz

Compare Promptfoo and DeepEval workflows, test cases, deterministic assertions, judge metrics, CI use, and the human review each still needs.

Question 1 of 8

What core problem do both Promptfoo and DeepEval address?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What core problem do both Promptfoo and DeepEval address?
    • Making LLM behavior testable through structured cases, repeatable runs, and suitable automated checks
    • Training new base models from scratch
    • Compressing model weights for faster inference
    • Translating prompts into multiple languages
  2. How is a Promptfoo evaluation typically configured?
    • Through a compiled binary with no config file
    • Through a YAML file listing prompts, providers, test cases and assertions
    • Through a spreadsheet macro
    • Through direct edits to the model's weights
  3. What does DeepEval's faithfulness metric primarily check?
    • Whether the output uses correct grammar
    • Whether the response is shorter than 100 words
    • Whether a RAG answer stays grounded in the retrieved context rather than adding unsupported claims
    • Whether the model responded within 1 second
  4. In the support-chatbot example, what caused the test suite to catch a regression?
    • The model started responding in a different language
    • The API key expired mid-test
    • The test suite was run twice by accident
    • A prompt edit accidentally dropped a required refund-policy disclaimer
  5. How does DeepEval structure an individual test case?
    • As an LLMTestCase object holding input, actual output, and optionally retrieval context and expected output
    • As a single string with no metadata
    • As a raw HTTP request log
    • As an image file
  6. What remains necessary when an evaluation framework reports passing metric scores?
    • Assume the application is safe in every context
    • Human review and task-specific validation for ambiguous or high-risk behavior
    • Retire the test dataset immediately
    • Trust the metric threshold as universal proof
  7. Why might a CI pipeline fail a build using one of these frameworks?
    • Because the repository has too many files
    • Because the model's API key is stored in an environment variable
    • Because a numeric metric score, such as faithfulness, falls below a configured threshold
    • Because the test suite runs on a weekend
  8. Why do many model-graded assertions and metrics require an extra API call per test case?
    • Because every test case must be translated first
    • Because the test framework requires two models to run simultaneously for redundancy
    • Because outputs must be compressed before scoring
    • Because they use a separate judge-model call to score the output against a rubric

Keep testing yourself

More quizzes picked for your level.

View all quizzes