Викторина, связанная с руководством · Средний Уровень
LLM Eval Frameworks: тест Promptfoo и DeepEval
Сравните рабочие процессы Promptfoo и DeepEval, тестовые сценарии, детерминированные утверждения, оценочные метрики, использование CI и человеческий анализ, в котором каждый из них все еще нуждается.
Вопрос 1 из 8
Какую основную проблему решают и Promptfoo, и DeepEval?
Предварительный просмотр каждого вопроса8 вопросы
Ответы и пояснения появляются по мере прохождения теста.
- Какую основную проблему решают и Promptfoo, и DeepEval?
- Making LLM behavior testable through structured cases, repeatable runs, and suitable automated checks
- Training new base models from scratch
- Compressing model weights for faster inference
- Перевод подсказок на несколько языков
- Как обычно настраивается оценка Promptfoo?
- Through a compiled binary with no config file
- Through a YAML file listing prompts, providers, test cases and assertions
- Through a spreadsheet macro
- Through direct edits to the model's weights
- Что в первую очередь проверяет метрика достоверности DeepEval?
- Whether the output uses correct grammar
- Whether the response is shorter than 100 words
- Whether a RAG answer stays grounded in the retrieved context rather than adding unsupported claims
- Whether the model responded within 1 second
- В примере чат‑бота поддержки, что вызвало обнаружение регрессии в тестовом наборе?
- The model started responding in a different language
- The API key expired mid-test
- The test suite was run twice by accident
- A prompt edit accidentally dropped a required refund-policy disclaimer
- Как DeepEval структурирует отдельный тестовый случай?
- As an LLMTestCase object holding input, actual output, and optionally retrieval context and expected output
- As a single string with no metadata
- As a raw HTTP request log
- As an image file
- What remains necessary when an evaluation framework reports passing metric scores?
- Assume the application is safe in every context
- Human review and task-specific validation for ambiguous or high-risk behavior
- Retire the test dataset immediately
- Trust the metric threshold as universal proof
- Why might a CI pipeline fail a build using one of these frameworks?
- Because the repository has too many files
- Because the model's API key is stored in an environment variable
- Because a numeric metric score, such as faithfulness, falls below a configured threshold
- Because the test suite runs on a weekend
- Why do many model-graded assertions and metrics require an extra API call per test case?
- Because every test case must be translated first
- Because the test framework requires two models to run simultaneously for redundancy
- Because outputs must be compressed before scoring
- Because they use a separate judge-model call to score the output against a rubric