Guide-linked quiz · Medium Level

RAG Evaluation Metrics and RAGAS Quiz

Check your understanding of RAG evaluation: faithfulness, answer relevance, context precision and recall, and how LLM judges score them.

Related guide pathsRAG Evaluation Metrics And Ragas
Question 1 of 8

Which metric checks whether every claim in the answer is supported by the retrieved context?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. Which metric checks whether every claim in the answer is supported by the retrieved context?
    • Answer relevance
    • Context recall
    • Faithfulness
    • Context precision
  2. Which of these metrics requires a ground-truth reference answer for each test question?
    • Context recall
    • Faithfulness
    • Answer relevance
    • Response latency
  3. What does context precision measure?
    • How quickly chunks are retrieved
    • Whether the answer is short
    • How many documents are in the index
    • Whether the relevant retrieved chunks are ranked near the top
  4. How does RAGAS estimate answer relevance?
    • By counting words shared by answer and context
    • By generating questions the answer would respond to and comparing them to the original question with embeddings
    • By asking the user to rate each answer
    • By checking the answer against a reference with exact string match
  5. A system shows high context recall but low context precision. What is the most likely fix?
    • Improve ranking, for example by adding a reranker
    • Switch to a smaller generation model
    • Remove the reference answers
    • Lower the number of test questions
  6. Why can a perfectly faithful answer still be wrong?
    • Faithfulness only checks grammar
    • Faithfulness ignores the retrieved context
    • It can faithfully repeat an outdated or incorrect document
    • Faithful answers are always too short
  7. Why should teams pin the judge model and settings when comparing RAG systems?
    • Pinning makes the judge free to run
    • Judges only work with one vector database
    • Pinning removes the need for test questions
    • LLM-judge scores vary with the judge model and prompt, so comparisons need identical settings
  8. Retrieval returns the right passages, but the answer adds unsupported details. Where should the team look first?
    • The chunking strategy
    • The prompt or the generation model
    • The embedding model
    • The number of documents indexed

Keep testing yourself

More quizzes picked for your level.

View all quizzes
Medium · 8 questionsPython Dependency Management for ML QuizTake quizMedium · 8 questionsTerraform for ML Infrastructure QuizTake quizMedium · 8 questionsTriton Language for Custom GPU Kernels QuizTake quiz