Тест на качество поиска

Ответы и пояснения появляются по мере прохождения теста.

1. Что измеряет Recall@k для поисковой системы?
   • The rank position of the first relevant document returned
   • The share of the top k results that are actually relevant
   • The share of all relevant documents found in the top k
   • The time taken for the system to return k results
2. A retrieval evaluation cares whether the best evidence appears near the top. What kind of metric addresses that concern?
   • A rank-aware metric
   • A server uptime metric
   • A count of stored files
   • A tokenizer vocabulary size
3. An exact order code is missed by semantic search, while a paraphrased question is missed by keyword search. What should the team inspect?
   • Only the number of documents stored
   • Whether every user asked the same question
   • The appearance of the search button
   • Each retrieval method’s blind spots on fixed examples
4. A team adds hybrid retrieval and reranking. How should it judge whether the change helped?
   • Compare only the highest similarity score after the change
   • Evaluate both versions on the same fixed labeled examples
   • Assume more retrieved passages always improve answers
   • Count the number of model parameters rather than results
5. A retrieved passage loses the sentence that qualifies its claim. What failure pattern should the reviewer flag?
   • The source document was necessarily false
   • The user must have asked an unanswerable question
   • A passage was cut away from its qualification
   • The search system returned too few languages
6. In the guide’s constructed example, five passages are returned and three are relevant. What is precision at five?
   • Three out of five, or 60%
   • Three out of four, or 75%
   • Five out of four, or 125%
   • One out of five, or 20%
7. Why should retrieval tests include access restrictions and unanswerable queries?
   • To increase the average similarity score
   • To check that the system avoids unauthorized evidence and unsupported answers
   • To ensure every query returns a document
   • To replace relevance labels
8. A passage is about the right product but describes an older version. What does this show?
   • The passage must rank first because it shares keywords
   • The query has no retrieval requirements
   • A high similarity score proves the answer
   • Topical relevance alone may not provide evidence for the specific version or date in the question