Quiz lié à un guide · Difficile Niveau

Quiz sur les benchmarks de l'IA

Vérifiez comment les tâches de référence, les données, les paramètres, l'incertitude, la contamination et les tests spécifiques à l'application façonnent ce qu'un score d'IA peut vous dire.

Chemins de guidage associésRepères d'IA
Question 1 de 8

What is an AI benchmark?

Prévisualisez chaque question8 des questions

Les réponses et les explications apparaissent au fur et à mesure que vous répondez au quiz.

  1. What is an AI benchmark?
    • A defined set of tasks, data, and scoring rules used to compare systems
    • A universal measure of intelligence
    • A guarantee the highest-scoring model is best for an application
    • A list of model parameters
  2. Two systems both report “accuracy.” What should be checked before comparing them?
    • Only model names
    • Only the number of parameters
    • Task definition, examples, scoring rules, and test conditions
    • Only whether the two reports use the same metric name
  3. Why record model version, prompt, tools, attempts, and evaluation date?
    • A record guarantees no data leakage
    • Those settings can change benchmark results
    • Settings do not affect model behavior
    • Only the date matters
  4. What is dataset contamination in an evaluation?
    • The benchmark includes multiple task types
    • A model uses a different prompt
    • A dataset contains timestamps
    • Test items or close variants were available during development
  5. A 100-question test shows one model answers 81 correctly and another 83. What does that difference alone establish?
    • Only the observed counts, not a reliable advantage by itself
    • That the second model is universally better
    • That the first model cannot solve the task
    • That the test set was contaminated
  6. Why can an average benchmark score hide weaknesses?
    • Averages always reveal every failure
    • A single metric measures all applications equally
    • Performance may differ across tasks, conditions, or groups
    • Scores do not depend on the task mix
  7. What should be compared alongside task success?
    • Only parameter count
    • Cost and latency
    • Only leaderboard position
    • Only how many times the result was cited
  8. Why add a private evaluation set for an application?
    • It guarantees a higher public score
    • It eliminates the need to document prompts
    • It prevents all model errors
    • It tests whether public-benchmark results transfer to the intended workflow