ガイド連動クイズ · 簡単 レベル

AI評価基礎クイズ

受け入れ基準を設定し、開発を保留データから分離し、ベースラインと比較し、タスク適合性チェックを選択し、不確実性と限界を文書化します。

関連するガイドパスAI評価の基礎
質問 1 の 8

What should a team define before evaluating an AI system?

すべての質問をプレビューする8 質問

クイズに答えると、答えと解説が表示されます。

  1. What should a team define before evaluating an AI system?
    • Acceptance criteria for inputs, expected outputs, tolerable errors, and escalation
    • A press release about model quality
    • A single average score
    • A deployment date only
  2. How should development examples differ from a held-out evaluation set?
    • Use the held-out set for prompt tuning every day
    • Use development examples for iteration and reserve held-out examples to test choices
    • Mix both sets before every run
    • Keep no record of which examples were used
  3. What happens when a team repeatedly tunes on its final test set?
    • The test set becomes larger
    • The score becomes a baseline
    • The test set becomes another development set
    • The model automatically generalizes better
  4. Which comparison can show whether added model complexity provides practical benefit?
    • Compare only with last week’s best score
    • Compare with a simple baseline
    • Compare against a random deployment date
    • Compare with a different task’s benchmark
  5. For a summarizer, which evaluation checks does the guide recommend?
    • Server uptime and account age
    • Pixel similarity and frame rate
    • Factual consistency and coverage
    • トレーニング エポックの数
  6. How should a team verify an AI agent’s completed action?
    • Trust its success message
    • Count how many tools it called
    • Check the authoritative final state and unintended side effects
    • Ask the same model whether it succeeded
  7. Why might a rare failure deserve more attention than many harmless wording differences?
    • Consequences of the failure can outweigh frequent low-impact variations
    • Rare events are always measurement errors
    • Wording differences cannot be evaluated
    • The average score must be ignored
  8. What should an evaluation document about its limitations?
    • That every real-world case was tested
    • Where the evaluation does not represent actual use
    • Only the name of the model
    • That uncertainty has been eliminated