引導式測驗 · 簡單 等級

AI 評估基礎知識測驗

設定驗收標準,將開發與保留的資料分開,與基準進行比較,選擇任務適合性檢查,並記錄不確定性和限制。

問題 1 的 8

What should a team define before evaluating an AI system?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What should a team define before evaluating an AI system?
    • Acceptance criteria for inputs, expected outputs, tolerable errors, and escalation
    • A press release about model quality
    • A single average score
    • A deployment date only
  2. How should development examples differ from a held-out evaluation set?
    • Use the held-out set for prompt tuning every day
    • Use development examples for iteration and reserve held-out examples to test choices
    • Mix both sets before every run
    • Keep no record of which examples were used
  3. What happens when a team repeatedly tunes on its final test set?
    • 測試集變得更大
    • The score becomes a baseline
    • The test set becomes another development set
    • The model automatically generalizes better
  4. Which comparison can show whether added model complexity provides practical benefit?
    • Compare only with last week’s best score
    • Compare with a simple baseline
    • Compare against a random deployment date
    • Compare with a different task’s benchmark
  5. For a summarizer, which evaluation checks does the guide recommend?
    • Server uptime and account age
    • Pixel similarity and frame rate
    • Factual consistency and coverage
    • 訓練紀元數
  6. How should a team verify an AI agent’s completed action?
    • Trust its success message
    • Count how many tools it called
    • Check the authoritative final state and unintended side effects
    • Ask the same model whether it succeeded
  7. Why might a rare failure deserve more attention than many harmless wording differences?
    • Consequences of the failure can outweigh frequent low-impact variations
    • Rare events are always measurement errors
    • Wording differences cannot be evaluated
    • The average score must be ignored
  8. What should an evaluation document about its limitations?
    • That every real-world case was tested
    • Where the evaluation does not represent actual use
    • Only the name of the model
    • That uncertainty has been eliminated