Cuestionario vinculado a una guía · fácil Nivel

Prueba de conceptos básicos de evaluación de IA

Establezca criterios de aceptación, separe el desarrollo de los datos retenidos, compárelos con una línea de base, elija comprobaciones de adecuación a las tareas y documente la incertidumbre y los límites.

Rutas de guía relacionadasConceptos básicos de evaluación de IA
Pregunta 1 de 8

What should a team define before evaluating an AI system?

Vista previa de cada pregunta8 preguntas

Las respuestas y explicaciones aparecen a medida que realiza el cuestionario.

  1. What should a team define before evaluating an AI system?
    • Acceptance criteria for inputs, expected outputs, tolerable errors, and escalation
    • A press release about model quality
    • A single average score
    • A deployment date only
  2. How should development examples differ from a held-out evaluation set?
    • Use the held-out set for prompt tuning every day
    • Use development examples for iteration and reserve held-out examples to test choices
    • Mix both sets before every run
    • Keep no record of which examples were used
  3. What happens when a team repeatedly tunes on its final test set?
    • The test set becomes larger
    • The score becomes a baseline
    • The test set becomes another development set
    • The model automatically generalizes better
  4. Which comparison can show whether added model complexity provides practical benefit?
    • Compare only with last week’s best score
    • Compare with a simple baseline
    • Compare against a random deployment date
    • Compare with a different task’s benchmark
  5. For a summarizer, which evaluation checks does the guide recommend?
    • Server uptime and account age
    • Pixel similarity and frame rate
    • Factual consistency and coverage
    • Número de épocas de entrenamiento
  6. How should a team verify an AI agent’s completed action?
    • Trust its success message
    • Count how many tools it called
    • Check the authoritative final state and unintended side effects
    • Ask the same model whether it succeeded
  7. Why might a rare failure deserve more attention than many harmless wording differences?
    • Consequences of the failure can outweigh frequent low-impact variations
    • Rare events are always measurement errors
    • Wording differences cannot be evaluated
    • The average score must be ignored
  8. What should an evaluation document about its limitations?
    • That every real-world case was tested
    • Where the evaluation does not represent actual use
    • Only the name of the model
    • That uncertainty has been eliminated