Guide-linked quiz · Medium Level

The AI Effect: Moving Goalposts Quiz

Distinguish changing AI labels from measured progress, identify unfairly shifting criteria, and preserve room for legitimate questions about limitations.

Question 1 of 8

A familiar text-recognition feature stops being described as AI. What does that change alone establish?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. A familiar text-recognition feature stops being described as AI. What does that change alone establish?
    • A change in labeling, not the disappearance of the capability
    • That its algorithm no longer exists
    • That recognition never involved engineering progress
    • That every user now receives perfect results
  2. A vendor adds “AI” to the name of an existing feature. What evidence is still needed to establish an improvement?
    • Only the new name
    • A comparison of actual behavior under defined conditions
    • Only the launch announcement’s length
    • An assumption that every relabeling changes the algorithm
  3. A critic accepts a benchmark improvement but asks about real deployment failures. How should that request be treated?
    • As proof the critic rejects all progress
    • As automatically dishonest
    • As a potentially legitimate additional evaluation question
    • As evidence the benchmark score was fabricated
  4. What makes a moving-goalpost problem different from an explicit new deployment requirement?
    • The new criterion uses technical language
    • The task has a short name
    • The system has more than one limitation
    • The original success criterion is silently changed to deny the demonstrated result
  5. How can a team make progress comparisons less dependent on changing labels?
    • Keep the task, data, metric and conditions explicit
    • Count how often a product is called intelligent
    • Use only the newest marketing term
    • Stop reporting failures after a successful demo
  6. A recognition system succeeds on clean pages. What follows when damaged-page reading becomes the next goal?
    • The clean-page result must be denied
    • Acknowledge the result and test the additional condition separately
    • Assume the new condition has already been solved
    • Change the old score without explaining the new data
  7. Which conclusion is not established by one genuine AI milestone?
    • That the measured task improved
    • That the recorded test had an outcome
    • That the system is suitable for every deployment
    • That the result may be worth investigating
  8. Two benchmark runs use different datasets and scoring rules. What should a comparison disclose?
    • Only the higher score
    • Only the models’ product names
    • Nothing if both reports use the word AI
    • The changed conditions rather than assuming the scores are directly comparable

Keep testing yourself

More quizzes picked for your level.

View all quizzes