Guide-linked quiz · Medium Level
The AI Effect: Moving Goalposts Quiz
Distinguish changing AI labels from measured progress, identify unfairly shifting criteria, and preserve room for legitimate questions about limitations.
Question 1 of 8
A familiar text-recognition feature stops being described as AI. What does that change alone establish?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- A familiar text-recognition feature stops being described as AI. What does that change alone establish?
- A change in labeling, not the disappearance of the capability
- That its algorithm no longer exists
- That recognition never involved engineering progress
- That every user now receives perfect results
- A vendor adds “AI” to the name of an existing feature. What evidence is still needed to establish an improvement?
- Only the new name
- A comparison of actual behavior under defined conditions
- Only the launch announcement’s length
- An assumption that every relabeling changes the algorithm
- A critic accepts a benchmark improvement but asks about real deployment failures. How should that request be treated?
- As proof the critic rejects all progress
- As automatically dishonest
- As a potentially legitimate additional evaluation question
- As evidence the benchmark score was fabricated
- What makes a moving-goalpost problem different from an explicit new deployment requirement?
- The new criterion uses technical language
- The task has a short name
- The system has more than one limitation
- The original success criterion is silently changed to deny the demonstrated result
- How can a team make progress comparisons less dependent on changing labels?
- Keep the task, data, metric and conditions explicit
- Count how often a product is called intelligent
- Use only the newest marketing term
- Stop reporting failures after a successful demo
- A recognition system succeeds on clean pages. What follows when damaged-page reading becomes the next goal?
- The clean-page result must be denied
- Acknowledge the result and test the additional condition separately
- Assume the new condition has already been solved
- Change the old score without explaining the new data
- Which conclusion is not established by one genuine AI milestone?
- That the measured task improved
- That the recorded test had an outcome
- That the system is suitable for every deployment
- That the result may be worth investigating
- Two benchmark runs use different datasets and scoring rules. What should a comparison disclose?
- Only the higher score
- Only the models’ product names
- Nothing if both reports use the word AI
- The changed conditions rather than assuming the scores are directly comparable
Keep testing yourself
More quizzes picked for your level.