Amazon's Scrapped AI Recruiting Tool Quiz

Answers and explanations appear as you take the quiz.

1. What data was Amazon's recruiting model trained on?
   • Anonymous public job postings from competitors
   • Résumés submitted to Amazon over a ten-year period
   • Synthetic résumés generated to be gender-balanced
   • Performance reviews of current senior executives only
2. Which word did Reuters report the model penalized?
   • "Manager"
   • "Executed"
   • "Captured"
   • "Women's"
3. Why did editing the model to ignore specific gendered terms fail to solve the problem?
   • There was no guarantee the model would not find other proxies to sort candidates in discriminatory ways
   • The edits made the model reject every applicant
   • The law forbade any changes to the model
   • The edits deleted all training data
4. What is the central lesson about the training label in this case?
   • Labels should always be generated at random
   • The model had too few parameters to learn fairly
   • The label reflected past hiring decisions, which contained bias
   • Labels do not affect what a model learns
5. Which verbs did the model reportedly favor because they appeared more often on men's résumés?
   • "Led" and "built"
   • "Executed" and "captured"
   • "Designed" and "coded"
   • "Managed" and "supported"
6. What is the selective labels problem in hiring models?
   • Recruiters choose which labels to delete
   • Models can only read résumés shorter than one page
   • Only certain job titles are allowed as labels
   • Outcomes are observed only for people who were hired, so rejected candidates' potential is never measured
7. Under the four-fifths rule of thumb, what signals possible adverse impact?
   • One group's selection rate is below 80 percent of the highest group's rate
   • More than four-fifths of applicants are rejected
   • A model uses more than five features
   • Fewer than four in five recruiters approve the tool
8. What does New York City's Local Law 144 require of employers using automated employment decision tools?
   • A ban on all AI in hiring
   • Paying candidates who are rejected by software
   • An independent bias audit and a published summary of results
   • Registering every applicant with the city