GUIDE Sosiete

Jumtukaayu jël ay ndaw ci IA bu Amazon

Amazon's AI recruiting tool was an experimental system, begun around 2014, that rated job applicants' résumés from one to five stars by learning from ten years of past résumés.

Déglul tegtal biiIA-nettali · ~3 min

Ci xët wii4 simili jàng
  1. Résumé
  2. Plongeur bu xóot
  3. Doxal ci àdduna dëgg
  4. The Future of Amazon's Scrapped AI Recruiting Tool
  5. Teg ko ci jëf
  6. Weyal di banneexu
  7. Laaj yi ñuy faral di laaj

Résumé

Amazon abandoned it after finding that it penalized résumés associated with women. It is now the standard example of how models trained on historical decisions reproduce past discrimination.

Plongeur bu xóot

Reuters reported in October 2018 that Amazon had built, and then quietly scrapped, a machine learning tool for ranking job candidates. A team began the work around 2014. The goal was to feed in a stack of résumés and get back the top few candidates to hire. The models were trained on résumés submitted to the company over ten years and learned which patterns tended to go with people who were hired. Most applicants for technical roles were men, so the patterns the model learned were male-coded. By 2015, the team saw that the system was not rating candidates for software developer and other technical jobs in a gender-neutral way. According to Reuters, it penalized résumés containing the word "women's," as in "women's chess club captain," and downgraded graduates of two all-women's colleges. It also favored verbs such as "executed" and "captured," which appeared more often on men's résumés. Engineers edited the models to be neutral to those particular terms. That gave no guarantee the system would not find other ways to sort candidates that were just as discriminatory. The project lost support and the team was disbanded by early 2017. Amazon said the tool was never used by its recruiters to evaluate candidates. Reuters reported that recruiters looked at its recommendations but did not rely on them alone. The key lesson is that the training label was the problem. The model accurately predicted whom Amazon had hired before, and those past hiring decisions contained bias. A common misconception is that the algorithm "became sexist" on its own. In fact it learned exactly what the data rewarded. Another is that removing gender fixes the problem. Text contains many correlated proxies, and a flexible model will find them. The case now comes up in regulatory debates, including New York City's bias audit law for automated hiring tools and the EU AI Act's classification of employment systems as high risk.

Doxal ci àdduna dëgg

A résumé listing "captain of the women's chess club" would score lower, because in the training data the word "women's" rarely appeared on the résumés of people who were hired.

An employer in New York City using an automated tool to screen candidates must commission an independent bias audit and publish a summary under Local Law 144, a requirement shaped by cases like Amazon's.

A vendor strips gender fields and gendered words from a screening model. A disparate impact test then shows women still pass at under four-fifths the rate of men, because of proxies such as hobbies or college names.

A company retrains a ranking model to predict "successful employees" and finds that the label itself reflects biased promotion decisions. It switches to structured, job-related skills assessments as the training target.

The Future of Amazon's Scrapped AI Recruiting Tool

Hiring is moving from keyword filters to large language models that summarize and rank candidates, and this raises the same risk at larger scale. Regulators are responding. New York City requires bias audits for automated employment decision tools, and the EU AI Act classifies recruitment systems as high risk, with obligations being phased in. In the US, existing anti-discrimination law such as Title VII still applies to algorithmic tools, even as federal agency guidance has shifted. Audit methods are still maturing, and a passing audit on one dataset does not guarantee fairness elsewhere. The lasting lesson of the Amazon case is to question what a model is trained to predict before trusting how well it predicts it.

Teg ko ci jëf

Lu tax mu am solo

  • Risk ak kaaraange. Gaañ-gaañu IA yu mag yi ak yu bës bu nekk yépp a ngi aju ci ki xam risk yi ak ki mëna def dara.
  • dogal yu gëna leer. Liggéeyukaay ak xam-xam bu ñépp bokk mooy wane ndax politiku kaaraange bu dëgër mën na am ci wàllu politik.
  • Dagg ci hype. Faram-fàcce yu leer dañuy wàññi li ñuy jàpp ci hype, PR lab, ak tiyaatar bu leerul.

Moytul

  • Jàppale risku nekk gi ni siyaas fiksioŋ fekk kàttan gi dafay yokk.
  • Jaxasoo kaaraange produit surface ak jubluwaay ci suufu autonomie bu kawe.
  • Bàyyi nit ñi xamul làkku Àngle ak ñi xamul làkku Angale, ñu am balluwaay yu baaxul.

Jéego yi ci topp

  1. Tàqale loraange yi ci produit bi, jëfandikoo bu baaxul, ak risku ñàkka mëna yor / ñàkka méngoo.
  2. Laajteel ban firnde mooy soppi sa xalaat ci kalendriye yi ak tar gi.
  3. Danga taamu balluwaay yu njëkk yi ak jàngat yu fëgër yi moo gën waxtaanu njaay mi.
  4. Xaarandil benn yoonu jëf: liggéey, politik, xaalis, wala xam-xam — du xam-xam kese.

Weyal di banneexu

Bulletin bu amul fayda

Jàngal xibaar AI bu bes

Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.

Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.

Nattal sa bopp

Take the Amazon's Scrapped AI Recruiting Tool quiz

Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.

Tambalil quiz

Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax

Laaj yi ñuy faral di laaj

What is Amazon's Scrapped AI Recruiting Tool?

Amazon's AI recruiting tool was an experimental system, begun around 2014, that rated job applicants' résumés from one to five stars by learning from ten years of past résumés. Amazon abandoned it after finding that it penalized résumés associated with women. It is now the standard example of how models trained on historical decisions reproduce past discrimination.

What data was Amazon's recruiting model trained on?

The model learned from about ten years of résumés submitted to Amazon, and most applicants for technical roles were men.

Which word did Reuters report the model penalized?

Résumés containing "women's," as in "women's chess club captain," were downgraded. "Executed" and "captured" were favored, not penalized.

Why did editing the model to ignore specific gendered terms fail to solve the problem?

Tëj ay baat yu néew du dindi ay signaux yu bokk ci biir téere bi, kon model bi mën na ba tay bàyyi ay wëlbëti-wëlbëti yeneen yoon.

Lan mooy njàngat bi gën a am solo ci li jëm ci taggat label bi ci biir kiimtaan bi?

Model bi dafa waxoon koo xam ne dañu ko jëloon liggéey bi bu jëkk. Ndaxte liggéey bu jëkk bi dafa amoon ay wëlbëti-wëlbëti, label bi dafa dëgëral wëlbëti-wëlbëti boobu te model bi reproduire ko.

Yanu verb la model bi gën a bëgg ndaxte dañu gën a feeñ ci résumés góor ñi?

Reuters dafa wax ne model bi dafa gën a bëgg "executed" ak "captured," yu gën a bari ci résumés góor ñi, loolu mooy misaal bu nekk ci proxy features yu neew.