Ci bi mu melGis bi ci topp
Mbindu Kattañu Kattañu Xët-xëtalu Twitter
Sosiete yi
GUIDE Sosiete
Amazon's AI recruiting tool was an experimental system, begun around 2014, that rated job applicants' résumés from one to five stars by learning from ten years of past résumés.
Déglul tegtal biiIA-nettali · ~3 min
Amazon abandoned it after finding that it penalized résumés associated with women. It is now the standard example of how models trained on historical decisions reproduce past discrimination.
Reuters reported in October 2018 that Amazon had built, and then quietly scrapped, a machine learning tool for ranking job candidates. A team began the work around 2014. The goal was to feed in a stack of résumés and get back the top few candidates to hire. The models were trained on résumés submitted to the company over ten years and learned which patterns tended to go with people who were hired. Most applicants for technical roles were men, so the patterns the model learned were male-coded. By 2015, the team saw that the system was not rating candidates for software developer and other technical jobs in a gender-neutral way. According to Reuters, it penalized résumés containing the word "women's," as in "women's chess club captain," and downgraded graduates of two all-women's colleges. It also favored verbs such as "executed" and "captured," which appeared more often on men's résumés. Engineers edited the models to be neutral to those particular terms. That gave no guarantee the system would not find other ways to sort candidates that were just as discriminatory. The project lost support and the team was disbanded by early 2017. Amazon said the tool was never used by its recruiters to evaluate candidates. Reuters reported that recruiters looked at its recommendations but did not rely on them alone. The key lesson is that the training label was the problem. The model accurately predicted whom Amazon had hired before, and those past hiring decisions contained bias. A common misconception is that the algorithm "became sexist" on its own. In fact it learned exactly what the data rewarded. Another is that removing gender fixes the problem. Text contains many correlated proxies, and a flexible model will find them. The case now comes up in regulatory debates, including New York City's bias audit law for automated hiring tools and the EU AI Act's classification of employment systems as high risk.
A résumé listing "captain of the women's chess club" would score lower, because in the training data the word "women's" rarely appeared on the résumés of people who were hired.
An employer in New York City using an automated tool to screen candidates must commission an independent bias audit and publish a summary under Local Law 144, a requirement shaped by cases like Amazon's.
A vendor strips gender fields and gendered words from a screening model. A disparate impact test then shows women still pass at under four-fifths the rate of men, because of proxies such as hobbies or college names.
A company retrains a ranking model to predict "successful employees" and finds that the label itself reflects biased promotion decisions. It switches to structured, job-related skills assessments as the training target.
Hiring is moving from keyword filters to large language models that summarize and rank candidates, and this raises the same risk at larger scale. Regulators are responding. New York City requires bias audits for automated employment decision tools, and the EU AI Act classifies recruitment systems as high risk, with obligations being phased in. In the US, existing anti-discrimination law such as Title VII still applies to algorithmic tools, even as federal agency guidance has shifted. Audit methods are still maturing, and a passing audit on one dataset does not guarantee fairness elsewhere. The lasting lesson of the Amazon case is to question what a model is trained to predict before trusting how well it predicts it.
Bulletin bu amul fayda
Ñatti jaar-jaari IA yuñ firndeel bis bu nekk ci ayu-bis bi, ñu bind leen ci Àngle bu leer. Gratis ba fàww, amul yëgle.
Benn email bis bu nekk ci ayu-bis bi. Benn klik nga dindi abonemaa bi. Dunu musa jaay wala séddoo sa adres.
Nattal sa bopp
Feedback ci saasi ci bépp tontu, ak sertifikaa buñ mëna séddoo ak ID buñ mëna saytu soo jàllee ci njàng mi.
Jàppale njàngum IA te doo fay. AI Understanding mbootaay la buy def te Yàlla tax — amul yëgle, amul paywall, musul am. Defal ab sarax
Amazon's AI recruiting tool was an experimental system, begun around 2014, that rated job applicants' résumés from one to five stars by learning from ten years of past résumés. Amazon abandoned it after finding that it penalized résumés associated with women. It is now the standard example of how models trained on historical decisions reproduce past discrimination.
The model learned from about ten years of résumés submitted to Amazon, and most applicants for technical roles were men.
Résumés containing "women's," as in "women's chess club captain," were downgraded. "Executed" and "captured" were favored, not penalized.
Tëj ay baat yu néew du dindi ay signaux yu bokk ci biir téere bi, kon model bi mën na ba tay bàyyi ay wëlbëti-wëlbëti yeneen yoon.
Model bi dafa waxoon koo xam ne dañu ko jëloon liggéey bi bu jëkk. Ndaxte liggéey bu jëkk bi dafa amoon ay wëlbëti-wëlbëti, label bi dafa dëgëral wëlbëti-wëlbëti boobu te model bi reproduire ko.
Reuters dafa wax ne model bi dafa gën a bëgg "executed" ak "captured," yu gën a bari ci résumés góor ñi, loolu mooy misaal bu nekk ci proxy features yu neew.
Weyal di jàng
Tann nañu yeneen njiit ngir topic bii
Ci bi mu melGis bi ci topp
Mbindu Kattañu Kattañu Xët-xëtalu Twitter
Sosiete yi