引導式測驗 · 中等 等級
語言模型中的性別偏見測驗
測試性別關聯如何在嵌入、翻譯、共指和模擬招募中出現,同時將結果限制在特定研究範圍內。
問題 1 的 8
What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
預覽每個問題8 問題
當您參加測驗時,就會出現答案和解釋。
- What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
- First names signaling race and gender while qualifications were held comparable
- Only candidates’ university grades
- Only the model’s temperature
- Different job descriptions with no candidate information
- Which two models did that study test?
- Only GPT-4o
- BERT and ELIZA
- GPT-3.5-Turbo and Llama 3-70B-Instruct
- Whisper and DeepSpeech
- What limitation should readers keep in mind about the study’s name-based findings?
- It measured only real-world job offers
- The prompt signals combined race and gender, and simulated recommendations do not prove deployed employer behavior
- It tested every hiring system used worldwide
- It included no names
- What does the WinoBias benchmark evaluate?
- Translation quality for numbers
- Speech-recognition accuracy by accent
- Coreference errors on stereotypical and anti-stereotypical occupation sentences
- Age-estimation error in faces
- How can machine translation introduce a gendered assumption?
- It always translates every noun into a verb
- It can add a gendered pronoun when the source language does not specify gender
- 它刪除所有標點符號
- It converts spoken words into images
- Why can removing every gender association from an embedding be problematic?
- It makes word vectors impossible to store
- It guarantees model bias increases
- It removes all occupations from a vocabulary
- Some gender information is relevant to meaning or identity and cannot be cleanly separated from stereotypes
- Which evaluation design best tests whether a gender cue changes a recommendation?
- Test one output from one model
- Compare unrelated people in different jobs
- Change gender, education, salary and occupation together
- Hold qualifications and task constant while varying the cue in repeated prompts
- What does a lower stereotype score on one benchmark establish?
- Only performance on that benchmark and setup, not fairness across every product use
- That translation quality is perfect
- 合法合規的保證
- That all gender bias has been removed