引導式測驗 · 中等 等級

語言模型中的性別偏見測驗

測試性別關聯如何在嵌入、翻譯、共指和模擬招募中出現,同時將結果限制在特定研究範圍內。

問題 1 的 8

What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
    • First names signaling race and gender while qualifications were held comparable
    • Only candidates’ university grades
    • Only the model’s temperature
    • Different job descriptions with no candidate information
  2. Which two models did that study test?
    • Only GPT-4o
    • BERT and ELIZA
    • GPT-3.5-Turbo and Llama 3-70B-Instruct
    • Whisper and DeepSpeech
  3. What limitation should readers keep in mind about the study’s name-based findings?
    • It measured only real-world job offers
    • The prompt signals combined race and gender, and simulated recommendations do not prove deployed employer behavior
    • It tested every hiring system used worldwide
    • It included no names
  4. What does the WinoBias benchmark evaluate?
    • Translation quality for numbers
    • Speech-recognition accuracy by accent
    • Coreference errors on stereotypical and anti-stereotypical occupation sentences
    • Age-estimation error in faces
  5. How can machine translation introduce a gendered assumption?
    • It always translates every noun into a verb
    • It can add a gendered pronoun when the source language does not specify gender
    • 它刪除所有標點符號
    • It converts spoken words into images
  6. Why can removing every gender association from an embedding be problematic?
    • It makes word vectors impossible to store
    • It guarantees model bias increases
    • It removes all occupations from a vocabulary
    • Some gender information is relevant to meaning or identity and cannot be cleanly separated from stereotypes
  7. Which evaluation design best tests whether a gender cue changes a recommendation?
    • Test one output from one model
    • Compare unrelated people in different jobs
    • Change gender, education, salary and occupation together
    • Hold qualifications and task constant while varying the cue in repeated prompts
  8. What does a lower stereotype score on one benchmark establish?
    • Only performance on that benchmark and setup, not fairness across every product use
    • That translation quality is perfect
    • 合法合規的保證
    • That all gender bias has been removed