Guide-linked quiz · Medium Level

Gender Bias in Language Models Quiz

Tests how gender associations appear in embeddings, translation, coreference and simulated hiring, while keeping results scoped to specific studies.

Question 1 of 8

What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
    • First names signaling race and gender while qualifications were held comparable
    • Only candidates’ university grades
    • Only the model’s temperature
    • Different job descriptions with no candidate information
  2. Which two models did that study test?
    • Only GPT-4o
    • BERT and ELIZA
    • GPT-3.5-Turbo and Llama 3-70B-Instruct
    • Whisper and DeepSpeech
  3. What limitation should readers keep in mind about the study’s name-based findings?
    • It measured only real-world job offers
    • The prompt signals combined race and gender, and simulated recommendations do not prove deployed employer behavior
    • It tested every hiring system used worldwide
    • It included no names
  4. What does the WinoBias benchmark evaluate?
    • Translation quality for numbers
    • Speech-recognition accuracy by accent
    • Coreference errors on stereotypical and anti-stereotypical occupation sentences
    • Age-estimation error in faces
  5. How can machine translation introduce a gendered assumption?
    • It always translates every noun into a verb
    • It can add a gendered pronoun when the source language does not specify gender
    • It removes all punctuation
    • It converts spoken words into images
  6. Why can removing every gender association from an embedding be problematic?
    • It makes word vectors impossible to store
    • It guarantees model bias increases
    • It removes all occupations from a vocabulary
    • Some gender information is relevant to meaning or identity and cannot be cleanly separated from stereotypes
  7. Which evaluation design best tests whether a gender cue changes a recommendation?
    • Test one output from one model
    • Compare unrelated people in different jobs
    • Change gender, education, salary and occupation together
    • Hold qualifications and task constant while varying the cue in repeated prompts
  8. What does a lower stereotype score on one benchmark establish?
    • Only performance on that benchmark and setup, not fairness across every product use
    • That translation quality is perfect
    • A guarantee of legal compliance
    • That all gender bias has been removed

Keep testing yourself

More quizzes picked for your level.

View all quizzes