Guide-linked quiz · Medium Level
Gender Bias in Language Models Quiz
Tests how gender associations appear in embeddings, translation, coreference and simulated hiring, while keeping results scoped to specific studies.
Question 1 of 8
What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- What did the 2024 EMNLP employment-recommendation study vary across its candidate prompts?
- First names signaling race and gender while qualifications were held comparable
- Only candidates’ university grades
- Only the model’s temperature
- Different job descriptions with no candidate information
- Which two models did that study test?
- Only GPT-4o
- BERT and ELIZA
- GPT-3.5-Turbo and Llama 3-70B-Instruct
- Whisper and DeepSpeech
- What limitation should readers keep in mind about the study’s name-based findings?
- It measured only real-world job offers
- The prompt signals combined race and gender, and simulated recommendations do not prove deployed employer behavior
- It tested every hiring system used worldwide
- It included no names
- What does the WinoBias benchmark evaluate?
- Translation quality for numbers
- Speech-recognition accuracy by accent
- Coreference errors on stereotypical and anti-stereotypical occupation sentences
- Age-estimation error in faces
- How can machine translation introduce a gendered assumption?
- It always translates every noun into a verb
- It can add a gendered pronoun when the source language does not specify gender
- It removes all punctuation
- It converts spoken words into images
- Why can removing every gender association from an embedding be problematic?
- It makes word vectors impossible to store
- It guarantees model bias increases
- It removes all occupations from a vocabulary
- Some gender information is relevant to meaning or identity and cannot be cleanly separated from stereotypes
- Which evaluation design best tests whether a gender cue changes a recommendation?
- Test one output from one model
- Compare unrelated people in different jobs
- Change gender, education, salary and occupation together
- Hold qualifications and task constant while varying the cue in repeated prompts
- What does a lower stereotype score on one benchmark establish?
- Only performance on that benchmark and setup, not fairness across every product use
- That translation quality is perfect
- A guarantee of legal compliance
- That all gender bias has been removed
Keep testing yourself
More quizzes picked for your level.