Guide-linked quiz · Medium Level
Dialect Bias and African American English in NLP Quiz
Tests AAE as a systematic dialect, evidence from toxicity and LLM studies, and limits on generalizing findings across speakers and tasks.
Question 1 of 8
How should AAE be described in an NLP evaluation?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- How should AAE be described in an NLP evaluation?
- As a single writing error shared by all Black people
- As a systematic, internally varied English dialect, not incorrect English
- As a language used only online
- As a reliable indicator of a person’s race
- What did Sap et al. find about toxicity datasets that included AAE tweets?
- AAE surface markers correlated with toxicity labels in several datasets
- AAE was always correctly identified by annotators
- Every AAE tweet was labeled harmless
- The study tested only speech-to-text systems
- In the Sap et al. study, how did models trained on those corpora treat AAE tweets in the tested data?
- They were less likely to be labeled offensive in all cases
- They were up to twice as likely to be labeled offensive compared with other tweets
- They were always excluded from moderation
- The model never used training labels
- What did dialect priming do for human annotators in the ACL study?
- It eliminated dialect variation
- It made every annotation identical
- Making annotators aware of AAE reduced the likelihood they labeled a tweet offensive
- It increased toxicity ratings
- What did matched-guise probing compare in the 2024 Nature study?
- Different medical diagnoses for people of different ages
- Two speech microphones in a noisy room
- Only offensive words and non-offensive words
- Model responses to comparable content expressed in AAE and Standard American English
- How many model versions did the 2024 Nature study examine?
- Twelve
- Two
- One hundred and fifteen
- Seven hundred and fifty thousand
- Which scope limitation applies to generalizing the dialect-bias results?
- The studies included every English dialect worldwide
- Every Black person uses the same dialect features
- AAE is a language spoken only by one age group
- AAE is not used by all Black people, and the study’s dialect samples do not represent every speaker or context
- What should a developer do when evaluating dialect bias?
- Rely only on an overall accuracy score
- Automatically rewrite every dialect into SAE
- Use meaning-matched tests, measure task-specific errors, and involve affected speakers
- Infer race from a dialect marker
Keep testing yourself
More quizzes picked for your level.