Guide-linked quiz · Medium Level

Dialect Bias and African American English in NLP Quiz

Tests AAE as a systematic dialect, evidence from toxicity and LLM studies, and limits on generalizing findings across speakers and tasks.

Question 1 of 8

How should AAE be described in an NLP evaluation?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. How should AAE be described in an NLP evaluation?
    • As a single writing error shared by all Black people
    • As a systematic, internally varied English dialect, not incorrect English
    • As a language used only online
    • As a reliable indicator of a person’s race
  2. What did Sap et al. find about toxicity datasets that included AAE tweets?
    • AAE surface markers correlated with toxicity labels in several datasets
    • AAE was always correctly identified by annotators
    • Every AAE tweet was labeled harmless
    • The study tested only speech-to-text systems
  3. In the Sap et al. study, how did models trained on those corpora treat AAE tweets in the tested data?
    • They were less likely to be labeled offensive in all cases
    • They were up to twice as likely to be labeled offensive compared with other tweets
    • They were always excluded from moderation
    • The model never used training labels
  4. What did dialect priming do for human annotators in the ACL study?
    • It eliminated dialect variation
    • It made every annotation identical
    • Making annotators aware of AAE reduced the likelihood they labeled a tweet offensive
    • It increased toxicity ratings
  5. What did matched-guise probing compare in the 2024 Nature study?
    • Different medical diagnoses for people of different ages
    • Two speech microphones in a noisy room
    • Only offensive words and non-offensive words
    • Model responses to comparable content expressed in AAE and Standard American English
  6. How many model versions did the 2024 Nature study examine?
    • Twelve
    • Two
    • One hundred and fifteen
    • Seven hundred and fifty thousand
  7. Which scope limitation applies to generalizing the dialect-bias results?
    • The studies included every English dialect worldwide
    • Every Black person uses the same dialect features
    • AAE is a language spoken only by one age group
    • AAE is not used by all Black people, and the study’s dialect samples do not represent every speaker or context
  8. What should a developer do when evaluating dialect bias?
    • Rely only on an overall accuracy score
    • Automatically rewrite every dialect into SAE
    • Use meaning-matched tests, measure task-specific errors, and involve affected speakers
    • Infer race from a dialect marker

Keep testing yourself

More quizzes picked for your level.

View all quizzes