Dialect Bias and African American English in NLP

African American English (AAE) is a rule-governed language variety with systematic grammatical, phonological and lexical patterns. Features can include habitual “be” to indicate recurring action, but AAE varies by speaker, region, context and style. Not all Black Americans use AAE, and AAE use does not identify a person’s race. NLP systems can still encode racialized judgments when training data or annotation practices conflate dialect markers with toxicity, low competence or nonstandard writing. Sap and colleagues’ 2019 ACL study found correlations between AAE surface markers and toxicity labels in several hate-speech datasets. Models trained on those datasets learned the association; AAE tweets and tweets by self-identified Black authors were up to twice as likely to be labeled offensive in the tested material. When annotators were explicitly told that a tweet used AAE, they were less likely to label it offensive. This is evidence about specific datasets and annotation conditions, not every moderation system or every AAE speaker. A 2024 Nature study introduced matched-guise probing: it compared model reactions to equivalent content expressed in AAE and Standard American English. Across 12 examined model versions, researchers found covert negative stereotypes in hypothetical judgments about character, employability and criminality. The study deliberately tests a stress case and does not show that a particular employer or court actually used these model judgments. Still, it demonstrates that a system can return polite surface language while assigning different hidden judgments. Speech recognition, toxicity classification, translation and text generation are different tasks, so each needs its own dialect-aware evaluation. Avoid “correcting” dialect by default; let the user control register and preserve meaning.