引導式測驗 · 中等 等級

自然語言處理中的方言偏見與非裔美國英語測驗

將 AAE 作為一種系統方言進行測試,來自毒性和法學碩士研究的證據,以及對跨演講者和任務的概括結果的限制。

問題 1 的 8

How should AAE be described in an NLP evaluation?

預覽每個問題8 問題

當您參加測驗時,就會出現答案和解釋。

  1. How should AAE be described in an NLP evaluation?
    • As a single writing error shared by all Black people
    • As a systematic, internally varied English dialect, not incorrect English
    • As a language used only online
    • As a reliable indicator of a person’s race
  2. What did Sap et al. find about toxicity datasets that included AAE tweets?
    • AAE surface markers correlated with toxicity labels in several datasets
    • AAE was always correctly identified by annotators
    • Every AAE tweet was labeled harmless
    • The study tested only speech-to-text systems
  3. In the Sap et al. study, how did models trained on those corpora treat AAE tweets in the tested data?
    • They were less likely to be labeled offensive in all cases
    • They were up to twice as likely to be labeled offensive compared with other tweets
    • They were always excluded from moderation
    • The model never used training labels
  4. What did dialect priming do for human annotators in the ACL study?
    • It eliminated dialect variation
    • It made every annotation identical
    • Making annotators aware of AAE reduced the likelihood they labeled a tweet offensive
    • It increased toxicity ratings
  5. What did matched-guise probing compare in the 2024 Nature study?
    • Different medical diagnoses for people of different ages
    • Two speech microphones in a noisy room
    • Only offensive words and non-offensive words
    • Model responses to comparable content expressed in AAE and Standard American English
  6. How many model versions did the 2024 Nature study examine?
    • Twelve
    • 兩個
    • One hundred and fifteen
    • Seven hundred and fifty thousand
  7. Which scope limitation applies to generalizing the dialect-bias results?
    • The studies included every English dialect worldwide
    • Every Black person uses the same dialect features
    • AAE is a language spoken only by one age group
    • AAE is not used by all Black people, and the study’s dialect samples do not represent every speaker or context
  8. What should a developer do when evaluating dialect bias?
    • Rely only on an overall accuracy score
    • Automatically rewrite every dialect into SAE
    • Use meaning-matched tests, measure task-specific errors, and involve affected speakers
    • Infer race from a dialect marker