SeterusnyaPanduan seterusnya
COMPAS dan Bias dalam Algoritma Recidivism
Masyarakat
PANDUAN Masyarakat
Content-moderation systems classify posts for review or removal, and their mistakes can fall unevenly across language varieties and topics.
Dengar panduan iniAI-diceritakan · ~3 min
Bias can enter through training examples, human labels, policy definitions, thresholds and appeal processes; a high overall accuracy score does not show whether false positives or false negatives cluster for a particular community.
Automated moderation usually combines classifiers, rules, user reports and human review. Classifiers may estimate whether text is toxic, hateful, spam or otherwise against a platform policy; their output is evidence for a decision, not the policy itself. Errors have different consequences: a false positive can hide benign speech or silence a user, while a false negative can leave abuse visible. Context matters, including quotation, counterspeech, satire, reclaimed terms and dialect. A well-studied source of bias is dataset labeling. Sap and colleagues’ 2019 ACL study found that AAE surface markers correlated with toxicity ratings in several widely used hate-speech datasets. Models trained on those datasets then labeled AAE tweets and tweets by self-identified Black authors as offensive up to twice as often in the tested material. Telling human annotators that a tweet used AAE reduced offensive ratings. A later ACL study on toxicity detection likewise examined how identity terms and annotator disagreement can affect models. These results are scoped to particular datasets, annotators and systems; they do not show that every moderation tool discriminates or that any dialect feature is itself harmful. A fair review therefore asks what “harmful” means in a published policy, who labeled examples, which contexts are represented, and how a score becomes an account action. Error thresholds, language coverage, human escalation and appeal procedures all shape outcomes. Platforms should report subgroup results, review policy examples with affected communities and preserve a route for users to challenge mistakes. Moderation is a sociotechnical decision process, not merely a model accuracy problem.
A platform checks whether toxicity scores change when a post is expressed in African American English (AAE) or Standard American English while keeping meaning similar.
A moderation team audits reclaimed identity terms and context instead of treating a keyword list as proof of abuse.
A researcher separates human annotation disagreement from model errors when reviewing a toxicity dataset.
A platform tracks removal and appeal outcomes by language variety, then changes its workflow if one group bears an unusual false-positive burden.
As platforms deploy multilingual and generative moderation tools, evaluation will need to include more language varieties, local contexts and user appeals. Publish performance and enforcement measures by task and language where privacy permits. Keep the limits of each benchmark visible, because policy choices and community expectations can change. Review the primary records again before describing a current system, since operating status and legal remedies can change. For research claims, revisit the original methods, sample, annotation procedure, comparison group, and publication corrections. A measured disparity in one dataset should prompt targeted testing, not a universal claim about every model or affected population.
Newsletter percuma
Tiga cerita AI yang disahkan setiap pagi hari bekerja, ditulis dalam Bahasa Inggeris biasa. Percuma selamanya, tiada iklan.
Satu e-mel setiap hari bekerja. Berhenti melanggan dalam satu klik. Kami tidak pernah menjual atau berkongsi alamat anda.
Uji diri anda
Maklum balas segera pada setiap jawapan, dan sijil yang boleh dikongsi dengan ID yang boleh disahkan sebaik sahaja anda lulus kursus.
Sokong pendidikan AI percuma. AI Understanding ialah pertubuhan bukan keuntungan 501(c)(3) — tiada iklan, tiada bayaran dinding, selamanya. Buat derma
Content-moderation systems classify posts for review or removal, and their mistakes can fall unevenly across language varieties and topics. Bias can enter through training examples, human labels, policy definitions, thresholds and appeal processes; a high overall accuracy score does not show whether false positives or false negatives cluster for a particular community.
A false positive is content that should not trigger the action but is flagged by the system.
The authors found correlations in datasets and that trained models propagated the association in their evaluation.
The paper reports AAE tweets and tweets by self-identified Black authors were up to twice as likely to be labeled offensive in the tested material.
Annotators were less likely to label a tweet offensive when told it used AAE.
The same term can be abusive, quoted or reclaimed, so context affects whether content violates policy.
Teruskan belajar
Lebih banyak panduan dipilih untuk topik ini
SeterusnyaPanduan seterusnya
COMPAS dan Bias dalam Algoritma Recidivism
Masyarakat