Bài kiểm tra có liên kết hướng dẫn · Trung bình Cấp độ
Bài kiểm tra về sự không đồng ý của người chú thích và sự thiên vị của người chú thích
Kiểm tra các nhãn chủ quan, nhận dạng và hướng dẫn của người chú thích, mục tiêu ủy quyền, sự bất đồng và các cách an toàn hơn để duy trì sự không chắc chắn trong dữ liệu đào tạo.
Câu hỏi 1 của 8
A model learns “toxic” labels that reflect annotators’ assumptions about dialect. What risk is most direct?
Xem trước mọi câu hỏi8 câu hỏi
Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.
- A model learns “toxic” labels that reflect annotators’ assumptions about dialect. What risk is most direct?
- Label bias
- Sampling latency
- Hardware failure
- Nén hình ảnh
- What did Sap and colleagues find in research on toxicity labels and African American English?
- Dialect markers were unrelated to annotation
- Annotator identity and beliefs were associated with toxicity ratings in their studies
- All annotators agreed on every post
- The model never used the training labels
- Why may disagreement on sarcasm or offensive language be meaningful?
- It always proves the dataset is corrupt
- It only occurs with missing data
- The task can be subjective and people may interpret context differently
- A model cannot accept labels
- What can happen when a team turns every disagreement into one majority label?
- Minority perspectives and uncertainty may be erased
- The model automatically becomes more fair
- The dataset becomes representative
- The labels become legally binding
- Why is health-care spending a problematic proxy for medical need?
- Spending is always random
- Past spending can reflect access barriers and unequal care
- Costs are never recorded
- Models cannot use numerical values
- Which practice helps audit label provenance?
- Store only the final majority label
- Delete annotator instructions
- Preserve individual annotations and instructions before aggregation
- Replace labels with model predictions
- When may a majority label be insufficient?
- When the application needs to represent ambiguity or multiple perspectives
- When the dataset has a file name
- When every judgment agrees
- When the model is deployed offline
- How should teams interpret low annotator agreement?
- As automatic proof of bad annotators
- As a possible sign of a subjective or underspecified task that needs investigation
- As proof the model is accurate
- As a reason to hide the labels