Bài kiểm tra có liên kết hướng dẫn · Trung bình Cấp độ

Câu đố về Kappa của Cohen

Khám phá sự đồng thuận được quan sát và mong đợi, độ tin cậy của kappa có trọng số và chú thích thông qua các ví dụ hoạt động về nhãn phân loại.

Đường dẫn hướng dẫn liên quanCohens Kappa
Câu hỏi 1 của 8

Two annotators have observed agreement of 0.8 and expected agreement of 0.5. Which kappa value follows?

Xem trước mọi câu hỏi8 câu hỏi

Câu trả lời và giải thích xuất hiện khi bạn làm bài kiểm tra.

  1. Two annotators have observed agreement of 0.8 and expected agreement of 0.5. Which kappa value follows?
    • 0.3
    • 0.4
    • 0.8
    • 0.6
  2. One annotator uses a category 60% of the time and the other uses it 50% of the time. How much does that category contribute to expected agreement?
    • 0.10
    • 0.30
    • 0.55
    • 1.10
  3. Why does the expected-agreement term in kappa not establish that annotators were guessing randomly?
    • It is calculated from the number of disagreements alone
    • It is identical to the observed agreement for every dataset
    • It is a reference derived from label frequencies, not an observation of their reasoning
    • It assumes every category must have equal frequency
  4. For low, medium and high incident severity, why might a team choose weighted kappa?
    • To give different penalties to one-level and two-level disagreements
    • To remove all disagreements from the calculation
    • To guarantee a higher reliability score
    • To avoid specifying a category order
  5. Both annotators assign every case to the same single category. Why should ordinary kappa not be reported as a straightforward reliability success?
    • Perfect raw agreement automatically makes kappa one
    • The score ignores the common category and evaluates only disagreements
    • Subtracting expected agreement alone gives a complete kappa calculation
    • Expected agreement is one, leaving a zero denominator
  6. A second reviewer copies the first reviewer's labels and achieves perfect agreement. Which conclusion is justified?
    • The labels have been independently validated
    • The agreement does not establish independent reliability
    • Every label is necessarily correct
    • The dataset can be used without checking instructions
  7. Which accompanying information helps interpret a kappa score when category frequencies are uneven?
    • Only the raw agreement percentage, without category counts
    • Only whether kappa exceeds a universal cutoff
    • Sample size, raw agreement and the agreement table
    • Only the weighted score, without specifying category order
  8. Two annotators both follow a mistaken rule and agree strongly. What does their high kappa fail to establish?
    • That their labels are factually correct
    • That their observed agreement exceeds its frequency-based expectation
    • That they used the same labels on many cases
    • That their assignments show consistency under the shared rule