गाइड से जुड़ी प्रश्नोत्तरी · मध्यम स्तर

Writing Annotation Guidelines for Data Labeling Quiz

Tests understanding of edge cases, guideline structure, versioning, and misconceptions in writing annotation guidelines.

संबंधित मार्गदर्शक पथWriting Annotation Guidelines
प्रश्न 1 का 8

Why does the guide say inconsistently labeled training data is a problem regardless of model architecture?

प्रत्येक प्रश्न का पूर्वावलोकन करें8 प्रश्न

जैसे ही आप प्रश्नोत्तरी लेते हैं उत्तर और स्पष्टीकरण दिखाई देते हैं।

  1. Why does the guide say inconsistently labeled training data is a problem regardless of model architecture?
    • Because the model learns noise instead of the intended concept from the labels it's given
    • Because inconsistent labels always crash the training pipeline
    • Because inconsistent labels increase GPU costs directly
    • Because labeling has no effect on model performance
  2. According to the guide, where does most inter-annotator disagreement originate?
    • From the color scheme of the labeling interface
    • From edge cases not clearly resolved in the guidelines
    • From the total number of labels applied per day
    • From annotators using different keyboards
  3. What does the guide recommend doing before scaling annotation guidelines to a full dataset?
    • Skipping any pilot and labeling everything at once for speed
    • Removing all edge-case examples to keep the document short
    • Piloting the guidelines on a small batch, reviewing disagreements, and revising before scaling up
    • Assigning the entire dataset to a single annotator only
  4. According to the guide, what is the documented misconception about guideline length and consistency?
    • That guidelines should never include examples
    • That short guidelines always cause more errors than long ones
    • That guidelines don't need updating once written
    • That more detailed guidelines always improve consistency, when overly long or conflicting rules can actually reduce it
  5. Why should guideline revisions be versioned and tied to specific data batches?
    • So that if agreement scores drop after a revision, the specific change and affected batch can be identified
    • So annotators can be paid differently per version
    • Because labeling software requires version numbers to function
    • So that the guideline document can be deleted after use
  6. What role do calibration sessions play according to the guide?
    • They replace the need for any written guideline document
    • They let annotators label the same sample and discuss disagreements together, aligning judgment on ambiguous cases
    • They are used only to test annotators' typing speed
    • They are a formal legal requirement for all labeling projects
  7. How does the guide describe the emerging relationship between annotation guidelines and LLMs?
    • LLMs have made annotation guidelines entirely obsolete
    • LLMs cannot use any form of written guideline
    • Guidelines are starting to double as prompts for models doing first-pass labeling, with humans reviewing afterward
    • Annotation guidelines are now only used for training LLMs, never humans
  8. In the named-entity example from the guide, how should a title like 'Dr.' attached to a name be tagged?
    • As a separate title category, distinct from the person entity
    • It should be left unlabeled entirely
    • It should be labeled as an organization entity
    • As part of the person entity, not a separate title category