Guide-linked quiz · Medium Level
Model-Assisted Pre-Labeling Quiz
Assess draft-label review, anchoring risks, confidence-based triage limits, fair workflow comparisons, and human checks for model errors.
Question 1 of 8
In a model-assisted pre-labeling workflow, what does the human annotator's job shift toward?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- In a model-assisted pre-labeling workflow, what does the human annotator's job shift toward?
- Verifying and correcting the model's draft labels instead of creating labels from scratch
- Writing the model's training code
- Only labeling data the model refuses to touch
- Deleting all model predictions before starting
- According to the guide, what is the main risk that anchoring bias introduces into a pre-labeling pipeline?
- The model always predicts every item correctly
- Reviewers can over-rely on the suggested label instead of independently judging the item
- Pre-labeling removes the need for guidelines
- Annotators become unable to use the interface
- How do some pipelines use model confidence scores to manage reviewer workload?
- They ignore confidence scores entirely
- They discard all predictions below 100% confidence
- They route low-confidence predictions for closer review and let high-confidence ones get lighter scrutiny
- They only compute confidence scores after training is finished
- Why might a pipeline deliberately hide the model's confidence score from a reviewer?
- To save on computing costs
- Because confidence scores are illegal to display
- Because hiding scores speeds up the model's inference
- To reduce anchoring bias by encouraging independent judgment rather than trust in the model's certainty
- According to the guide, what is one way teams measure whether pre-labeling has actually preserved accuracy rather than just raising throughput?
- Comparing error rates between a pre-labeled review workflow and a from-scratch labeling workflow on the same held-out sample
- Counting how many labels were produced per hour only
- Asking the model to grade its own predictions
- Measuring how long the software takes to load
- According to the guide, what is a common pattern in how companies build early pre-labeling models?
- They avoid training any model until the dataset is fully complete
- They train an earlier, weaker model specifically so it can pre-label the next batch, creating an iterative improvement loop
- They only use pre-labeling for audio data
- They use a single frozen model forever with no updates
- In the invoice-processing example, what does the OCR-plus-layout model do before a human clerk gets involved?
- It deletes incorrect invoices automatically
- It sends the invoice directly to payment without review
- It pre-fills fields like invoice number and total for the clerk to verify or correct
- It only extracts the company logo
- Why is anchoring bias described as not unique to data annotation?
- Because it only happens with audio data
- Because it was invented specifically for AI pipelines
- Because it only affects computer systems, not people
- Because it is a well-documented pattern in human decision-making generally, seen in many judgment contexts
Keep testing yourself
More quizzes picked for your level.