Guide-linked quiz · Medium Level
CUAD and Legal NLP Datasets Quiz
Tests CUAD’s corpus and labels, its benchmark scope, licensing and dataset limits, split design, evaluation, and legal-review boundaries.
Question 1 of 8
How does the Atticus Project describe CUAD v1?
Preview every question8 questions
Answers and explanations appear as you take the quiz.
- How does the Atticus Project describe CUAD v1?
- 510 commercial contracts with label metadata but no manually annotated clause spans
- 510 commercial contracts with 13,000-plus labels across 41 clause types
- A corpus of court opinions tagged with contract-law doctrines
- A dataset of full-contract risk scores without annotated clause spans
- Which task was CUAD created to support?
- Locating salient contract language for human review
- Classifying agreements as enforceable or unenforceable across jurisdictions
- Extracting party names and dates only, without clause-level text
- Predicting which contract disputes will be filed in court
- A model scores highly on CUAD. What does that establish most directly?
- That the model legally interprets all 41 clause types under current law
- That it generalizes to every contract type and jurisdiction
- Performance on the benchmark task and data under the reported evaluation setup
- That an expert-labelled score clears the model for transaction deployment
- Why examine the split design before using CUAD for evaluation?
- A train/test split makes template overlap irrelevant to generalization
- Related templates or documents across splits can make results look better than generalization warrants
- The benchmark cannot be trained on when labels are present in the test set
- Split design alone establishes that results apply to new contract populations
- A lawyer wants to use a CUAD-trained model on a non-disclosure agreement in another jurisdiction. What is the best next step?
- Evaluate only common CUAD clauses and assume they represent NDA language
- Use a held-out CUAD split from the same source and treat it as proof of jurisdictional transfer
- Change the jurisdiction metadata and reuse the original score without testing
- Evaluate on representative agreements from that intended setting
- Why report results by clause category as well as one aggregate score?
- A single aggregate is sufficient when the 41 labels have balanced counts
- An aggregate can hide that extraction works differently across label types
- A score for each clause type by itself guarantees transfer to new agreements
- Category breakdowns are needed only when overall accuracy falls below chance
- What should a researcher check before redistributing CUAD data?
- The dataset version and license terms stated by its maintainers
- Assume a top-level code license covers every dataset and annotation file
- Treat public availability as permission to redistribute without checking conditions
- Use the model’s license as the data license for CUAD
- Do CUAD’s expert annotations certify that any model output is legally correct for a live contract?
- Yes; lawyer-supervised labels certify future model decisions
- Yes; the dataset covers 41 clause types, so all contract risks are included
- No; labels support research but do not certify future outputs or legal conclusions
- Yes, when benchmark scores are high on the labeled categories
Keep testing yourself
More quizzes picked for your level.