Guide-linked quiz · Medium Level

CUAD and Legal NLP Datasets Quiz

Tests CUAD’s corpus and labels, its benchmark scope, licensing and dataset limits, split design, evaluation, and legal-review boundaries.

Question 1 of 8

How does the Atticus Project describe CUAD v1?

Preview every question8 questions

Answers and explanations appear as you take the quiz.

  1. How does the Atticus Project describe CUAD v1?
    • 510 commercial contracts with label metadata but no manually annotated clause spans
    • 510 commercial contracts with 13,000-plus labels across 41 clause types
    • A corpus of court opinions tagged with contract-law doctrines
    • A dataset of full-contract risk scores without annotated clause spans
  2. Which task was CUAD created to support?
    • Locating salient contract language for human review
    • Classifying agreements as enforceable or unenforceable across jurisdictions
    • Extracting party names and dates only, without clause-level text
    • Predicting which contract disputes will be filed in court
  3. A model scores highly on CUAD. What does that establish most directly?
    • That the model legally interprets all 41 clause types under current law
    • That it generalizes to every contract type and jurisdiction
    • Performance on the benchmark task and data under the reported evaluation setup
    • That an expert-labelled score clears the model for transaction deployment
  4. Why examine the split design before using CUAD for evaluation?
    • A train/test split makes template overlap irrelevant to generalization
    • Related templates or documents across splits can make results look better than generalization warrants
    • The benchmark cannot be trained on when labels are present in the test set
    • Split design alone establishes that results apply to new contract populations
  5. A lawyer wants to use a CUAD-trained model on a non-disclosure agreement in another jurisdiction. What is the best next step?
    • Evaluate only common CUAD clauses and assume they represent NDA language
    • Use a held-out CUAD split from the same source and treat it as proof of jurisdictional transfer
    • Change the jurisdiction metadata and reuse the original score without testing
    • Evaluate on representative agreements from that intended setting
  6. Why report results by clause category as well as one aggregate score?
    • A single aggregate is sufficient when the 41 labels have balanced counts
    • An aggregate can hide that extraction works differently across label types
    • A score for each clause type by itself guarantees transfer to new agreements
    • Category breakdowns are needed only when overall accuracy falls below chance
  7. What should a researcher check before redistributing CUAD data?
    • The dataset version and license terms stated by its maintainers
    • Assume a top-level code license covers every dataset and annotation file
    • Treat public availability as permission to redistribute without checking conditions
    • Use the model’s license as the data license for CUAD
  8. Do CUAD’s expert annotations certify that any model output is legally correct for a live contract?
    • Yes; lawyer-supervised labels certify future model decisions
    • Yes; the dataset covers 41 clause types, so all contract risks are included
    • No; labels support research but do not certify future outputs or legal conclusions
    • Yes, when benchmark scores are high on the labeled categories

Keep testing yourself

More quizzes picked for your level.

View all quizzes