AI Litigation Outcome Prediction Quiz

Answers and explanations appear as you take the quiz.

1. According to the guide, what do most litigation analytics tools mainly report?
   • Guaranteed forecasts of individual case outcomes
   • Descriptive historical base rates drawn from dockets
   • Judges' personal political preferences
   • Private settlement terms from sealed agreements
2. Why did critics question the high accuracy reported by the 2016 study of European Court of Human Rights cases?
   • It used too few cases to measure accuracy
   • It ignored the judges' names
   • It used the court's own summary of the facts, which is written after the outcome is known
   • It predicted only cases that had already settled
3. How does the selection effect associated with the Priest-Klein hypothesis limit outcome prediction?
   • Decided cases are not a random sample because most cases settle
   • Judges are randomly assigned, so no patterns exist
   • Federal dockets exclude all civil cases
   • Only appellate decisions are recorded
4. What did France prohibit in 2019?
   • All electronic access to court records
   • The use of AI by lawyers in court filings
   • Publication of any court decisions online
   • Using judges' identity data to evaluate or predict their professional practices
5. When a model has only a few past rulings for a specific judge and motion type, what does the guide suggest instead of reporting the raw rate?
   • Excluding that judge from the analysis permanently
   • Using hierarchical or Bayesian methods to pull the estimate toward the court-wide rate
   • Assuming a 50 percent rate for every thin sample
   • Doubling the sample by counting each ruling twice
6. In litigation prediction, what does it mean for a model to be well calibrated?
   • It is right on every case in its training data
   • It uses the newest machine learning architecture
   • Outcomes it gives a 70 percent probability happen about 70 percent of the time
   • It produces the same prediction for every judge
7. What is leakage in a litigation outcome model?
   • Using features that are only known after the outcome
   • Losing docket data because of storage errors
   • Sharing model results with opposing counsel
   • Training on too many state court cases
8. According to the guide, where are litigation predictions likely to be most useful?
   • Deciding a single novel dispute with no precedent
   • Replacing legal research on controlling authority
   • Choosing which judge will be assigned
   • Budgeting, settlement ranges and decisions across many cases, where errors average out