概述
The best-known figure is about 298 out of 400, reported in 2023. These results show that models can handle the written, rules-heavy format of a licensing test. They measure test performance under particular grading and comparison choices, not the full judgment a practicing lawyer needs.
深入探討
The Uniform Bar Exam (UBE) has three parts. The Multistate Bar Examination (MBE) has 200 multiple-choice questions and counts for half the score. The Multistate Essay Examination (MEE) has six essays. The Multistate Performance Test (MPT) has two tasks in which the examinee works from a closed file of facts and law to write a memo or brief. Scores run to 400, and each jurisdiction sets its own passing line, commonly between 260 and 270. In late 2022, Michael Bommarito and Daniel Martin Katz tested GPT-3.5 on MBE practice questions. It answered about half correctly: well above chance, but below passing. In March 2023, Katz, Bommarito, Shang Gao and Pablo Arredondo reported that GPT-4 scored about 297 on a full simulated UBE. OpenAI's GPT-4 technical report cited roughly 298 and a percentile near the 90th. The percentile drew the sharpest criticism. In a 2024 paper in the journal Artificial Intelligence and Law, Eric Martínez showed that the 90th-percentile figure was based on February test takers. The February group includes an unusually large share of people retaking the exam after failing. Compared with first-time July takers, his estimate fell to around the 60th percentile, and lower on the essays. He also noted that the essays were scored by the researchers against published sample answers, not by trained bar graders, which adds uncertainty. Contamination is another concern: past questions circulate online and may have been in the training data. A common misconception is that passing the bar means a model can practice law. The exam tests recall of rules and structured analysis within a closed set of facts. Practice also requires investigating facts, counseling clients, planning strategy, judging risk and taking professional responsibility. In Mata v. Avianca (2023), lawyers filed a brief containing citations that ChatGPT had invented. The case showed that exam-level fluency does not guarantee reliable legal research.
戰略影響
速度與規模
語言工作流程可以在不犧牲一致性的情況下更快地移動。
交通與覆蓋範圍
它擴展了跨語言和溝通方式的訪問。
更明確的決策
團隊可以花更多時間進行判斷，而自動化則可以處理重複。
The Future of How AI Performs on the Bar Exam
Bar scores are becoming less useful as a benchmark. Top models already score near the ceiling on multiple-choice legal questions, and contamination is hard to rule out. Researchers have moved toward task collections such as LegalBench, a collaborative benchmark released in 2023 that breaks legal reasoning into many narrower tasks. They have also moved toward tests built on real work, such as research memos and contract review, graded by practicing lawyers. The NCBE's NextGen bar exam, planned to begin in 2026, gives more weight to integrated skills such as research and drafting. That may make comparisons with AI more informative. Whether models can pass tests is largely settled. The open question is how reliably they perform, and how they fail, on unfamiliar real matters.
現實世界的實施
OpenAI's March 2023 GPT-4 technical report listed a simulated Uniform Bar Exam score of about 298 out of 400 and a percentile near the 90th, a figure that was then widely repeated in headlines.
A law professor runs past MBE-style multiple-choice questions through a chatbot and compares its answers with her students'. The model is strong on black-letter rules but less reliable when the facts are deliberately ambiguous.
An evaluator checks whether a practice question was posted online before a model's training cutoff. A model that has already seen the question may be recalling an answer rather than reasoning to it.
A bar applicant uses an AI tutor to explain the MBE questions she missed. She checks each explanation against a commercial outline because the tutor sometimes presents a minority rule as the majority rule.
風險與防護欄
幻覺的事實可以悄悄地進入報告、支持流程或研究成果。
及時的敏感性可能會在類似的請求中產生不一致的結果。
如果存取控制薄弱，敏感文字資料可能會暴露。
實施路線圖
在推出之前定義輸出格式、語氣和品質標準。
當準確性很重要時，請使用可信任來源進行地面回應。
為高風險輸出保留人工審查檢查點。
追蹤故障模式並定期重新訓練提示或工作流程。
不斷探索
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the How AI Performs on the Bar Exam quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
常見問題
What is How AI Performs on the Bar Exam?
Large language models such as GPT-4 have scored above typical passing lines on simulated versions of the Uniform Bar Exam. The best-known figure is about 298 out of 400, reported in 2023. These results show that models can handle the written, rules-heavy format of a licensing test. They measure test performance under particular grading and comparison choices, not the full judgment a practicing lawyer needs.
Which Uniform Bar Exam component has 200 multiple-choice questions and counts for half the total score?
The MBE is the 200-question multiple-choice section and makes up 50 percent of the UBE score. The MEE is six essays, and the MPT is two closed-file tasks.
What simulated Uniform Bar Exam score did OpenAI's GPT-4 technical report cite in 2023?
OpenAI reported roughly 298 out of 400. That is above the 260 to 270 passing range common across jurisdictions, and it was paired with a percentile near the 90th.
According to Eric Martínez's 2024 critique, why was GPT-4's 90th-percentile figure misleading?
The February group includes a large share of people retaking the exam after failing. Compared with first-time July takers, Martínez estimated GPT-4 fell to around the 60th percentile overall.
Who graded GPT-4's essay answers in the 2023 simulated bar exam, according to the critique described in the guide?
The essays were scored by the study's authors against published sample answers, not by trained bar graders. Martínez noted this adds uncertainty to the written-section scores.
What did Bommarito and Katz find when they tested GPT-3.5 on MBE practice questions in late 2022?
GPT-3.5 got roughly half the questions right. That is well above the 25 percent expected from guessing on four-option questions, but below a passing level.
繼續學習
相關指南
為此主題精選的更多指南