ต่อไปคำแนะนำต่อไป
KServe และโมเดลที่ให้บริการบน Kubernetes
เทคนิค
คู่มือ AI ภาษา
How AI Performs on the Bar Exam
Large language models such as GPT-4 have scored above typical passing lines on simulated versions of the Uniform Bar Exam.
บนหน้านี้อ่าน 4 นาที
ภาพรวม
The best-known figure is about 298 out of 400, reported in 2023. These results show that models can handle the written, rules-heavy format of a licensing test. They measure test performance under particular grading and comparison choices, not the full judgment a practicing lawyer needs.
เจาะลึก
The Uniform Bar Exam (UBE) has three parts. The Multistate Bar Examination (MBE) has 200 multiple-choice questions and counts for half the score. The Multistate Essay Examination (MEE) has six essays. The Multistate Performance Test (MPT) has two tasks in which the examinee works from a closed file of facts and law to write a memo or brief. Scores run to 400, and each jurisdiction sets its own passing line, commonly between 260 and 270. In late 2022, Michael Bommarito and Daniel Martin Katz tested GPT-3.5 on MBE practice questions. It answered about half correctly: well above chance, but below passing. In March 2023, Katz, Bommarito, Shang Gao and Pablo Arredondo reported that GPT-4 scored about 297 on a full simulated UBE. OpenAI's GPT-4 technical report cited roughly 298 and a percentile near the 90th. The percentile drew the sharpest criticism. In a 2024 paper in the journal Artificial Intelligence and Law, Eric Martínez showed that the 90th-percentile figure was based on February test takers. The February group includes an unusually large share of people retaking the exam after failing. Compared with first-time July takers, his estimate fell to around the 60th percentile, and lower on the essays. He also noted that the essays were scored by the researchers against published sample answers, not by trained bar graders, which adds uncertainty. Contamination is another concern: past questions circulate online and may have been in the training data. A common misconception is that passing the bar means a model can practice law. The exam tests recall of rules and structured analysis within a closed set of facts. Practice also requires investigating facts, counseling clients, planning strategy, judging risk and taking professional responsibility. In Mata v. Avianca (2023), lawyers filed a brief containing citations that ChatGPT had invented. The case showed that exam-level fluency does not guarantee reliable legal research.
ผลกระทบเชิงกลยุทธ์
ความเร็วและขนาด
ขั้นตอนการทำงานของภาษาสามารถดำเนินไปได้เร็วขึ้นโดยไม่กระทบต่อความสม่ำเสมอ
เข้าถึงและเข้าถึง
ขยายการเข้าถึงภาษาและรูปแบบการสื่อสาร
การตัดสินใจที่ชัดเจนยิ่งขึ้น
ทีมสามารถใช้เวลามากขึ้นในการตัดสิน ในขณะที่ระบบอัตโนมัติจัดการกับการทำซ้ำ
The Future of How AI Performs on the Bar Exam
Bar scores are becoming less useful as a benchmark. Top models already score near the ceiling on multiple-choice legal questions, and contamination is hard to rule out. Researchers have moved toward task collections such as LegalBench, a collaborative benchmark released in 2023 that breaks legal reasoning into many narrower tasks. They have also moved toward tests built on real work, such as research memos and contract review, graded by practicing lawyers. The NCBE's NextGen bar exam, planned to begin in 2026, gives more weight to integrated skills such as research and drafting. That may make comparisons with AI more informative. Whether models can pass tests is largely settled. The open question is how reliably they perform, and how they fail, on unfamiliar real matters.
การใช้งานจริงในโลกแห่งความเป็นจริง
OpenAI's March 2023 GPT-4 technical report listed a simulated Uniform Bar Exam score of about 298 out of 400 and a percentile near the 90th, a figure that was then widely repeated in headlines.
A law professor runs past MBE-style multiple-choice questions through a chatbot and compares its answers with her students'. The model is strong on black-letter rules but less reliable when the facts are deliberately ambiguous.
An evaluator checks whether a practice question was posted online before a model's training cutoff. A model that has already seen the question may be recalling an answer rather than reasoning to it.
A bar applicant uses an AI tutor to explain the MBE questions she missed. She checks each explanation against a commercial outline because the tutor sometimes presents a minority rule as the majority rule.
ความเสี่ยงและรั้ว
ข้อเท็จจริงที่หลอนประสาทสามารถเข้าสู่รายงาน กระแสสนับสนุน หรือผลการวิจัยได้อย่างเงียบๆ
ความละเอียดอ่อนของการแจ้งเตือนสามารถสร้างผลลัพธ์ที่ไม่สอดคล้องกันในคำขอที่คล้ายกัน
ข้อมูลข้อความที่ละเอียดอ่อนอาจถูกเปิดเผยหากการควบคุมการเข้าถึงอ่อนแอ
แผนงานการดำเนินงาน
กำหนดรูปแบบเอาต์พุต โทนเสียง และมาตรฐานคุณภาพก่อนเปิดตัว
การตอบสนองภาคพื้นดินกับแหล่งข้อมูลที่เชื่อถือได้เมื่อใดก็ตามที่ความแม่นยำมีความสำคัญ
รักษาจุดตรวจสอบการตรวจสอบโดยมนุษย์สำหรับผลลัพธ์ที่มีเดิมพันสูง
ติดตามรูปแบบความล้มเหลวและฝึกอบรมพร้อมท์หรือเวิร์กโฟลว์เป็นประจำ
สำรวจต่อไป
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the How AI Performs on the Bar Exam quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
คำถามที่พบบ่อย
What is How AI Performs on the Bar Exam?
Large language models such as GPT-4 have scored above typical passing lines on simulated versions of the Uniform Bar Exam. The best-known figure is about 298 out of 400, reported in 2023. These results show that models can handle the written, rules-heavy format of a licensing test. They measure test performance under particular grading and comparison choices, not the full judgment a practicing lawyer needs.
Which Uniform Bar Exam component has 200 multiple-choice questions and counts for half the total score?
The MBE is the 200-question multiple-choice section and makes up 50 percent of the UBE score. The MEE is six essays, and the MPT is two closed-file tasks.
What simulated Uniform Bar Exam score did OpenAI's GPT-4 technical report cite in 2023?
OpenAI reported roughly 298 out of 400. That is above the 260 to 270 passing range common across jurisdictions, and it was paired with a percentile near the 90th.
According to Eric Martínez's 2024 critique, why was GPT-4's 90th-percentile figure misleading?
The February group includes a large share of people retaking the exam after failing. Compared with first-time July takers, Martínez estimated GPT-4 fell to around the 60th percentile overall.
Who graded GPT-4's essay answers in the 2023 simulated bar exam, according to the critique described in the guide?
The essays were scored by the study's authors against published sample answers, not by trained bar graders. Martínez noted this adds uncertainty to the written-section scores.
What did Bommarito and Katz find when they tested GPT-3.5 on MBE practice questions in late 2022?
GPT-3.5 got roughly half the questions right. That is well above the 25 percent expected from guessing on four-option questions, but below a passing level.
เรียนรู้ต่อไป
คำแนะนำที่เกี่ยวข้อง
คำแนะนำเพิ่มเติมที่เลือกสำหรับหัวข้อนี้