Tiếp theoHướng dẫn tiếp theo
AI trên thiết bị và AI trên đám mây trên điện thoại
kỹ thuật
HƯỚNG DẪN KỸ THUẬT
LegalBench là một bộ chuẩn được cộng đồng xây dựng gồm 162 nhiệm vụ, thiết kế để đo lường các loại suy luận pháp lý khác nhau trong các mô hình tiếng Anh.
Nghe hướng dẫn nàyAI thuật lại · ~3 min
Its tasks provide evidence about performance on those evaluations; a score is not a general legal-competence certificate or proof that a model can advise clients.
LegalBench is an open-science benchmark assembled collaboratively by lawyers, legal researchers, and computer scientists. The authors describe 162 tasks covering six types of legal reasoning and report empirical evaluation of 20 open-source and commercial language models. The repository includes task datasets and instructions, with task-specific licenses that users must follow. Tasks include different input and output forms, such as legal questions, evidence descriptions, and passages to interpret. Benchmark performance depends on task design, prompts, scoring, data split, model version, and whether examples have appeared in training. LegalBench’s breadth is useful for comparing task-specific behavior, but its authors present it as a way to study what types of reasoning models perform, not as a certification of legal practice. A result on hearsay classification does not show competence in drafting, jurisdiction-specific advice, negotiation, or fact investigation. Even a high score may conceal errors on particular subgroups or input formats. For a meaningful evaluation, state the task subset, prompt, model version, scoring method, and data provenance. Check licenses and instructions, compare baselines, inspect errors, and use held-out examples from the intended workflow. Human legal experts should evaluate outputs when the intended use has legal consequences. Do not present one benchmark score as proof that a system is safe, accurate, or ready to replace professional judgment. The repository provides task prompts and answer guides alongside datasets, allowing evaluators to inspect how each task is scored and what source material it uses.
A research team reports separate LegalBench results by task instead of one overall label.
A lawyer checks whether a benchmark task matches the intended legal workflow.
An evaluator examines scoring instructions and source datasets before comparing models.
A team tests a current model on held-out or private examples to reduce benchmark contamination.
Benchmarks will evolve as legal tasks and model capabilities change. Researchers can improve measurement with new task coverage, error analysis, and private evaluation sets. Users still need to match the evaluation to a concrete workflow and verify that the model’s performance holds under current law and actual deployment conditions. Legal rules and model versions change over time, and benchmark tasks may remain static. Re-run evaluations on current materials and held-out examples before relying on results. High benchmark performance can help identify promising tools but cannot establish authorization to practice or suitability for a client matter.
Bản tin miễn phí
Ba câu chuyện AI đã được xác minh vào mỗi buổi sáng các ngày trong tuần, được viết bằng tiếng Anh đơn giản. Miễn phí mãi mãi, không có quảng cáo.
Một email mỗi ngày trong tuần. Hủy đăng ký trong một cú nhấp chuột. Chúng tôi không bao giờ bán hoặc chia sẻ địa chỉ của bạn.
Tự kiểm tra
Phản hồi tức thì về mọi câu trả lời và chứng chỉ có thể chia sẻ với ID có thể xác minh sau khi bạn vượt qua khóa học.
Hỗ trợ giáo dục AI miễn phí. AI Understanding là tổ chức phi lợi nhuận 501(c)(3) — không bao giờ có quảng cáo, không có tường phí. Hãy quyên góp
LegalBench is a collaboratively built benchmark of 162 tasks designed to measure different kinds of legal reasoning in English-language models. Its tasks provide evidence about performance on those evaluations; a score is not a general legal-competence certificate or proof that a model can advise clients.
The paper presents LegalBench as a benchmark for studying distinct legal reasoning tasks.
A result on one task does not establish broader professional competence.
The benchmark spans distinct tasks; subgroup results can show meaningful differences.
Reproducibility depends on the actual task and evaluation setup.
A comparison is interpretable only when evaluation conditions are aligned.
Tiếp tục học hỏi
Đã chọn thêm hướng dẫn cho chủ đề này
Tiếp theoHướng dẫn tiếp theo
AI trên thiết bị và AI trên đám mây trên điện thoại
kỹ thuật