LegalBench is a collection of benchmark tasks for evaluating legal reasoning in large language models.

Dataset Card for GreekBarBench 🇬🇷🏛️⚖️

GreekBarBench is a benchmark designed to evaluate LLMs on challenging legal reasoning questions across five different legal areas from the Greek Bar exams, requiring citations to statutory articles and case facts.

  Dataset Details





  Dataset Description

GreekBarBench (GBB) comprises legal questions sourced from the Greek Bar exams held between 2015 and 2024. The dataset aims to simulate the open-book format of these… See the full description on the dataset page: https://huggingface.co/datasets/AUEB-NLP/greek-bar-bench.

UniCourt provides legal data on law firms that’s been normalized by our AI and enriched with other public data sets to connect real-world law firms to their attorneys and clients, judges they’ve faced and types of litigation they’ve handled across practice areas and state and federal (PACER) courts.

AI Normalized Law Firms

• UniCourt’s AI locates and gathers variations of law firm names and spelling errors contained in court data and combines them with bar data, business data, and judge data to connect real-world law firms to their litigation. • Avoid bad data caused by frequent law firm name changes due to firm mergers, named partners leaving, and firms dissolving, leading to lost business and bad analytics. • UniCourt’s unique normalized IDs for law firms let you quickly search for and download all of the litigation involving the specific firms you’re interested in. • Uncover the associations and relationships between law firms, their lawyers, their clients, judges, and their top practice areas across different jurisdictions.

Using APIs to Dig Deeper

• See a full list of all of the businesses and individuals a law firm has represented as clients in litigation. • Easily vet the bench strength of law firms by looking at the volume and specific types of cases their lawyers have handled. • Drill down into a law firm’s experience to confirm which judges they’ve appeared before in court. • Identify which law firms and lawyers a particular firm has faced as opposing counsel, and the judgments they obtained.

Bulk Access to Law Firm Data

• UniCourt’s Law Firm Data API provides you with structured, cleaned, and organized legal data that you can easily connect to your case management systems, CRM, and other internal applications. • Get bulk access to law firm Secretary of State registration data and the names, emails, phone numbers, and physical addresses for all of a firm’s lawyers. • Use our APIs to create tailored legal marketing campaigns for law firms and their attorneys with the exact practice area expertise and the right geographic coverage you want to target. • Power your case research, business intelligence, and analytics with bulk access to litigation data for all the court cases a firm has handled and set up automated data feeds to find new cases they’re involved in.

LegalBenchConsumerContractsQA An MTEB dataset Massive Text Embedding Benchmark

The dataset includes questions and answers related to contracts.

Task category t2t

Domains Legal, Written

Reference https://huggingface.co/datasets/nguha/legalbench/viewer/consumer_contracts_qa

  How to evaluate on this task

You can evaluate an embedding model on this dataset using the following code: import mteb

task = mteb.get_tasks(["LegalBenchConsumerContractsQA"])… See the full description on the dataset page: https://huggingface.co/datasets/mteb/legalbench_consumer_contracts_qa.

This database contains documents from the Fair Work Commission - Full Bench 2013-.

This database contains documents from the Fair Work Australia - Full Bench 2009-2012.

OAB-Bench

| Paper | Code | OAB-Bench is a benchmark for evaluating Large Language Models (LLMs) on legal writing tasks, specifically designed for the Brazilian Bar Examination (OAB). The benchmark comprises 105 questions across seven areas of law from recent editions of the exam.

OAB-Bench evaluates LLMs on their ability to write legal documents and answer discursive questions The benchmark includes comprehensive evaluation guidelines used by human examiners Results show that… See the full description on the dataset page: https://huggingface.co/datasets/maritaca-ai/oab-bench.

OABench: Brazilian Bar Exams Benchmark Dataset

  Overview

OABench is a benchmark dataset designed to evaluate the performance of Large Language Models (LLMs) on Brazilian legal exams. It is based on the Unified Bar Exam of the Brazilian Bar Association (OAB), a comprehensive and challenging exam required for law graduates to practice law in Brazil. This dataset provides a rigorous and realistic testbed for LLMs in the legal domain, covering a wide range of legal topics… See the full description on the dataset page: https://huggingface.co/datasets/felipeoes/oab_bench.

The global market for Legal for Trade Scales is experiencing robust growth, driven by increasing regulatory compliance requirements across various industries, including food processing, pharmaceuticals, and logistics. The demand for accurate and reliable weighing solutions is paramount in these sectors to ensure fair trade practices and prevent potential legal issues. Technological advancements, such as the integration of digital technologies and improved connectivity features in modern scales, are further fueling market expansion. This allows for enhanced data management, traceability, and remote monitoring, improving operational efficiency and reducing errors. The market is segmented by type (e.g., platform scales, bench scales, counting scales), application (e.g., retail, industrial, laboratory), and region. The competitive landscape is characterized by the presence of established players like Mettler-Toledo, OHAUS, and Avery Weigh-Tronix, alongside several regional and niche players. These companies are continuously innovating to meet the evolving needs of customers, focusing on aspects such as improved accuracy, enhanced durability, and user-friendly interfaces. The market's growth is projected to continue at a healthy rate, fueled by rising global trade, stricter regulations, and ongoing technological progress. While precise market size figures are not provided, a reasonable estimation can be made based on typical industry growth rates and the presence of numerous major players. Let's assume a 2025 market size of $5 billion (USD), based on the prominence of the listed companies and the widespread use of legal for trade scales across various sectors. A conservative Compound Annual Growth Rate (CAGR) of 5% over the forecast period (2025-2033) is plausible given the industry trends. This would project a market value exceeding $7 billion by 2033. Growth will be influenced by factors such as economic conditions in key regions, the pace of technological innovation, and the evolution of industry regulations. Furthermore, potential restraints include the high initial investment costs associated with advanced scales and the need for ongoing calibration and maintenance. However, the long-term benefits of accurate weighing, improved efficiency, and legal compliance outweigh these challenges, ensuring sustained market growth.

This database contains documents from the Australian Industrial Relations Commission - Full Bench 2007-2009.

Legal_SGD-LawQA-v2

Legal_SGD-LawQA-v2 is a synthetic question-answering dataset generated from the legal_full_v3 dataset. It is designed to benchmark and evaluate retrieval systems, particularly for legal domain question-answering tasks.

  Data Fields

question: The generated legal question. answer: The corresponding answer. chunk_id: Identifier linking each QA pair to its source chunk in legal_full_v3.

This dataset provides a controlled benchmark environment for testing… See the full description on the dataset page: https://huggingface.co/datasets/QomSSLab/Legal_SyntheticLegalQA-Bench-v2.

This database contains documents from the Australian Industrial Relations Commission - Full Bench Decision Summaries 2007-2010.

Case-level judicial data from Lahore High Courts extracted from District Judiciary Punjab’s case management system website for the year 2013, to establish a baseline to empirically understand Pakistan’s legal system.

The data repository contains the following files:

raw_data_lahore_highcourt_2013_v1.xlsx: This file contains the raw data extracted from the Punjab High Court website.

to_clean_data_lahore_highcourt_2013_v1.do: This file contains the Stata do file used to clean the raw data.

stata_cleaned_data_lahore_highcourt_2013_v1.dta: This file contains the cleaned data in Stata format.

cleaned_data_lahore_highcourt_2013_v1.csv: This file contains the cleaned data in CSV format. The variable names for cleaned data are human-readable and mostly make clear what information the variable contains. The variables available are as follows:

case_no: Case Number

case_status: Case Status

inst_no: A unique number representing the case in the system

inst_date: Date when the case was officially filed or registered in the court.

type_of_case: The type or category of the case (e.g., First Class Cases, Bail Applications).

offense: The description of crime or violation for which the case was filed (e.g., Cheating, Rape, etc).

procedure_or_documentation: Indicates the type of procedure involved, such as pre-arrest bail or other legal processes.

fir_no: The First Information Report (FIR) number associated with the case.

fir_year: The year in which the FIR was registered.

police_station: The name of the police station where the FIR was lodged.

primary_offence: The main crime or charge the case deals with

presiding_judge: The name of the judge presiding over the case.

bench: The judicial authority or composition of the bench (e.g., Civil Judge or Additional District).

decision_date: The date when the court issued a decision or verdict.

decision_type: The type of decision made by the court (e.g., Acquittal, Conviction).

contested: Indicates whether the case was contested or uncontested.

acquitted_convicted: Indicates whether the accused was acquitted or convicted.

LawDual-Bench: A Dual-Task Benchmark and Chain-of-Thought Impact Study for Legal Reasoning

随着大语言模型（LLMs）在法律应用中的快速发展，系统评估其在法律文档处理和判决预测中的推理能力变得尤为迫切。目前公开的法律测评基准缺少统一的评估架构，对这两个任务的支持并不好。为填补这一空白，我们提出了 LawDual-Bench，填补了中文法律自然语言处理领域中结构化推理评估的关键空白，并为法律垂类大模型系统的评估与优化提供了坚实基础。更多详情可查看我们的论文。

  📄 介绍

LawDual-Bench 经精心设计，可以对大模型的法律文档理解和案情分析推理能力进行精确评估。我们设计了一套半自动化的数据集构建方案，通过人工+LLM的方式，构建了一个全面的内幕交易数据集，同时也可以很轻易地扩展数据集的数量与案情的种类。再次基础上，我们设计了结构化信息抽取 和 案件事实分析与判决预测… See the full description on the dataset page: https://huggingface.co/datasets/Yuwh07/LawDual-Bench.

This bound volume gives the dates of appointment and names of the following: Chief Justices 1882-1901, Supreme Court Judges 1882-1913, Bench of the High Court, Barristers 1882-1917, Solicitors 1882-1917 and various government legal office appointees.

(6/5493). 1 vol.

Note:

This description is extracted from Concise Guide to the State Archives of New South Wales, 3rd Edition 2000.

Protecting Privacy in Multimodal Large Language Models with MLLMU-Bench

  Abstract

Generative models such as Large Language Models (LLM) and Multimodal Large Language models (MLLMs) trained on massive web corpora can memorize and disclose individuals' confidential and private data, raising legal and ethical concerns. While many previous works have addressed this issue in LLM via machine unlearning, it remains largely unexplored for MLLMs. To tackle this challenge, we… See the full description on the dataset page: https://huggingface.co/datasets/MLLMMU/MLLMU-Bench.

Prior to 1891 people were admitted to practice either as barristers or as attorneys, solicitors and proctors.

The enactment of the Legal Profession Practice Act 1891 (1229) legally fused the barristers' and solicitors' branches of the legal profession in Victoria and the distinction is maintained in the current Act, the Legal Profession Practice Act 1958. In practice, however, the branches remain quite separate, for a person admitted as a barrister and solicitor of the Supreme Court must make an election whether they wish to be inscribed on the Roll of Counsel or on the Roll of Solicitors.

If a person elects to practice as a barrister in Victoria, he or she must make an application to sign the Roll of Counsel. The Roll of Counsel constituting the Victorian Bar is kept by the Victorian Bar Council, who upon being satisfied as to the applicant's qualification, intention to practise as counsel in Victoria and obtaining further undertakings as required, may, subject to its discretion, consent to the applicant's signing of the roll.

The rules regarding the admission to practice as lawyers were outlined in the Supreme Court rules. In April 1853 the Supreme Court promulgated Rules and Regulations for Admission to Practices as Barristers and as Attorneys, Solicitors and Proctors, in the Supreme Court of Victoria. Rule 1 established two Boards of Examiners, one for Barristers and one for Attorneys.

The Board of Examiners for Legal Practitioners (body situated within Supreme Court of Victoria) is the body responsible for arranging admission to practice in the Supreme Court of Victoria by persons wishing to be lawyers. The Board grants (or withholds) the Certificates upon which the Supreme Court relies when ordering that persons be admitted.

The Roll of Barristers records the names of those 'called to the bar' that is allowed to practice as a barrister in the state of Victoria. .

The Roll records the barrister's full name or signature, (from 1892 onwards), date of admission and remarks, (barrister's death, or elevation to the bench or disbarment).

A separate Roll of Solicitors was maintained (Refer to VPRS 16237).

HSE-Bench

HSE-Bench is a graduate-level benchmark designed to evaluate the legal and safety reasoning capabilities of Large Language Models (LLMs) in high-stakes, regulation-intensive domains concerning Health, Safety, and the Environment (HSE). This benchmark focuses on scenario-based, single-choice questions crafted under the IRAC (Issue, Rule, Application, Conclusion) reasoning framework, with all content presented in English.

  Overview

HSE-Bench comprises 1,020… See the full description on the dataset page: https://huggingface.co/datasets/Joysouo/hse-bench.

This is the official dataset for GraphRAG-Bench: Challenging Domain-Specific Reasoning for Evaluating Graph Retrieval-Augmented Generation. It contains 5 question types spanning 16 disciplines and a corpus of 7 million words from 20 computer science textbooks.

  license

This dataset can only be used for academic research and cannot be used for any commercial purposes. It is prohibited to distribute or modify the content of the dataset. Any legal consequences resulting from violating… See the full description on the dataset page: https://huggingface.co/datasets/Awesome-GraphRAG/GraphRAG-Bench.

GRBench

GRBench is a comprehensive benchmark dataset to support the development of methodology and facilitate the evaluation of the proposed models for Augmenting Large Language Models with External Textual Graphs.

  Dataset Details





  Dataset Description

GRBench includes 10 real-world graphs that can serve as external knowledge sources for LLMs from five domains including academic, e-commerce, literature, healthcare, and legal domains. Each sample in GRBench… See the full description on the dataset page: https://huggingface.co/datasets/PeterJinGo/GRBench.