Large-scale Multi-modality Models Evaluation Suite

Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval

🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets

  This Dataset

This is a formatted version of ChartQA. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models. @article{masry2022chartqa, title={ChartQA: A benchmark for question answering about charts with visual and… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab/ChartQA.

ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

source: https://huggingface.co/datasets/HuggingFaceM4/ChartQA paper: https://arxiv.org/pdf/2203.10244

Dataset Description

This is a VQA dataset based on Charts from ChartQA dataset from ChartQA.

  Load the dataset

from datasets import load_dataset import csv

def load_beir_qrels(qrels_file): qrels = {} with open(qrels_file) as f: tsvreader = csv.DictReader(f, delimiter="\t") for row in tsvreader: qid = row["query-id"] pid = row["corpus-id"] rel = int(row["score"]) if qid in qrels:… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/VisRAG-Ret-Test-ChartQA.

MLP-VLM2/llava-cot-20k-docvqa-chartqa dataset hosted on Hugging Face and contributed by the HF Datasets community

DaveKevin/ChartQA dataset hosted on Hugging Face and contributed by the HF Datasets community

Dataset

This dataset was created by Lê Thành Nghĩa

Contents

docintel/ChartQA dataset hosted on Hugging Face and contributed by the HF Datasets community

LIME-DATA/ChartQA dataset hosted on Hugging Face and contributed by the HF Datasets community

anhdang000/ChartQA-QueryTable dataset hosted on Hugging Face and contributed by the HF Datasets community

The dataset used for testing the Vary-base model, containing DocVQA and ChartQA datasets.

zz990906/chartQA-text dataset hosted on Hugging Face and contributed by the HF Datasets community

ChartQA Dataset

This dataset is derived from the ChartQA dataset, reformatting the test split with modified field names, so that it can be used in the ViDoRe benchmark. The text_description column contains OCR text extracted from the images using EasyOCR. This particular dataset is a subsample of 1000 random rows from the full dataset which can be found here.

  Disclaimer

This dataset may contain publicly available images or text data. All data is provided for research… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/ChartQA.

MaIlz/curve-chartqa dataset hosted on Hugging Face and contributed by the HF Datasets community

yobro4619/chartqa-restructured dataset hosted on Hugging Face and contributed by the HF Datasets community

Dataset Card for RealWorld-ChartQA

  Summary

RealWorld-ChartQA is a benchmark dataset for chart question answering (CQA), derived from real-world analytical narratives. It contains 205 manually validated multiple-choice question–answer pairs grounded in student-authored literate visualization notebooks. Unlike previous CQA datasets, RealWorld-ChartQA includes multi-view and interactive charts, along with questions rooted in ecologically valid analytical workflows.… See the full description on the dataset page: https://huggingface.co/datasets/maevehutch/realworld-chartqa.

yobro4619/chartqa-restructured-train-chunks dataset hosted on Hugging Face and contributed by the HF Datasets community

rweics5cs7/exo3-original-ChartQA-text-v3 dataset hosted on Hugging Face and contributed by the HF Datasets community

This dataset is modified version of ChartQA made for Llama-3.2 (11B vision) script on colab.

MrZilinXiao/MMEB-eval-ChartQA-beir dataset hosted on Hugging Face and contributed by the HF Datasets community

Description

French translation of the lmms-lab/ChartQA dataset that we processed.

  Citation

@article{masry2022chartqa, title={ChartQA: A benchmark for question answering about charts with visual and logical reasoning}, author={Masry, Ahmed and Long, Do Xuan and Tan, Jia Qing and Joty, Shafiq and Hoque, Enamul}, journal={arXiv preprint arXiv:2203.10244}, year={2022} }