lscpku/MMBench-Video dataset hosted on Hugging Face and contributed by the HF Datasets community

MMBench is a multi-modality benchmark. It methodically develops a comprehensive evaluation pipeline, primarily comprised of two elements. The first element is a meticulously curated dataset that surpasses existing similar benchmarks in terms of the number and variety of evaluation questions and abilities. The second element introduces a novel CircularEval strategy and incorporates the use of ChatGPT. This implementation is designed to convert free-form predictions into pre-defined choices, thereby facilitating a more robust evaluation of the model's predictions.

This is a subset of the video understanding benchmark MMBench-Video.

[🍎 Home Page] [📖 Technical Report] [\ud83d\udcca Models] [\ud83d\ude80 Demo] This repository contains KC-MMBench, a new benchmark dataset meticulously tailored for real-world short-video scenarios, as presented in the paper "Kwai Keye-VL Technical Report". Constructed from Kuaishou short video data, KC-MMBench comprises 6 distinct datasets designed to evaluate the performance of Vision-Language Models (VLMs) like Kwai Keye-VL-8B, Qwen2.5-VL, and InternVL in comprehending dynamic… See the full description on the dataset page: https://huggingface.co/datasets/Kwai-Keye/KC-MMbench.