MLVU: Multi-task Long Video Understanding Benchmark

This repo contains the annotation data and evaluation code for the paper "MLVU: A Comprehensive Benchmark for Multi-Task Long Video Understanding".

  🔔 News:

🆕 7/28/2024: The data for the MLVU-Test set has been released (🤗 Link)! The test set includes 11 different tasks, featuring our newly added Sports Question Answering (SQA, single-detail LVU) and Tutorial… See the full description on the dataset page: https://huggingface.co/datasets/MLVU/MVLU.

Directional Guidance

This dataset provides a benchmark for evaluating Vision-Language Models (VLMs) in their ability to guide users to adjust an image to better answer a relevant question.

  Dataset Description

The Directional Guidance dataset focuses on Visual Question Answering (VQA) tasks where a model needs to evaluate visual information sufficiency and guide the user on where to reposition the camera if the image lacks necessary details. This dataset addresses a unique… See the full description on the dataset page: https://huggingface.co/datasets/LeoLee7/Directional_Guidance.

📊 PsTuts-RAG Q&A Dataset

This dataset contains question-answer pairs generated using RAGAS from Photoshop tutorial video transcripts published in PsTuts-VQA Dataset. It's designed for training and evaluating RAG (Retrieval-Augmented Generation) systems focused on Photoshop tutorials.

  📝 Dataset Description







  Dataset Summary

The dataset contains 100 question-answer pairs related to Photoshop usage, generated from video transcripts using RAGAS's… See the full description on the dataset page: https://huggingface.co/datasets/mbudisic/pstuts_rag_qa.

Dataset Card for Commodore 64 Dataset

  Dataset Details

This dataset is derived from the "Commodore 64 Programmer's Reference Guide," encompassing text chunks from the book, their summarized versions, and questions derived from the summaries. It aims to facilitate research and development in natural language processing tasks such as text summarization, question generation, and question answering, particularly in the context of programming and computer science historical… See the full description on the dataset page: https://huggingface.co/datasets/theoracle/commodore64.

AITU Admissions Guide Dataset

  Dataset Details





  Dataset Description

This dataset contains questions, answers, and categories related to the admission process at Astana IT University (AITU). It is designed to assist in automating applicant consultations and can be used for chatbot training, recommendation systems, and NLP-based question-answering models.

Curated by: Astana IT University Funded by [optional]: Arailym Tleubayeva, Alina Mitroshina, Alpar Arman… See the full description on the dataset page: https://huggingface.co/datasets/Arailym-tleubayeva/AITUAdmissionsGuideDataset.

Tutorials Question Text Dataset

This is the question text dataset of sysmlv2's official tutorials pdf. With the question text (only questions, no answers here) generated based on the tutorials, organized in both Chinese and English natural language text. Useful for training LLM and teach it the basic knowledge and conceptions of sysmlv2.

855 records in total.

id group_id type page_ids question_zh question_en

855 56 CHECK 181… See the full description on the dataset page: https://huggingface.co/datasets/sysmlv2research/tutorials_questions.