Dataset Card for PLM-Video Human

PLM-Video-Human is a collection of human-annotated resources for training Vision Language Models, focused on detailed video understanding. Training tasks include: fine-grained open-ended question answering (FGQA), Region-based Video Captioning (RCap), Region-based Dense Video Captioning (RDCap) and Region-based Temporal Localization (RTLoc). [📃 Tech Report] [📂 Github]

  Dataset Structure






  Fine-Grained Question Answering… See the full description on the dataset page: https://huggingface.co/datasets/facebook/PLM-Video-Human.

Dataset Card for "PPLM-PQA"

More Information needed

Dataset Card for PLM-Image Auto

[📃 Tech Report] [📂 Github] Sythetic image captions and QAs used in PLM, please refer to the paper, Section 3, for more details. The sythetic annotations covers: SA1B, Openimages, Obejct365, ArxivQA, UCSF, PDFAcc.

  Dataset Structure





  Image Captions (SA1B, Openimages, Obejct365)

Data fields are :

image_id: a string feature, unique identifier for the image. image: a string feature, the actual image path in the correspoding data… See the full description on the dataset page: https://huggingface.co/datasets/facebook/PLM-Image-Auto.

Dataset Card for PLM-Video Auto

[📃 Tech Report] [📂 Github] Sythetic video captions and MCQs used in PLM, please refer to the paper, Section 3, for more details. The sythetic annotations covers: YT-1B, Ego4d with captions, YT-1B with MCQAs and Ego4d with QAs.

  Dataset Structure





  YT-1B Captions (yt1b_cap)

Data fields are :

video_id: a string feature, unique identifier for the YouTube videoid. scene_id: a string feature, unique identifier for the scene_id.… See the full description on the dataset page: https://huggingface.co/datasets/facebook/PLM-Video-Auto.

Dataset Summary

PLM-VideoBench is a collection of human-annotated resources for evaluating Vision Language models, focused on detailed video understanding. [📃 Tech Report] [📂 Github]

  Supported Tasks

PLM-VideoBench includes evaluation data for the following tasks:

  FGQA

In this task, a model must answer a multiple-choice question (MCQ) that probes fine-grained activity understanding. Given a question and multiple options that differ in a… See the full description on the dataset page: https://huggingface.co/datasets/facebook/PLM-VideoBench.

xiaokangz/plm-qa dataset hosted on Hugging Face and contributed by the HF Datasets community

YanSong97/plm-env dataset hosted on Hugging Face and contributed by the HF Datasets community

Battam/PLM-Finetune dataset hosted on Hugging Face and contributed by the HF Datasets community

This is the Pretraining Dataset for PLM. Due to the upload limit, we split the original dataset into parts that smaller than 50GB. We provide the merge and split scripts under scripts folder.

Spatial Visualization Benchmark

This repository contains the Spatial Visualization Benchmark. The evaluation code is released on: wangst0181/Spatial-Visualization-Benchmark.

  Dataset Description

The SpatialViz-Bench aims to evaluate the spatial visualization capabilities of multimodal large language models, which is a key component of spatial abilities. Targeting 4 sub-abilities of Spatial Visualization, including mental rotation, mental folding, visual penetration, and… See the full description on the dataset page: https://huggingface.co/datasets/PLM-Team/Spatial-Visualization-Benchmark.