Dataset Card for "commonsense_qa"

  Dataset Summary

CommonsenseQA is a new multiple-choice question answering dataset that requires different types of commonsense knowledge to predict the correct answers . It contains 12,102 questions with one correct answer and four distractor answers. The dataset is provided in two major training/validation/testing set splits: "Random split" which is the main evaluation split, and "Question token split", see paper for details.… See the full description on the dataset page: https://huggingface.co/datasets/tau/commonsense_qa.

The Cosmos QA dataset is a large-scale dataset of 35.6K problems that require commonsense-based reading comprehension, formulated as multiple-choice questions. The dataset focuses on reading between the lines over a diverse collection of people's everyday narratives, asking questions concerning on the likely causes or effects of events that require reasoning beyond the exact text spans in the context.

This allows for much more sophisticated models to be built and evaluated, and could lead to better performance on real-world tasks

How to use the dataset In order to use the Cosmos QA dataset, you will need to first download the data files from the Kaggle website. Once you have downloaded the files, you will need to unzip them and then place them in a directory on your computer.

Once you have the data files placed on your computer, you can begin using the dataset for commonsense-based reading comprehension tasks. The first step is to load the context file into a text editor such as Microsoft Word or Adobe Acrobat Reader. Once the context file is open, you will need to locate the section of text that contains the question that you want to answer.

Once you have located the section of text containing the question, you will need to read through thecontext in order to determine what type of answer would be most appropriate. After carefully reading throughthe context, you should then look at each of the answer choices and selectthe one that best fits with what you have read

Research Ideas This dataset can be used to develop and evaluate commonsense-based reading comprehension models. This dataset can be used to improve and customize question answering systems for educational or customer service applications. This dataset can be used to study how human beings process and understand narratives, in order to better design artificial intelligence systems that can do the same

Columns File: validation.csv

Column name Description context The context of the question. (String) answer0 The first answer option. (String) answer1 The second answer option. (String) answer2 The third answer option. (String) answer3 The fourth answer option. (String) label The correct answer to the question. (String) File: train.csv

Column name Description context The context of the question. (String) answer0 The first answer option. (String) answer1 The second answer option. (String) answer2 The third answer option. (String) answer3 The fourth answer option. (String) label The correct answer to the question. (String) File: test.csv

Column name Description context The context of the question. (String) answer0 The first answer option. (String) answer1 The second answer option. (String) answer2 The third answer option. (String) answer3 The fourth answer option. (String) label The correct answer to the question. (String)

License

CC0

Original Data Source: Cosmos QA (Commonsense QA)

nerdai/fedrag-commonsense-qa dataset hosted on Hugging Face and contributed by the HF Datasets community

This dataset contains 1789 data instances with problem identification, missing resource, time-dependent questions and answers pairs for disaster management.

Commonsense QA CoT (Partial, Raw, No Human Annotation)

  Dataset Summary

Seeded by the CommonsenseQA dataset (tau/commonsense_qa) this preliminary set randomly samples 1,000 question-answer entries and uses Mixtral (mistralai/Mixtral-8x7B-Instruct-v0.1) to generate 3 unique CoT (Chain-of-Thought) rationales. This was created as the preliminary step towards fine-tuning a LM (language model) to specialize on commonsense reasoning. The working hypothesis, inspired by the… See the full description on the dataset page: https://huggingface.co/datasets/peterkchung/commonsense_cot_partial_raw.

Commonsense QA CoT (Partial, Annotated) v0.1

  Dataset Summary

This dataset is a human-annotated subset of randomly sampled question-answer entries from the CommonsenseQA dataset (tau/commonsense_qa). The 'rationales' for each QA pair were created using a two-part method. First, Mixtral (mistralai/Mixtral-8x7B-Instruct-v0.1) was used to generate 3 unique CoT (Chain-of-Thought) explanations. Next, human evaluation was applied to distill the random sampling down to a cohesive… See the full description on the dataset page: https://huggingface.co/datasets/peterkchung/commonsense_cot_partial_annotated_v0.1.

We introduce Social IQa: Social Interaction QA, a new question-answering benchmark for testing social commonsense intelligence. Contrary to many prior benchmarks that focus on physical or taxonomic knowledge, Social IQa focuses on reasoning about people’s actions and their social implications. For example, given an action like "Jesse saw a concert" and a question like "Why did Jesse do this?", humans can easily infer that Jesse wanted "to see their favorite performer" or "to enjoy the music", and not "to see what's happening inside" or "to see if it works". The actions in Social IQa span a wide variety of social situations, and answer candidates contain both human-curated answers and adversarially-filtered machine-generated candidates. Social IQa contains over 37,000 QA pairs for evaluating models’ abilities to reason about the social implications of everyday events and situations. (Less)

Physical IQa: Physical Interaction QA, a new commonsense QA benchmark for naive physics reasoning focusing on how we interact with everyday objects in everyday situations. This dataset focuses on affordances of objects, i.e., what actions each physical object affords (e.g., it is possible to use a shoe as a doorstop), and what physical interactions a group of objects afford (e.g., it is possible to place an apple on top of a book, but not the other way around). The dataset requires reasoning about both the prototypical use of objects (e.g., shoes are used for walking) and non-prototypical but practically plausible use of objects (e.g., shoes can be used as a doorstop). The dataset includes 20,000 QA pairs that are either multiple-choice or true/false questions.

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('piqa', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

PIQA is a dataset for commonsense reasoning, and was created to investigate the physical knowledge of existing models in NLP.

Rainbow is multi-task benchmark for common-sense reasoning that uses different existing QA datasets: aNLI, Cosmos QA, HellaSWAG. Physical IQa, Social IQa, WinoGrande.

Russian reading comprehension with Commonsense reasoning (RuCoS) is a large-scale reading comprehension dataset that requires commonsense reasoning. RuCoS consists of queries automatically generated from CNN/Daily Mail news articles; the answer to each query is a text span from a summarizing passage of the corresponding news. The goal of RuCoS is to evaluate a machine`s ability of commonsense reasoning in reading comprehension.

Example {'source': 'Lenta', 'passage': { 'text': 'Мать двух мальчиков, брошенных отцом в московском аэропорту Шереметьево, забрала их. Об этом сообщили ТАСС в пресс-службе министерства образования и науки Хабаровского края. Сейчас младший ребенок посещает детский сад, а старший ходит в школу. В учебных заведениях с ними по необходимости работают штатные психологи. Также министерство социальной защиты населения рассматривает вопрос о бесплатном оздоровлении детей в летнее время. Через несколько дней после того, как Виктор Гаврилов бросил своих детей в аэропорту, он явился с повинной к следователям в городе Батайске Ростовской области. @context Бросившего детей в Шереметьево отца задержали за насилие над женой @context Россиянина заподозрили в истязании брошенных в Шереметьево детей @context Оставивший двоих детей в Шереметьево россиянин сам пришел к следователям', 'entities': [ {'start': 60, 'end': 71, 'text': 'Шереметьево'}, {'start': 102, 'end': 106, 'text': 'ТАСС'}, {'start': 155, 'end': 172, 'text': 'Хабаровского края'}, {'start': 470, 'end': 485, 'text': 'Виктор Гаврилов'}, {'start': 563, 'end': 571, 'text': 'Батайске'}, {'start': 572, 'end': 590, 'text': 'Ростовской области'}, {'start': 620, 'end': 631, 'text': 'Шереметьево'}, {'start': 725, 'end': 736, 'text': 'Шереметьево'}, {'start': 778, 'end': 789, 'text': 'Шереметьево'} ] }, 'qas': [ { 'query': '26 января @placeholder бросил сыновей в возрасте пяти и семи лет в Шереметьево.', 'answers': [ {'start': 470, 'end': 485, 'text': 'Виктор Гаврилов'} ], 'idx': 0 } ], 'idx': 0 }

How did we collect data? All text examples were collected from open news sources, then automatically filtered with QA systems to prevent obvious questions to infiltrate the dataset. The texts were then filtered by IPM frequency of the contained words and, finally, manually reviewed.

Synthetic CommonSense

Generated using ChatGPT4, originally from https://huggingface.co/datasets/commonsense_qa Notebook at https://github.com/mesolitica/malaysian-dataset/tree/master/question-answer/chatgpt4-commonsense

synthetic-commonsense.jsonl, 36332 rows, 7.34 MB.

  Example data

{'question': '1. Seseorang yang bersara mungkin perlu kembali bekerja jika mereka apa? A. mempunyai hutang B. mencari pendapatan C. meninggalkan pekerjaan D. memerlukan… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/chatgpt4-commonsense-qa.

Visual Question Answering (VQA) v2.0 is a dataset containing open-ended questions about images. These questions require an understanding of vision, language and commonsense knowledge to answer. It is the second version of the VQA dataset.

265,016 images (COCO and abstract scenes) At least 3 questions (5.4 questions on average) per image 10 ground truth answers per question 3 plausible (but likely incorrect) answers per question Automatic evaluation metric

The first version of the dataset was released in October 2015.