StrategyQA is a question answering benchmark where the required reasoning steps are implicit in the question, and should be inferred using a strategy. It includes 2,780 examples, each consisting of a strategy question, its decomposition, and evidence paragraphs. Questions in StrategyQA are short, topic-diverse, and cover a wide range of strategies.

Ko-StrategyQA

This dataset represents a conversion of the Ko-StrategyQA dataset into the BeIR format, making it compatible for use with mteb. The original dataset was designed for multi-hop QA, so we processed the data accordingly. First, we grouped the evidence documents tagged by annotators into sets, and excluded unit questions containing 'no_evidence' or 'operation'.

njf/StrategyQA dataset hosted on Hugging Face and contributed by the HF Datasets community

dogtooth/strategyqa dataset hosted on Hugging Face and contributed by the HF Datasets community

Dataset

This dataset was created by Mahbubur Rahman

Contents

CR-LT-KGQA is the first Knowledge Graph Query Answering dataset with Natural Language queries targeted on long-tail Wikidata entities, answering which requires commonsense reasoning and is submitted to SIGIR'24. CR-LT-KGQA: A Knowledge Graph Question Answering Dataset Requiring Commonsense Reasoning and Long-Tail Knowledge The dataset contains two subsets targetting two different tasks: (i) Question Answering subset containing 200 questions based on StrategyQA dataset and (ii) Claim… See the full description on the dataset page: https://huggingface.co/datasets/yuansui/CL-LT-KGQA.

Ko-StrategyQA An MTEB dataset Massive Text Embedding Benchmark

Ko-StrategyQA

Task category t2t

Domains None

Reference

  How to evaluate on this task

You can evaluate an embedding model on this dataset using the following code: import mteb

task = mteb.get_tasks(["Ko-StrategyQA"]) evaluator = mteb.MTEB(task)

model = mteb.get_model(YOUR_MODEL) evaluator.run(model)

To learn more about how to run models on mteb task check out the GitHub repitory.… See the full description on the dataset page: https://huggingface.co/datasets/mteb/Ko-StrategyQA.

kozistr/ko-strategy-qa dataset hosted on Hugging Face and contributed by the HF Datasets community