Forecast: Re-Import of Beer to the UK 2022 - 2026 Discover more data with ReportLinker!

PMC-Patients ReCDS BenchmarkThe PMC-Patients ReCDS benchmark is presented as retrieval tasks and the data format is the same as BEIR benchmark. To be specific, there are queries, corpus, and qrels (annotations).### QueriesReCDS-PAR and ReCDS-PPR tasks share the same query patient set and dataset split.For each split (train, dev, and test), queries are stored a jsonl file that contains a list of dictionaries, each with two fields: - _id: unique query identifier represented by patient_uid.- text: query text represented by patient summary text.### CorpusCorpus is shared by different splits. For ReCDS-PAR, the corpus contains 11.7M PubMed articles, and for ReCDS-PPR, the corpus contains 155.2k reference patients from PMC-Patients. The corpus is also presented by a jsonl file that contains a list of dictionaries with three fields:- _id: unique document identifier represented by PMID of the PubMed article in ReCDS-PAR, and patient_uid of the candidate patient in ReCDS-PPR.- title: : title of the article in ReCDS-PAR, and empty string in ReCDS-PPR.- text: abstract of the article in ReCDS-PAR, and patient summary text in ReCDS-PPR.### QrelsQrels are TREC-style retrieval annotation files in tsv format.A qrels file contains three tab-separated columns, i.e. the query identifier, corpus identifier, and score in this order. The scores (2 or 1) indicate the relevance level in ReCDS-PAR or similarity level in ReCDS-PPR.Note that the qrels may not be the same as relevant_articles and similar_patients in PMC-Patients.json due to dataset split (see our manuscript for details).

Dataset Card

  Dataset Details

This dataset contains a set of candidate documents for second-stage re-ranking on nfcorpus (test split in BEIR). Those candidate documents are composed of hard negatives mined from gtr-t5-xl as Stage 1 ranker and ground-truth documents that are known to be relevant to the query. This is a release from our paper Policy-Gradient Training of Language Models for Ranking, so please cite it if using this dataset.

  Direct Use

You… See the full description on the dataset page: https://huggingface.co/datasets/NeuralPGRank/nfcorpus-hard-negatives.

Dataset Card

  Dataset Details

This dataset contains a set of candidate documents for second-stage re-ranking on trec-covid (test split in BEIR). Those candidate documents are composed of hard negatives mined from gtr-t5-xl as Stage 1 ranker and ground-truth documents that are known to be relevant to the query. This is a release from our paper Policy-Gradient Training of Language Models for Ranking, so please cite it if using this dataset.

  Direct Use

You… See the full description on the dataset page: https://huggingface.co/datasets/NeuralPGRank/trec-covid-hard-negatives.

Dataset Card

  Dataset Details

This dataset contains a set of candidate documents for second-stage re-ranking on climate-fever (test split in BEIR). Those candidate documents are composed of hard negatives mined from gtr-t5-xl as Stage 1 ranker and ground-truth documents that are known to be relevant to the query. This is a release from our paper Policy-Gradient Training of Language Models for Ranking, so please cite it if using this dataset.

  Direct Use… See the full description on the dataset page: https://huggingface.co/datasets/NeuralPGRank/climate-fever-hard-negatives.

Dataset Card

  Dataset Details

This dataset contains a set of candidate documents for second-stage re-ranking on webis-touche2020 (test split in BEIR). Those candidate documents are composed of hard negatives mined from gtr-t5-xl as Stage 1 ranker and ground-truth documents that are known to be relevant to the query. This is a release from our paper Policy-Gradient Training of Language Models for Ranking, so please cite it if using this dataset.

  Direct Use… See the full description on the dataset page: https://huggingface.co/datasets/NeuralPGRank/webis-touche2020-hard-negatives.