Saved datasets
arrow_drop_downLast updated
arrow_drop_downDownload format
arrow_drop_downUsage rights
License from data provider
Please review the applicable license to make sure your contemplated use is permitted.
arrow_drop_downTopic
Free
Cost to access
Described as free to access or have a license that allows redistribution.
2 datasets found
- z
Webis Wikipedia Text Reuse Corpus 2018 (Webis-Wikipedia-Text-Reuse-18)
- zenodo.org
gz, zipUpdated Jul 5, 2018 - z
Webis Wikipedia Text Reuse Corpus 2018 (Webis-Wikipedia-Text-Reuse-18)
- zenodo.org
bz2, xzUpdated Jul 5, 2018 
Not seeing a result you expected?
Learn how you can add new datasets to our index.
Share
Facebook
Twitter
EmailClick to copy link
Link copied
Cite
Milad Alshomary; Michael Völske; Henning Wachsmuth; Benno Stein; Matthias Hagen; Martin Potthast (2018). Webis Wikipedia Text Reuse Corpus 2018 (Webis-Wikipedia-Text-Reuse-18) [Dataset]. http://doi.org/10.5281/zenodo.3372485
gz, zipAvailable download formats
Unique identifier
https://doi.org/10.5281/zenodo.3372485Dataset updated
Jul 5, 2018Dataset provided by
Universität Leipzig Bauhaus-Universität Weimar
Universität Paderborn
Martin-Luther-Universität Halle-Wittenberg
Authors
Milad Alshomary; Michael Völske; Henning Wachsmuth; Benno Stein; Matthias Hagen; Martin Potthast License
Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
Description
The Wikipedia Text Reuse Corpus 2018 (Webis-Wikipedia-Text-Reuse-18) containing text reuse cases extracted from within Wikipedia and in between Wikipedia and a sample of the Common Crawl.
The corpus has following structure:
- wikipedia.tar.gz: Each line, representing a Wikipedia article, contains a json array of article_id, article_title, and article_body
- within-wikipedia-tr-01.gz: Each line, representing a text reuse case, contains a json array of s_id (source article id), t_id (target article id), s_text (source text), t_text (target text)
- within-wikipedia-tr-02.gz: Each line, representing a text reuse case, contains a json array of s_id (source article id), t_id (target article id), s_text (source text), t_text (target text)
The datasets were extracted in the work by Alshomary et al. 2018 that aimed to study the text reuse phenomena related to Wikipedia at scale. A pipeline for large scale text reuse extraction was developed and used on Wikipedia and the CommonCrawl.