Dataset Summary Early beta release of pre-parsed English and French Wikipedia articles including infoboxes. Inviting feedback.

This dataset contains all articles of the English and French language editions of Wikipedia, pre-parsed and outputted as structured JSON files with a consistent schema. Each JSON line holds the content of one full Wikipedia article stripped of extra markdown and non-prose sections (references, etc.).

Invitation for Feedback The dataset is built as part of the Structured Contents initiative and based on the Wikimedia Enterprise html snapshots. It is an early beta release to improve transparency in the development process and request feedback. This first version includes pre-parsed Wikipedia abstracts, short descriptions, main images links, infoboxes and article sections, excluding non-prose sections (e.g. references). More elements (such as lists and tables) may be added over time. For updates follow the project’s blog and our Mediawiki Quarterly software updates on MediaWiki. As this is an early beta release, we highly value your feedback to help us refine and improve this dataset. Please share your thoughts, suggestions, and any issues you encounter either on the discussion page of Wikimedia Enterprise’s homepage on Meta wiki, or on the discussion page for this dataset here on Kaggle.

The contents of this dataset of Wikipedia articles is collectively written and curated by a global volunteer community. All original textual content is licensed under the GNU Free Documentation License (GFDL) and the Creative Commons Attribution-Share-Alike 4.0 License. Some text may be available only under the Creative Commons license; see the Wikimedia Terms of Use for details. Text written by some authors may be released under additional licenses or into the public domain.

The dataset in its structured form is generally helpful for a wide variety of tasks, including all phases of model development, from pre-training to alignment, fine-tuning, updating/RAG as well as testing/benchmarking. We would love to hear more about your use cases.

Data Fields The data fields are the same among all, noteworthy included fields: name - title of the article. identifier - ID of the article. url - URL of the article. version: metadata related to the latest specific revision of the article version.editor - editor-specific signals that can help contextualize the revision version.scores - returns assessments by ML models on the likelihood of a revision being reverted. main entity - Wikidata QID the article is related to. abstract - lead section, summarizing what the article is about. description - one-sentence description of the article for quick reference. image - main image representing the article's subject. infoboxes - parsed information from the side panel (infobox) on the Wikipedia article. sections - parsed sections of the article, including links. Note: excludes other media/images, lists, tables and references or similar non-prose sections. Full data dictionary is available here: https://enterprise.wikimedia.com/docs/data-dictionary/

Curation Rationale This dataset has been created as part of the larger Structured Contents initiative at Wikimedia Enterprise with the aim of making Wikimedia data more machine readable. These efforts are both focused on pre-parsing Wikipedia snippets as well as connecting the different projects closer together. Even if Wikipedia is very structured to the human eye, it is a non-triv...

Wikipedia dataset containing cleaned articles of all languages. The datasets are built from the Wikipedia dump (https://dumps.wikimedia.org/) with one split per language. Each example contains the content of one full Wikipedia article with cleaning to strip markdown and unwanted sections (references, etc.).

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('wikipedia', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

Dataset Summary

This is a dataset that can be used for research into machine learning and natural language processing. It contains all titles and summaries (or introductions) of English Wikipedia articles, extracted in September of 2017. The dataset is different from the regular Wikipedia dump and different from the datasets that can be created by gensim because ours contains the extracted summaries and not the entire unprocessed page body. This could be useful if one wants to use… See the full description on the dataset page: https://huggingface.co/datasets/jordiclive/wikipedia-summary-dataset.

Dataset 1: Wikipedia Article Metadata and Content Distribution (2019–2023)

This dataset represents metadata and structural information extracted from Wikipedia articles across multiple language editions between January 2019 and December 2023. The data was collected through the Wikimedia REST API and Wikidata Query Service, focusing on high-level article characteristics such as content length, number of references, topic classification, and readership activity. Each row corresponds to a unique Wikipedia article identified by an article_id and includes metadata describing its topic category (e.g., Politics, Science, Culture), geographic focus, and quality assessment.

The dataset was designed to help quantify content inequality and topic bias across languages. For example, English and German editions tend to have more extensive coverage of scientific and technological topics, while Swahili and Arabic editions show higher representation of local cultural and geographical content but fewer high-quality (“Featured Article”) designations. Article-level metrics like word_count, references_count, and page_views were gathered to provide indicators of article depth, credibility, and public engagement. The last_edit_date variable helps capture how frequently articles are updated, indicating editorial activity over time.

Temporal coverage: 2019–2023 Data sources: Wikimedia REST API, Wikidata Query Service, Pageview Analytics Primary purpose: To analyze disparities in article depth, topic diversity, and regional focus across Wikipedia’s major language editions.

Dataset 2: Wikipedia Editor Demographics and Contribution Data (2018–2023)

This dataset summarizes demographic and contribution patterns of active Wikipedia editors from 2018 to 2023, based on public edit histories available through the Wikimedia Dumps and MediaWiki API. Each record corresponds to a unique editor identified by editor_id, containing attributes such as country, primary language of editing, total edit counts, and dominant topic area.

Although Wikipedia does not directly record personal information, country and language data were inferred using IP-based geolocation for anonymous edits and user-declared data for registered contributors. The dataset was sampled to capture editors across seven major languages (English, French, Spanish, German, Swahili, Arabic, and Chinese). Demographic variables like gender and education_level are approximations derived from community surveys conducted by the Wikimedia Foundation in 2019 and 2021, used here to represent broad participation trends rather than individual identities.

This dataset provides insight into editorial imbalance, highlighting, for example, that editors from Europe and North America contribute disproportionately more to technical and scientific topics compared to those from Africa or South America. Fields such as total_edits, articles_edited, and avg_edit_size reflect productivity and depth of engagement, while active_since helps trace editor retention and historical participation.

Temporal coverage: 2018–2023 Data sources: Wikimedia Dumps, MediaWiki API, Wikimedia Community Surveys (2019, 2021) Primary purpose: To analyze demographic participation gaps and editing activity distribution across languages and regions.

Dataset 3: Wikipedia Language and Geographic Coverage Statistics (2023)

This dataset presents aggregated statistics at the language edition level, representing Wikipedia’s overall content and contributor structure as of December 2023. The data was compiled from the Wikimedia Statistics Portal and Meta-Wiki language reports, which provide high-level metrics such as total number of articles, average article length, number of active editors, and editing intensity per language.

Each entry represents one Wikipedia language edition, capturing its global footprint and coverage balance. The column coverage_score is a composite index derived from article volume, diversity of covered topics, and proportional representation of countries and regions. underrepresented_regions indicates the number of global regions (out of ten defined by the UN geoscheme) that have low coverage or minimal article representation in that language edition. The dataset allows researchers to identify which language Wikipedias most effectively cover global topics and which remain regionally or linguistically constrained.

simple-wikipedia

Processed, text-only dump of the Simple Wikipedia (English). Contains 23,886,673 words.

The dataset used in the paper is a subset of the Wikipedia corpus, consisting of 7500 English Wikipedia articles belonging to one of the following categories: People, Cities, Countries, Universities, and Novels.

Using Wikipedia data to study AI ethics.

Dataset Card for WIT

  Dataset Summary

Wikimedia's version of the Wikipedia-based Image Text (WIT) Dataset, a large multimodal multilingual dataset. From the official blog post:

The core training data is taken from the Wikipedia Image-Text (WIT) Dataset, a large curated set of more than 37 million image-text associations extracted from Wikipedia articles in 108 languages that was recently released by Google Research. The WIT dataset offers extremely valuable data about the… See the full description on the dataset page: https://huggingface.co/datasets/wikimedia/wit_base.

We provide a corpus of discussion comments from English Wikipedia talk pages. Comments are grouped into different files by year. Comments are generated by computing diffs over the full revision history and extracting the content added for each revision. See our wiki for documentation of the schema and our research paper for documentation on the data collection and processing methodology.

Quantitative features extracted from Wikipedia dumps for the inference of computational trust. Dumps provided at:https://dumps.wikimedia.org/Files used:XML dump Portuguese: ptwiki-20200820-stub-meta-history.xmlXML dump Italian: itwiki-20200801-stub-meta-history.xml

Dataset

This dataset was created by JAYAPRAKASHPONDY

Released under CC0: Public Domain

Contents

Wizard of Wikipedia is a recent, large-scale dataset of multi-turn knowledge-grounded dialogues between a “apprentice” and a “wizard”, who has access to information from Wikipedia documents.

Dataset

This dataset was created by Mohamed Fawzy

Contents

This dataset comprises raw data extracted from Wikipedia, encompassing various types of content including articles, metadata, and user interactions. The dataset is in its unprocessed form, providing an excellent opportunity for data enthusiasts and professionals to engage in data cleaning and preprocessing tasks. It is ideal for those looking to practice and enhance their data cleaning skills, as well as for researchers and developers who require a rich and diverse corpus for natural language processing (NLP) projects.

Dataset

This dataset was created by ABEL BIHINDA

Released under Apache 2.0

Contents

Three corpora in different domains extracted from Wikipedia.For all datasets, the figures and tables have been filtered out, as well as the categories and "see also" sections.The article structure, and particularly the sub-titles and paragraphs are kept in these datasets.

Wines: Wikipedia wines dataset consists of 1635 articles from the wine domain. The extracted dataset consists of a non-trivial mixture of articles, including different wine categories, brands, wineries, grape types, and more. The ground-truth recommendations were crafted by a human sommelier, which annotated 92 source articles with ~10 ground-truth recommendations for each sample. Examples for ground-truth expert-based recommendations are Dom Pérignon - Moët & Chandon, Pinot Meunier - Chardonnay.

Movies: The Wikipedia movies dataset consists of 100385 articles describing different movies. The movies' articles may consist of text passages describing the plot, cast, production, reception, soundtrack, and more. For this dataset, we have extracted a test set of ground truth annotations for 50 source articles using the "BestSimilar" database. Each source articles is associated with a list of ${\scriptsize \sim}12$ most similar movies. Examples for ground-truth expert-based recommendations are Schindler's List - The PianistLion King - The Jungle Book.

Video games: The Wikipedia video games dataset consists of 21,935 articles reviewing video games from all genres and consoles. Each article may consist of a different combination of sections, including summary, gameplay, plot, production, etc. Examples for ground-truth expert-based recommendations are: Grand Theft Auto - Mafia, Burnout Paradise - Forza Horizon 3.

Wikipedia is the largest and most read online free encyclopedia currently existing. As such, Wikipedia offers a large amount of data on all its own contents and interactions around them, as well as different types of open data sources. This makes Wikipedia a unique data source that can be analyzed with quantitative data science techniques. However, the enormous amount of data makes it difficult to have an overview, and sometimes many of the analytical possibilities that Wikipedia offers remain unknown. In order to reduce the complexity of identifying and collecting data on Wikipedia and expanding its analytical potential, after collecting different data from various sources and processing them, we have generated a dedicated Wikipedia Knowledge Graph aimed at facilitating the analysis, contextualization of the activity and relations of Wikipedia pages, in this case limited to its English edition. We share this Knowledge Graph dataset in an open way, aiming to be useful for a wide range of researchers, such as informetricians, sociologists or data scientists.

There are a total of 9 files, all of them in tsv format, and they have been built under a relational structure. The main one that acts as the core of the dataset is the page file, after it there are 4 files with different entities related to the Wikipedia pages (category, url, pub and page_property files) and 4 other files that act as "intermediate tables" making it possible to connect the pages both with the latter and between pages (page_category, page_url, page_pub and page_link files).

The document Dataset_summary includes a detailed description of the dataset.

Thanks to Nees Jan van Eck and the Centre for Science and Technology Studies (CWTS) for the valuable comments and suggestions.

A subset of articles extracted from the French Wikipedia XML dump. Data published here include 5 different categories : Economy (Economie), History (Histoire), Informatics (Informatique), Health (Medecine) and Law (Droit). The Wikipedia dump was downloaded on November 8, 2016 from https://dumps.wikimedia.org/. Each article is a xml file extracted from the dump and save as UTF8 plain text. The characteristics of dataset is :

• Economy : 44'876 articles
• History : 92'041 articles
• Informatics : 25'408 articles
• Health : 22'143 articles
• Law : 9'964 articles

User interaction networks of Wikipedia of 28 different languages. Nodes (orininal wikipedia user IDs) represent users of the Wikipedia, and an edge from user A to user B denotes that user A wrote a message on the talk page of user B at a certain timestamp.

More info: http://yfiua.github.io/academic/2016/02/14/wiki-talk-datasets.html

With this feature the user is able to extend CSV datasets with existing information in the Wikidata KG. The tool applies entity linking to all concepts in the same column and enable the user to use the extracted entities to extend the dataset.