CC100 English
- kaggle.com
Updated Nov 23, 2022+ more versions
Share
Facebook
Twitter
EmailClick to copy linkLink copiedCiteKrish Baisoya (2022). CC100 English [Dataset]. https://www.kaggle.com/datasets/krishbaisoya/cc100-english/codeCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedNov 23, 2022AuthorsKrish BaisoyaDescriptionDataset
This dataset was created by Krish Baisoya
Contents
- h
filtered_cc100_25gb
- huggingface.co
Updated Mar 26, 2025ShareFacebookTwitterEmailClick to copy linkLink copiedCiteXingming Li (2025). filtered_cc100_25gb [Dataset]. https://huggingface.co/datasets/xmli/filtered_cc100_25gbDataset updatedMar 26, 2025AuthorsXingming LiLicenseApache License, v2.0https://www.apache.org/licenses/LICENSE-2.0
License information was derived automaticallyDescription
CC100 dataset comprises of monolingual data for 100+ languages and also includes data for romanized languages. This was constructed using the urls and paragraph indices provided by the CC-Net repository by processing January-December 2018 Commoncrawl snapshots. Each file comprises of documents separated by double-newlines and paragraphs within the same document separated by a newline. The data is generated using the open source CC-Net repository. This dataset loader implements streaming to iterate over CC100 dataset. It applies strict filtering criteria to remove short, noisy, or repetitive sentences and keeps the language proportions similar to the ones used for XLM-R pre-training. The filtered CC100 dataset is ~25 GB. - h
filtered_cc100_minimal
- huggingface.co
Updated Mar 23, 2025ShareFacebookTwitterEmailClick to copy linkLink copiedCiteXingming Li (2025). filtered_cc100_minimal [Dataset]. https://huggingface.co/datasets/xmli/filtered_cc100_minimalDataset updatedMar 23, 2025AuthorsXingming LiLicenseApache License, v2.0https://www.apache.org/licenses/LICENSE-2.0
License information was derived automaticallyDescriptionCC100 dataset comprises of monolingual data for 100+ languages and also includes data for romanized languages. This was constructed using the urls and paragraph indices provided by the CC-Net repository by processing January-December 2018 Commoncrawl snapshots. Each file comprises of documents separated by double-newlines and paragraphs within the same document separated by a newline. The data is generated using the open source CC-Net repository. This dataset loader implements streaming to iterate over CC100 dataset. It applies strict filtering criteria to remove short, noisy, or repetitive sentences and keeps the language proportions similar to the ones used for XLM-R pre-training. The filtered CC100 dataset is ~25 GB.
- h
cc100-ko-390M-uncleaned
- huggingface.co
Updated Apr 4, 2023+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCiteWOO HWAN PARK (2023). cc100-ko-390M-uncleaned [Dataset]. https://huggingface.co/datasets/richard-park/cc100-ko-390M-uncleanedCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedApr 4, 2023AuthorsWOO HWAN PARKDescriptionrichard-park/cc100-ko-390M-uncleaned dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
cc100-ko-only-5-of-5
- huggingface.co
+ more versionsShareFacebookTwitterEmailClick to copy linkLink copiedCiteChang W Lee, cc100-ko-only-5-of-5 [Dataset]. https://huggingface.co/datasets/lcw99/cc100-ko-only-5-of-5CroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.AuthorsChang W LeeDescriptionlcw99/cc100-ko-only-5-of-5 dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
cc-100-korean-processing
- huggingface.co
Updated Mar 18, 2025ShareFacebookTwitterEmailClick to copy linkLink copiedCitecc-100-korean-processing [Dataset]. https://huggingface.co/datasets/CocoRoF/cc-100-korean-processingCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedMar 18, 2025AuthorsHaryeomDescriptionCocoRoF/cc-100-korean-processing dataset hosted on Hugging Face and contributed by the HF Datasets community
- h
cc-100-korean
- huggingface.co
Updated Mar 18, 2025ShareFacebookTwitterEmailClick to copy linkLink copiedCiteHaryeom (2025). cc-100-korean [Dataset]. https://huggingface.co/datasets/CocoRoF/cc-100-koreanCroissantCroissant is a format for machine-learning datasets. Learn more about this at mlcommons.org/croissant.Dataset updatedMar 18, 2025AuthorsHaryeomDescriptionCC-100-ko
Reference: (https://data.statmt.org/cc-100/)
- O
nepalitext-language-model-dataset
- opendatalab.com
- huggingface.co
zipUpdated Dec 28, 2023ShareFacebookTwitterEmailClick to copy linkLink copiedCite(2023). nepalitext-language-model-dataset [Dataset]. https://opendatalab.com/OpenDataLab/nepalitext-language-model-datasetzipAvailable download formatsDataset updatedDec 28, 2023LicenseCC0 1.0 Universal Public Domain Dedicationhttps://creativecommons.org/publicdomain/zero/1.0/
License information was derived automaticallyDescriptionNepaliText" language modeling dataset is a collection of over 13 million Nepali text sequences (phrases/sentences/paragraphs) extracted by combining the datasets: OSCAR , cc100 and a set of scraped Nepali articles on Wikipedia.
Not seeing a result you expected?
Learn how you can add new datasets to our index.
FacebookTwitterEmailDataset
This dataset was created by Krish Baisoya