GitHub Issues & Kaggle Notebooks

  Description

GitHub Issues & Kaggle Notebooks is a collection of two code datasets intended for language models training, they are sourced from GitHub issues and notebooks in Kaggle platform. These datasets are a modified part of the StarCoder2 model training corpus, precisely the bigcode/StarCoder2-Extras dataset. We reformat the samples to remove StarCoder2's special tokens and use natural text to delimit comments in issues and display… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceTB/issues-kaggle-notebooks.

Dataset

This dataset was created by BowlOFruits

Contents

kaggle-map/data dataset hosted on Hugging Face and contributed by the HF Datasets community

All GPT-4 Generated Datasets

Every chat dataset generated by GPT-4 from Huggingface at the same format

From [Huggingface datasets]

About this dataset

How to use the dataset

The dataset includes all chat conversations generated by GPT-4 that are hosted on open Huggingface datasets. Everything is converted to the same format so the datasets can be easily merged and used for large scale training of LLMs.

Acknowledgements

This dataset is a collection of several single chat datasets. If you use this dataset in your research, please credit the original authors of the internal datasets. Data Source

License

License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Gholamreza/test-dataset-kaggle dataset hosted on Hugging Face and contributed by the HF Datasets community

This dataset is a subset of https://huggingface.co/datasets/hieunguyenminh/roleplay

It is used to exemplify the fine-tune of Gemma 2 2B model with roleplay data where we use samples of dialogues user/agent (with a system prompt/description) for each character (personality) we want to teach Gemma to imitate.

For training, we will process the "text" column to extract triplets {system | user | assistant} and compose the prompts with which we fine-tune the model.

Dataset

This dataset was created by xhlulu

Contents

https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2352583%2F868a18fb09d7a1d3da946d74a9857130%2FLogo.PNG?generation=1604973725053566&alt=media" alt="">

Medical Dataset for Abbreviation Disambiguation for Natural Language Understanding (MeDAL) is a large medical text dataset curated for abbreviation disambiguation, designed for natural language understanding pre-training in the medical domain. It was published at the ClinicalNLP workshop at EMNLP.

💻 Code 🤗 Dataset (Hugging Face) 💾 Dataset (Kaggle) 💽 Dataset (Zenodo) 📜 Paper (ACL) 📝 Paper (Arxiv) ⚡ Pre-trained ELECTRA (Hugging Face)

Downloading the data

We recommend downloading from Kaggle if you can authenticate through their API. The advantage to Kaggle is that the data is compressed, so it will be faster to download. Links to the data can be found at the top of the readme.

First, you will need to create an account on kaggle.com. Afterwards, you will need to install the kaggle API: pip install kaggle

Then, you will need to follow the instructions here to add your username and key. Once that's done, you can run: kaggle datasets download xhlulu/medal-emnlp

Now, unzip everything and place them inside the data directory: unzip -nq crawl-300d-2M-subword.zip -d data mv data/pretrain_sample/* data/

Loading FastText Embeddings

For the LSTM models, we will need to use the fastText embeddings. To do so, first download and extract the weights: wget -nc -P data/ https://dl.fbaipublicfiles.com/fasttext/vectors-english/crawl-300d-2M-subword.zip unzip -nq data/crawl-300d-2M-subword.zip -d data/

Model Quickstart

Using Torch Hub

You can directly load LSTM and LSTM-SA with torch.hub: ```python import torch

lstm = torch.hub.load("BruceWen120/medal", "lstm") lstm_sa = torch.hub.load("BruceWen120/medal", "lstm_sa") ```

If you want to use the Electra model, you need to first install transformers: pip install transformers Then, you can load it with torch.hub: python import torch electra = torch.hub.load("BruceWen120/medal", "electra")

Using Huggingface transformers

If you are only interested in the pre-trained ELECTRA weights (without the disambiguation head), you can load it directly from the Hugging Face Repository:

from transformers import AutoModel, AutoTokenizer

model = AutoModel.from_pretrained("xhlu/electra-medal")
tokenizer = AutoTokenizer.from_pretrained("xhlu/electra-medal")

Citation

Download the bibtex here, or copy the text below: @inproceedings{wen-etal-2020-medal, title = "{M}e{DAL}: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining", author = "Wen, Zhi and Lu, Xing Han and Reddy, Siva", booktitle = "Proceedings of the 3rd Clinical Natural Language Processing Workshop", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", url = "https://www.aclweb.org/anthology/2020.clinicalnlp-1.15", pages = "130--135", }

License, Terms and Conditions

The ELECTRA model is licensed under Apache 2.0. The license for the libraries used in this project (transformers, pytorch, etc.) can be found in their respective GitHub repository. Our model is released under a MIT license.

The original dataset was retrieved and modified from the NLM website. By using this dataset, you are bound by the terms and conditions specified by NLM:

INTRODUCTION

Downloading data from the National Library of Medicine FTP servers indicates your acceptance of the following Terms and Conditions: No charges, usage fees or royalties are paid to NLM for this data.

MEDLINE/PUBMED SPECIFIC TERMS

NLM freely provides PubMed/MEDLINE data. Please note some PubMed/MEDLINE abstracts may be protected by copyright.

GENERAL TERMS AND CONDITIONS

• Users of the data agree to:

  • acknowledge NLM as the source of the data by including the phrase "Courtesy of the U.S. National Library of Medicine" in a clear and conspicuous manner,
  • properly use registration and/or trademark symbols when referring to NLM products, and
  • not indicate or imply that NLM has endorsed its products/services/applications.

• Users who republish or redistribute the data (services, products or raw data) agree to:

  • maintain the most current version of all distributed data, or
  • make known in a clear and conspicuous manner that the products/services/applications do not reflect the most current/accurate data available from NLM.

• These data are produced with a reasonable standard of care, but NLM makes no warranties express or implied, including no warranty of merchantability or fitness for particular purpose, regarding the accuracy or completeness of the data. Users agree to hold NLM and the U.S. Government harmless from any liability resulting from errors in the data. NLM disclaims any liability for any consequences due to use, misuse, or interpretation of information contained or not contained in the data.

• NLM does not provide legal advice regarding copyright, fair use, or other aspects of intellectual property rights. See the NLM Copyright page.

• NLM reserves the right to change the type and format of its machine-readable data. NLM will take reasonable steps to inform users of any changes to the format of the data before the data are distributed via the announcement section or subscription to email and RSS updates.

Dataset

This dataset was created by Moein Shariatnia

Contents

Dataset

This dataset was created by Kuro

Contents

Dataset

This dataset was created by Liyan Tang

Contents

From https://www.piwheels.org/project/huggingface-hub/

Dataset

This dataset was created by FullEmpty

Contents

Bitsandbytes 0.44.1

kaggle-aimo/aime_filtered dataset hosted on Hugging Face and contributed by the HF Datasets community

tsk-18/refined-kaggle dataset hosted on Hugging Face and contributed by the HF Datasets community

Dataset

This dataset was created by raghavendrakuttala

Contents

Dataset

This dataset was created by Maris Galesloot

Contents

tasksource/kaggle-claim-type dataset hosted on Hugging Face and contributed by the HF Datasets community

The Community-Driven Model Service Platform market is experiencing robust growth, projected to reach $35.14 billion in 2025 and maintain a Compound Annual Growth Rate (CAGR) of 10.1% from 2025 to 2033. This expansion is fueled by several key factors. The increasing adoption of machine learning and artificial intelligence across diverse sectors, coupled with the need for readily accessible and collaboratively improved models, is driving significant demand. The open-source nature of many platforms fosters innovation and reduces barriers to entry for both developers and businesses. Furthermore, the rise of cloud-based solutions offers scalability and cost-effectiveness, contributing to market expansion. The platform's segmentation into adult and children's applications reflects diverse use cases, ranging from sophisticated research projects to educational tools, further broadening its appeal. The presence of established players like Kaggle, GitHub, and Hugging Face indicates a maturing market with strong community engagement, while the existence of on-premises options caters to businesses with stringent data security requirements. Geographical expansion is also a significant contributor to growth, with North America and Europe currently leading the market, while Asia-Pacific is poised for significant future expansion driven by increasing digitalization and technological advancements. The market's continued growth is anticipated to be driven by advancements in model training techniques, the development of more user-friendly interfaces, and the increasing integration of these platforms with other data science tools and workflows. Challenges remain, however, such as ensuring data quality and addressing potential biases in community-contributed models. Furthermore, regulatory concerns around data privacy and model transparency will need to be carefully addressed to maintain sustainable growth. The competitive landscape is expected to remain dynamic, with ongoing innovation and consolidation among existing players and the emergence of new entrants. The strategic focus on improving model accessibility, enhancing community engagement, and expanding into new geographical markets will be key determinants of success in this rapidly evolving sector.

Dataset

This dataset was created by tk

Contents