LAION Occupation

This dataset is a subset of LAION-2B-en containing 1.8M samples, each assigned to one of 153 occupations. This dataset was curated as part of our investigation into gender-occupation biases in LAION presented in Fair Diffusion. For downloading the images, check out img2dataset.

  Data Collection

We identified relevant images in the dataset by computing their CLIP similarity to a textual description of the target occupation. All descriptions were in the… See the full description on the dataset page: https://huggingface.co/datasets/AIML-TUDA/laion-occupation.

LAION 5B is a large-scale dataset for research purposes consisting of 5,85B CLIP-filtered image-text pairs. 2,3B contain English language, 2,2B samples from 100+ other languages and 1B samples have texts that do not allow a certain language assignment (e.g. names ). Additionally, we provide several nearest neighbor indices, an improved web interface for exploration & subset creation as well as detection scores for watermark and NSFW.

LAION-400M is a dataset with CLIP-filtered 400 million image-text pairs, their CLIP embeddings and kNN indices that allow efficient similarity search.

⚠️ Disclaimer & Content Warning (from the authors) Our filtering protocol only removed NSFW images detected as illegal, but the dataset still has NSFW content accordingly marked in the metadata. When freely navigating through the dataset, keep in mind that it is a large-scale, non-curated set crawled from the internet for research purposes, such that collected links may lead to discomforting and disturbing content. Therefore, please use the demo links with caution. You can extract a “safe” subset by filtering out samples drawn with NSFW or via stricter CLIP filtering.

There is a certain degree of duplication because we used URL+text as deduplication criteria. The same image with the same caption may sit at different URLs, causing duplicates. The same image with other captions is not, however, considered duplicated.

Using KNN clustering should make it easy to further deduplicate by image content.

LAION-400M The world’s largest openly available image-text-pair dataset with 400 million samples. # Concept and Content The LAION-400M dataset is completely openly, freely accessible. All images and texts in the LAION-400M dataset have been filtered with OpenAI‘s CLIP by calculating the cosine similarity between the text and image embeddings and dropping those with a similarity below 0.3 The threshold of 0.3 had been determined through human evaluations and seems to be a good heuristic for estimating semantic image-text-content matching. The image-text-pairs have been extracted from the Common Crawl web data dump and are from random web pages crawled between 2014 and 2021. # Download Information You can find The CLIP image embeddings (NumPy files) The parquet files KNN index of image embeddings # LAION-400M Dataset Statistics The LAION-400M and future even bigger ones are in fact datasets of datasets. For instance, it can be filtered out by image sizes into smaller datasets like th

About this Repository

This repository is the training split of the complete FreeSound LAION 640k dataset, limited only to licenses that permit commercial works, resampled to 16khz using torchaudio.transforms.Resample. This is ideal for use cases where a variety of audio is desired but fidelity and labels are unnecessary, such as background audio for augmenting other datasets.

  Dataset Versions

You are looking at the full dataset which contains 403,146 unique sounds… See the full description on the dataset page: https://huggingface.co/datasets/benjamin-paine/freesound-laion-640k-commercial-16khz-full.

A large-scale dataset of text and images for training next-generation language models.

The LAION-2B dataset used in the pre-training of the diffusion model. The dataset consists of 2.17B images, including public and private domains.

The dataset used in the paper is LAION-2B, which is a large-scale image-text dataset. The authors fine-tune a pre-trained diffusion model with a subset of LAION-2B with 10k randomly selected samples.

About this Repository

This repository is a re-upload of the FreeSound.org dataset as curated by LAION for the larger LAION-Audio-630k dataset, with the following changes:

Limited columns to only the audio and basic metadata. Incorporated necessary information for licensing and attribution. Removed ambiguously licensed samples, amounting to around 1,000 total samples.

  What about download links?

Links were ommitted for the sake of size, as they can be constructed from… See the full description on the dataset page: https://huggingface.co/datasets/benjamin-paine/freesound-laion-640k.

The dataset used in the paper is not explicitly described, but it is mentioned that the authors used the LAION-5B dataset for training.

LAION-COCO is the world’s largest dataset of 600M generated high-quality captions for publicly available web-images. The images are extracted from the english subset of Laion-5B with an ensemble of BLIP L/14 and 2 CLIP versions (L/14 and RN50x64). This dataset allow models to produce high quality captions for images.

A single line street base map representing the city's streets and other linear geographic features, along with feature names and address ranges for each addressable street segment. This dataset includes the Nodes file. The Nodes file contains a point feature and unique NodeID for each node that exists in the LION file. The Node_StreetName.txt file lists the street names associated with those nodes. Most nodes, representing intersections, will have at least 2 street names associated in the Node_StreetName.txt file. All previously released versions of this data are available at BYTES of the BIG APPLE - Archive.

Stable Diffusion and LAION are used as training datasets for the FakeInversion model.

A subset of the LAION 5B samples with English captions, obtained using LAION-Aesthetics_Predictor V2 625K image-text pairs with predicted aesthetics scores of 6.5 or higher available at https://huggingface.co/datasets/ChristophSchuhmann/improved_aesthetics_6.5plus

Dataset

This dataset was created by Nathan Smith

Released under Attribution 4.0 International (CC BY 4.0)

Contents

LAION-Aesthetics 6.5+ dataset contains 625K image-text pairs.

The dataset used in the paper is LAION-Aesthetic, a large-scale image dataset.

The LION Differences File (LDF) documents segment and node level changes that have occurred in the LION file between two subsequent releases. This file allows a user who “ties” organizational data to DCP’s Segment ID and/or Node ID to migrate their data appropriately when these changes occur.

Dataset

This dataset was created by 卡皮巴拉

Contents

laion/laions_got_talent_enhanced_flash_annotations_and_long_captions dataset hosted on Hugging Face and contributed by the HF Datasets community