AI in Generative Adversarial Networks (GANs) Market Outlook

According to our latest research, the global AI in Generative Adversarial Networks (GANs) market size reached USD 2.65 billion in 2024, reflecting robust growth driven by rapid advancements in deep learning and artificial intelligence. The market is expected to register a remarkable CAGR of 31.4% from 2025 to 2033, accelerating the adoption of GANs across diverse industries. By 2033, the market is forecasted to achieve a value of USD 32.78 billion, underscoring the transformative impact of GANs in areas such as image and video generation, data augmentation, and synthetic content creation. This trajectory is supported by the increasing demand for highly realistic synthetic data and the expansion of AI-driven applications across enterprise and consumer domains.

A primary growth factor for the AI in Generative Adversarial Networks market is the exponential increase in the availability and complexity of data that organizations must process. GANs, with their unique adversarial training methodology, have proven exceptionally effective for generating realistic synthetic data, which is crucial for industries like healthcare, automotive, and finance where data privacy and scarcity are significant concerns. The ability of GANs to create high-fidelity images, videos, and even text has enabled organizations to enhance their AI models, improve data diversity, and reduce bias, thereby accelerating the adoption of AI-driven solutions. Furthermore, the integration of GANs with cloud-based platforms and the proliferation of open-source GAN frameworks have democratized access to this technology, enabling both large enterprises and SMEs to harness its potential for innovative applications.

Another significant driver for the AI in Generative Adversarial Networks market is the surge in demand for advanced content creation tools in media, entertainment, and marketing. GANs have revolutionized the way digital content is produced by enabling hyper-realistic image and video synthesis, deepfake generation, and automated design. This has not only streamlined creative workflows but also opened new avenues for personalized content, virtual influencers, and immersive experiences in gaming and advertising. The rapid evolution of GAN architectures, such as StyleGAN and CycleGAN, has further enhanced the quality and scalability of generative models, making them indispensable for enterprises seeking to differentiate their digital offerings and engage customers more effectively in a highly competitive landscape.

The ongoing advancements in hardware acceleration and AI infrastructure have also played a pivotal role in propelling the AI in Generative Adversarial Networks market forward. The availability of powerful GPUs, TPUs, and AI-specific chips has significantly reduced the training time and computational costs associated with GANs, making them more accessible for real-time and large-scale applications. Additionally, the growing ecosystem of AI services and consulting has enabled organizations to overcome technical barriers, optimize GAN deployments, and ensure compliance with evolving regulatory standards. As investment in AI research continues to surge, the GANs market is poised for sustained innovation and broader adoption across sectors such as healthcare diagnostics, autonomous vehicles, financial modeling, and beyond.

From a regional perspective, North America continues to dominate the AI in Generative Adversarial Networks market, accounting for the largest share in 2024, driven by its robust R&D ecosystem, strong presence of leading technology companies, and early adoption of AI technologies. Europe follows closely, with significant investments in AI research and regulatory initiatives promoting ethical AI development. The Asia Pacific region is emerging as a high-growth market, fueled by rapid digital transformation, expanding AI talent pool, and increasing government support for AI innovation. Latin America and the Middle East & Africa are also witnessing steady growth, albeit from a smaller base, as enterprises in these regions begin to explore the potential of GANs for industry-specific applications.

Component Analysis

The AI in Generative Adversarial Networks market is segmented by component into software, hardware, and services, each playing a vital role in the ecosystem’s development and adoption. Software solutions constitute the largest share of the market in 2024, reflecting the growing demand for ad

The potential of medical image analysis with neural networks is limited by the restricted availability of extensive data sets. The incorporation of synthetic training data is one approach to bypass this shortcoming, as synthetic data offer accurate annotations and unlimited data size. We evaluated eleven CycleGAN for the synthesis of computed tomography (CT) images based on XCAT body phantoms.

Context

All the images of faces here are generated using https://thispersondoesnotexist.com/

https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1842206%2F4c3d3569f4f9c12fc898d76390f68dab%2FBeFunky-collage.jpg?generation=1662079836729388&alt=media" alt="">

Copyrighting of AI Generated images

Under US copyright law, these images are technically not subject to copyright protection. Only "original works of authorship" are considered. "To qualify as a work of 'authorship' a work must be created by a human being," according to a US Copyright Office's report [PDF].

https://www.theregister.com/2022/08/14/ai_digital_artwork_copyright/

Tagging

I manually tagged all images as best as I could and separated them between the two classes below

• Female- 3860 images
• Male- 3013 images

Some may pass either female or male, but I will leave it to you to do the reviewing. I included toddlers and babies under Male/ Female

How it works

Each of the faces are totally fake, created using an algorithm called Generative Adversarial Networks (GANs).

A generative adversarial network (GAN) is a class of machine learning frameworks designed by Ian Goodfellow and his colleagues in June 2014. Two neural networks contest with each other in a game (in the form of a zero-sum game, where one agent's gain is another agent's loss).

Given a training set, this technique learns to generate new data with the same statistics as the training set. For example, a GAN trained on photographs can generate new photographs that look at least superficially authentic to human observers, having many realistic characteristics. Though originally proposed as a form of generative model for unsupervised learning, GANs have also proved useful for semi-supervised learning, fully supervised learning,and reinforcement learning.

• https://www.youtube.com/watch?v=u8qPvzk0AfY
• https://www.youtube.com/watch?v=dCKbRCUyop8
• https://www.youtube.com/watch?v=SWoravHhsUU

Github implementation of website

• https://github.com/NVlabs/stylegan2
• https://github.com/lucidrains/stylegan2-pytorch
• https://github.com/lucidrains/lightweight-gan

How I gathered the images

Just a simple Jupyter notebook that looped and invoked the website https://thispersondoesnotexist.com/ , saving all images locally

GAN-Synthesized Augmented Radiology Dataset Market Outlook

According to our latest research, the GAN-Synthesized Augmented Radiology Dataset market size reached USD 412 million in 2024, supported by a robust surge in the adoption of artificial intelligence across healthcare imaging. The market demonstrated a strong CAGR of 25.7% from 2021 to 2024 and is on track to reach a valuation of USD 3.2 billion by 2033. The primary growth factor fueling this expansion is the increasing demand for high-quality, diverse, and annotated radiology datasets to train and validate advanced AI diagnostic models, especially as regulatory requirements for clinical validation intensify globally.

The exponential growth of the GAN-Synthesized Augmented Radiology Dataset market is being driven by the urgent need for large-scale, diverse, and unbiased datasets in medical imaging. Traditional methods of acquiring and annotating radiological images are time-consuming, expensive, and often limited by patient privacy concerns. Generative Adversarial Networks (GANs) have emerged as a transformative technology, enabling the synthesis of high-fidelity, realistic medical images that can augment existing datasets. This not only enhances the statistical power and generalizability of AI models but also helps overcome the challenge of data imbalance, especially for rare diseases and underrepresented demographic groups. As AI-driven diagnostics become integral to clinical workflows, the reliance on GAN-augmented datasets is expected to intensify, further propelling market growth.

Another significant growth driver is the increasing collaboration between radiology departments, AI technology vendors, and academic research institutes. These partnerships are focused on developing standardized protocols for dataset generation, annotation, and validation, leveraging GANs to create synthetic images that closely mimic real-world clinical scenarios. The resulting datasets facilitate the training of AI algorithms for a wide array of applications, including disease detection, anomaly identification, and image segmentation. Additionally, the proliferation of cloud-based platforms and open-source AI frameworks has democratized access to GAN-synthesized datasets, enabling even smaller healthcare organizations and startups to participate in the AI-driven transformation of radiology.

The regulatory landscape is also evolving to support the responsible use of synthetic data in healthcare. Regulatory agencies in North America, Europe, and Asia Pacific are increasingly recognizing the value of GAN-generated datasets for algorithm validation, provided they meet stringent standards for data quality, privacy, and clinical relevance. This regulatory endorsement is encouraging more hospitals, diagnostic centers, and research institutions to adopt GAN-augmented datasets, further accelerating market expansion. Moreover, the ongoing advancements in GAN architectures, such as StyleGAN and CycleGAN, are enhancing the realism and diversity of synthesized images, making them virtually indistinguishable from real patient scans and boosting their acceptance in both clinical and research settings.

From a regional perspective, North America is currently the largest market for GAN-Synthesized Augmented Radiology Datasets, driven by substantial investments in healthcare AI, the presence of leading technology vendors, and proactive regulatory support. Europe follows closely, with a strong emphasis on data privacy and cross-border research collaborations. The Asia Pacific region is witnessing the fastest growth, fueled by rapid digital transformation in healthcare, rising investments in AI infrastructure, and increasing disease burden. Latin America and the Middle East & Africa are also emerging as promising markets, albeit at a slower pace, as healthcare systems in these regions begin to adopt AI-driven radiology solutions.

Dataset Type Analysis

The dataset type segment of the GAN-Synthesized Augmented Radiology Dataset market is pi

PRLx-GAN

Repository for Synthetic Generation and Latent Projection Denoising of Rim Lesions in Multiple Sclerosis published in Synthetic Data at CVPR 2025.

  Summary

Paramagnetic rim lesions (PRLs) are a rare but highly prognostic lesion subtype in multiple sclerosis, visible only on susceptibility ($\chi$) contrasts. This work presents a generative framework to:

Synthesize new rim lesion maps that address class imbalance in training data Enable a novel denoising… See the full description on the dataset page: https://huggingface.co/datasets/agr78/PRLx-GAN-synthetic-rim.

The NSL-KDD V2 is an extended version of NSL-KDD original dataset. The dataset is normalised and 1 additional class is synthesised by mixing multiple non-benign classes. To cite the dataset, please reference the original paper with DOI: 10.1109/SmartNets61466.2024.10577645. The paper is published in IEEE SmartNets and can be accessed https://www.researchgate.net/publication/382034618_Blender-GAN_Multi-Target_Conditional_Generative_Adversarial_Network_for_Novel_Class_Synthetic_Data_Generation . Citation info: Madhubalan, Akshayraj & Gautam, Amit & Tiwary, Priya. (2024). Blender-GAN: Multi-Target Conditional Generative Adversarial Network for Novel Class Synthetic Data Generation. 1-7. 10.1109/SmartNets61466.2024.10577645. This dataset was made by Abluva Inc, a Palo Alto based, research-driven Data Protection firm. Our data protection platform empowers customers to secure data through advanced security mechanisms such as Fine Grained Access control and sophisticated depersonalization algorithms (e.g. Pseudonymization, Anonymization and Randomization). Abluva's Data Protection solutions facilitate data democratization within and outside the organizations, mitigating the concerns related to theft and compliance. The innovative intrusion detection algorithm by Abluva employs patented technologies for an intricately balanced approach that excludes normal access deviations, ensuring intrusion detection without disrupting the business operations. Abluva’s Solution enables organizations to extract further value from their data by enabling secure Knowledge Graphs and deploying Secure Data as a Service among other novel uses of data. Committed to providing a safe and secure environment, Abluva empowers organizations to unlock the full potential of their data.

This dataset is part of the publication "Classification of Craniosynostosis Trained Only On Synthetic Data Using GANs, PCA, and Statistical Shape Models".

dataset28.zip includes 2D distance maps constructed of surface scans of craniosynostosis patients: sagittal suture fusion (scaphocephaly), metopic suture fusion (trigonocephaly), coronal suture fusion (brachycephaly and anterior plagiocephaly), and a control model (normocephaly and positional plagiocephaly).

synthetic_1000.zip contains are random 1000 samples per class created from each individual synthetic data source (GAN, PCA, statistical shape model).

This repository contains only the images. To synthesize your own data, please use the github repository.

The dataset is an extended version of UNSW-NB 15. It has 1 additional class synthesised and the data is normalised for ease of use. To cite the dataset, please reference the original paper with DOI: 10.1109/SmartNets61466.2024.10577645. The paper is published in IEEE SmartNets and can be accessed here: https://www.researchgate.net/publication/382034618_Blender-GAN_Multi-Target_Conditional_Generative_Adversarial_Network_for_Novel_Class_Synthetic_Data_Generation. Citation info: Madhubalan, Akshayraj & Gautam, Amit & Tiwary, Priya. (2024). Blender-GAN: Multi-Target Conditional Generative Adversarial Network for Novel Class Synthetic Data Generation. 1-7. 10.1109/SmartNets61466.2024.10577645. This dataset was made by Abluva Inc, a Palo Alto based, research-driven Data Protection firm. Our data protection platform empowers customers to secure data through advanced security mechanisms such as Fine Grained Access control and sophisticated depersonalization algorithms (e.g. Pseudonymization, Anonymization and Randomization). Abluva's Data Protection solutions facilitate data democratization within and outside the organizations, mitigating the concerns related to theft and compliance. The innovative intrusion detection algorithm by Abluva employs patented technologies for an intricately balanced approach that excludes normal access deviations, ensuring intrusion detection without disrupting the business operations. Abluva’s Solution enables organizations to extract further value from their data by enabling secure Knowledge Graphs and deploying Secure Data as a Service among other novel uses of data. Committed to providing a safe and secure environment, Abluva empowers organizations to unlock the full potential of their data.

Currently, predicting a person’s facial appearance many years later based on early facial features remains a core technical challenge. In this paper, we propose a cross-age face prediction framework based on Generative Adversarial Networks (GANs). This framework extracts key features from early photos of the target individual and predicts their facial appearance at different ages in the future. Within our framework, we designed a GAN-based image restoration algorithm to enhance image deblurring capabilities and improve the generation of fine details, thereby increasing image resolution. Additionally, we introduced a semi-supervised learning algorithm called Multi-scale Feature Aggregation Scratch Repair (Semi-MSFA), which leverages both synthetic datasets and real historical photos to better adapt to the task of restoring old photographs. Furthermore, we developed a generative adversarial network incorporating a self-attention mechanism to predict age-progressed face images, ensuring the generated images maintain relatively stable personal characteristics across different ages. To validate the robustness and accuracy of our proposed framework, we conducted qualitative and quantitative analyses on open-source portrait databases and volunteer-provided data. Experimental results demonstrate that our framework achieves high prediction accuracy and strong generalization capabilities.

The CIC-IDS-V2 is an extended version of the original CIC-IDS 2017 dataset. The dataset is normalised and 1 new class called "Comb" is added which is a combination of synthesised data of multiple non-benign classes.

To cite the dataset, please reference the original paper with DOI: 10.1109/SmartNets61466.2024.10577645. The paper is published in IEEE SmartNets and can be accessed here.

Citation info:

Madhubalan, Akshayraj & Gautam, Amit & Tiwary, Priya. (2024). Blender-GAN: Multi-Target Conditional Generative Adversarial Network for Novel Class Synthetic Data Generation. 1-7. 10.1109/SmartNets61466.2024.10577645.

This dataset was made by Abluva Inc, a Palo Alto based, research-driven Data Protection firm. Our data protection platform empowers customers to secure data through advanced security mechanisms such as Fine Grained Access control and sophisticated depersonalization algorithms (e.g. Pseudonymization, Anonymization and Randomization). Abluva's Data Protection solutions facilitate data democratization within and outside the organizations, mitigating the concerns related to theft and compliance. The innovative intrusion detection algorithm by Abluva employs patented technologies for an intricately balanced approach that excludes normal access deviations, ensuring intrusion detection without disrupting the business operations. Abluva’s Solution enables organizations to extract further value from their data by enabling secure Knowledge Graphs and deploying Secure Data as a Service among other novel uses of data. Committed to providing a safe and secure environment, Abluva empowers organizations to unlock the full potential of their data.

TifGAN

Monthly nighttime lights (NTL) can clearly depict an area's prevailing intra-year socio-economic dynamics. The Earth Observation Group at Colorado School of Mines provides monthly NTL products from the Day Night Band (DNB) sensor on board the Visible and Infrared Imaging Suite (VIIRS) satellite (April 2012 onwards) and from Operational Linescan System (OLS) sensor onboard the Defense Meteorological Satellite Program (DMSP) satellites (April 1992 onwards). In the current study, an attempt has been made to generate synthetic monthly VIIRS-like products of 1992-2012, using a deep learning-based image translation network. Initially, the defects of the 216 monthly DMSP images (1992-2013) were corrected to remove geometric errors, background noise, and radiometric errors. Correction on monthly VIIRS imagery to remove background noise and ephemeral lights was done using low and high thresholds. Improved DMSP and corrected VIIRS images from April 2012 - December 2013 are used in a conditional generative adversarial network (cGAN) along with Land Use Land Cover, as auxiliary input, to generate VIIRS-like imagery from 1992-2012. The modelled imagery was aggregated annually and showed an R2 of 0.94 with the results of other annual-scale VIIRS-like imagery products of India, R2 of 0.85 w.r.t GDP and R2 of 0.69 w.r.t population. Regression analysis of the generated VIIRS-like products with the actual VIIRS images for the years 2012 and 2013 over India indicated a good approximation with an R2 of 0.64 and 0.67 respectively, while the spatial density relation depicted an under-estimation of the brightness values by the model at extremely high radiance values with an R2 of 0.56 and 0.53 respectively. Qualitative analysis for also performed on both national and state scales. Visual analysis over 1992-2013 confirms a gradual increase in the brightness of the lights indicating that the cGAN model images closely represent the actual pattern followed by the nighttime lights. Finally, a synthetically generated monthly VIIRS-like product is delivered to the research community which will be useful for studying the changes in socio-economic dynamics over time.

The Turku UAS DeepSeaSalama-GAN dataset 1 (TDSS-G1) is a comprehensive image dataset obtained from a maritime environment. This dataset was assembled in the southwest Finnish archipelago area at Taalintehdas, using two stationary RGB fisheye cameras in the month of August 2022. The technical setup is described in the section “Sensor Platform design” in report “Development of Applied Research Platforms for Autonomous and Remotely Operated Systems” (https://www.theseus.fi/handle/10024/815628).

The data collection and annotation process was carried out in the Autonomous and Intelligent Systems laboratory at Turku University of Applied Sciences. The dataset is a blend of original images captured by our cameras and synthetic data generated by a Generative Adversarial Network (GAN), simulating 18 distinct weather conditions.

The TDSS-G1 dataset comprises 199 original images and a substantial addition of 3582 synthetic images, culminating in a total of 3781 annotated images. These images provide a diverse representation of various maritime objects, including motorboats, sailing boats, and seamarks.

The creation of TDSS-G1 involved extracting images from videos recorded in MPEG format, with a resolution of 720p at 30 frames per second (FPS). An image was extracted every 100 milliseconds.

The distribution of labels within TDSS-G1 is as follows: motorboats (62.1%), sailing boats (16.8%), and seamarks (21.1%).

This distribution highlights a class imbalance, with motorboats being the most represented class and sailing boats being the least. This imbalance is an important factor to consider during the model training process, as it could influence the model’s ability to accurately recognize underrepresented classes. In the future synthetic datasets, vision Transformers will be used to tackle this problem.

The TDSS-G1 dataset is organized into three distinct subsets for the purpose of training and evaluating machine learning models. These subsets are as follows:

• Training Set: Located in dataset/train/images, this set is used to train the model. It learns to recognize the different classes of maritime objects from this data.
• Validation Set: Stored in dataset/valid/images, this set is used to tune the model parameters and to prevent overfitting during the training process.
• Test Set: Found in dataset/test/images, this set is used to evaluate the final performance of the model. It provides an unbiased assessment of how the model will perform on unseen data.

The dataset comprises three classes (nc: 3), each representing a different type of maritime object. The classes are as follows:

1. Motor Boat (motor_boat)
2. Sailing Boat (sailing_boat)
3. Seamark (seamark)

These labels correspond to the annotated objects in the images. The model trained on this dataset will be capable of identifying these three types of maritime objects. As mentioned earlier, the distribution of these classes is imbalanced, which is an important factor to consider during the training process.

Deep learning models for diagnostic applications require large amounts of sensitive patient data, raising privacy concerns under centralized training paradigms. We propose FedGAN, a federated learning framework for synthetic medical image generation that combines Generative Adversarial Networks (GANs) with cross-silo federated learning. Our approach pretrains a DCGAN on abdominal CT scans and fine-tunes it collaboratively across clinical silos using diabetic retinopathy datasets. By federating the GAN’s discriminator and generator via the Federated Averaging (FedAvg) algorithm, FedGAN generates high-quality synthetic retinal images while complying with HIPAA and GDPR. Experiments demonstrate that FedGAN achieves a realism score of 0.43 (measured by a centralized discriminator). This work bridges data scarcity and privacy challenges in medical AI, enabling secure collaboration across institutions.

cigFacies is a dataset create by the Computational Interpretation Group (CIG) for the AI-based automatic seismic facies classification in 3-D seismic data, Hui Gao, Xinming Wu, Xiaoming Sun and Mingcai Hou are the main contributors to the dataset.

This is the benchmark skeletonization datasets of seismic facies, guided by the knowledge graph of seismic facies and constructed from three different stategies (field seismic data, synthetic data and GAN-based generation).

Below are some brief desription of the datasets:

1) The "The benchmark skeletonization datasets" file consists of 5 classes of seismic facies.

2) The "parallel_class", "clinoform_class", "fill_class", "hummocky_class" and "chaotic_class" consist of 2000, 1500, 1500, 1500, 1500 stratigraphic skeletonization data constructed from field seismic data, synthetic data and GAN-based generation, respectively.

The Mineral Photos dataset is a vast collection of over 39,000 images of mineral specimens across 15 distinct mineral categories. It is primarily designed for use in machine learning and computer vision tasks, particularly for mineral classification and image generation. Each category contains images of the minerals in various forms and lighting conditions, making it a comprehensive resource for training models to recognize and generate mineral images.

Key Features: Total Images: Over 39,000 high-quality photographs.

Mineral Categories: The dataset includes images from the following 15 mineral categories:

1. Azurite
2. Baryte
3. Beryl
4. Calcite
5. Cerussite
6. Copper
7. Fluorite
8. Gypsum
9. Hematite
10. Malachite
11. Pyrite
12. Pyromorphite
13. Quartz
14. Smithsonite 15.Wulfenite

Purpose: The dataset is perfect for training models in mineral classification and image generation tasks.

Use Case: Ideal for machine learning, image recognition, and deep learning applications in the classification and generation of images related to mineral ores.

Use Cases: Mineral Classification: Building models that can automatically classify mineral ores based on image data.

Image Generation: Using the dataset for generating synthetic images of minerals, which can be useful for training data augmentation or GAN-based projects.

Computer Vision: Training deep learning models for object recognition and classification in the field of mineralogy.

This dataset offers a valuable resource for those working on image-based machine learning models related to mineral identification, image synthesis, and visual pattern recognition in mineralogy.

IntroductionThe challenges associated with data availability, class imbalance, and the need for data augmentation are well-recognized in the field of plant disease detection. The collection of large-scale datasets for plant diseases is particularly demanding due to seasonal and geographical constraints, leading to significant cost and time investments. Traditional data augmentation techniques, such as cropping, resizing, and rotation, have been largely supplanted by more advanced methods. In particular, the utilization of Generative Adversarial Networks (GANs) for the creation of realistic synthetic images has become a focal point of contemporary research, addressing issues related to data scarcity and class imbalance in the training of deep learning models. Recently, the emergence of diffusion models has captivated the scientific community, offering superior and realistic output compared to GANs. Despite these advancements, the application of diffusion models in the domain of plant science remains an unexplored frontier, presenting an opportunity for groundbreaking contributions.MethodsIn this study, we delve into the principles of diffusion technology, contrasting its methodology and performance with state-of-the-art GAN solutions, specifically examining the guided inference model of GANs, named InstaGAN, and a diffusion-based model, RePaint. Both models utilize segmentation masks to guide the generation process, albeit with distinct principles. For a fair comparison, a subset of the PlantVillage dataset is used, containing two disease classes of tomato leaves and three disease classes of grape leaf diseases, as results on these classes have been published in other publications.ResultsQuantitatively, RePaint demonstrated superior performance over InstaGAN, with average Fréchet Inception Distance (FID) score of 138.28 and Kernel Inception Distance (KID) score of 0.089 ± (0.002), compared to InstaGAN’s average FID and KID scores of 206.02 and 0.159 ± (0.004) respectively. Additionally, RePaint’s FID scores for grape leaf diseases were 69.05, outperforming other published methods such as DCGAN (309.376), LeafGAN (178.256), and InstaGAN (114.28). For tomato leaf diseases, RePaint achieved an FID score of 161.35, surpassing other methods like WGAN (226.08), SAGAN (229.7233), and InstaGAN (236.61).DiscussionThis study offers valuable insights into the potential of diffusion models for data augmentation in plant disease detection, paving the way for future research in this promising field.

gender

Synthetic Faces High Quality (SFHQ) part 4

This dataset consists of 125,754 high quality 1024x1024 curated face images, and was created by first creating large amount of "text to image" generations (most from stable diffusion v2.1, some from stable diffusion v1.4) model and then creating several photo-realistic candidate images using a process similar to what is described in this short twitter thread which involve encoding the images into StyleGAN2 latent space and performing a small manipulation that turns each image into a high quality photo-realistic image candidate. Finally, we then sift through the resulting candidate images and keep only the good ones for the dataset.

The dataset also contains facial landmarks (extended set) and face parsing semantic segmentation maps. An example script is provided and demonstrates how to access landmarks, segmentation maps, and textually search withing the dataset (with CLIP image/text feature vectors), and also performs some exploratory analysis of the dataset. link to github repo of the dataset.

The process that corrects images generated by stable-diffusion and creates several candidate photo-realistic images is illustrated below: https://raw.githubusercontent.com/SelfishGene/SFHQ-dataset/main/images/bring_to_life_process_stable_diffusion.jpg" alt="">

More Details

1. The original inspiration images are generated images using stable diffusion v2.1 model (mostly) and using various face portrait prompts that span a wide range of ethnicities, ages, expressions, hairstyles, etc. Note that stable diffusion faces often contain errors in the generation so cannot be used to create a photo-reallistic dataset without a correcting model or an extremely lengthy manual curation process. we do both here.
2. Each inspiration image was encoded by encoder4editing (e4e) into StyleGAN2 latent space (StyleGAN2 is a generative face model tained on FFHQ dataset) and multiple candidate images were generated from each inspiration image
3. These candidate images were then further curated and verified as being photo-realistic and high quality by a single human (me) and a machine learning assistant model that was trained to approximate my own human judgments and helped me scale myself to asses the quality of all images in the dataset
4. Near duplicates and images that were too similar were removed using CLIP features (no two images in the dataset have CLIP similarity score of greater than ~0.9)
5. From each image various pre-trained features were extracted and provided here for convenience, in particular CLIP features for fast textual query of the dataset
6. From each image, semantic segmentation maps were extracted using Face Parsing BiSeNet and are provided in the dataset under "segmentations"
7. From each image, an extended landmark set was extracted that also contain inner and outer hairlines (these are unique landmarks that are usually not extracted by other algorithms). These landmarks were extracted using Dlib, Face Alignment and some post processing of Face Parsing BiSeNet and are provided in the dataset under "landmarks"
8. NOTE: semantic segmentation and landmarks were first calculated on scaled down version of 384x384 images, and then upscaled to 1024x1024

Parts 1,2,3,4

• Part 1 of the dataset consists of 89,785 HQ 1024x1024 curated face images. It uses "inspiration" images from Artstation-Artistic-face-HQ dataset (AAHQ), Close-Up Humans dataset and UIBVFED dataset.
• Part 2 of the dataset consists of 91,361 HQ 1024x1024 curated face images. It uses "inspiration" images from Face Synthetics dataset and by sampling from the Stable Diffusion v1.4 text to image generator using varied face portrait prompts.
• Part 3 of the dataset consists of 118,358 HQ 1024x1024 curated face images. It uses "inspiration" images by sampling from [StyleGAN2](https://git...