Motivation

This dataset was created to pilot techniques for creating synthetic data from datasets containing sensitive and protected information in the local government context. Synthetic data generation replaces actual data with representative data generated from statistical models; this preserves the key data properties that allow insights to be drawn from the data while protecting the privacy of the people included in the data. We invite you to read the Understanding Synthetic Data white paper for a concise introduction to synthetic data.

This effort was a collaboration of the Urban Institute, Allegheny County’s Department of Human Services (DHS) and CountyStat, and the University of Pittsburgh’s Western Pennsylvania Regional Data Center.

Collection

The source data for this project consisted of 1) month-by-month records of services included in Allegheny County's data warehouse and 2) demographic data about the individuals who received the services. As the County’s data warehouse combines this service and client data, this data is referred to as “Integrated Services data”. Read more about the data warehouse and the kinds of services it includes here.

Preprocessing

Synthetic data are typically generated from probability distributions or models identified as being representative of the confidential data. For this dataset, a model of the Integrated Services data was used to generate multiple versions of the synthetic dataset. These different candidate datasets were evaluated to select for publication the dataset version that best balances utility and privacy. For high-level information about this evaluation, see the Synthetic Data User Guide.

For more information about the creation of the synthetic version of this data, see the technical brief for this project, which discusses the technical decision making and modeling process in more detail.

Recommended Uses

This disaggregated synthetic data allows for many analyses that are not possible with aggregate data (summary statistics). Broadly, this synthetic version of this data could be analyzed to better understand the usage of human services by people in Allegheny County, including the interplay in the usage of multiple services and demographic information about clients.

Known Limitations/Biases

Some amount of deviation from the original data is inherent to the synthetic data generation process. Specific examples of limitations (including undercounts and overcounts for the usage of different services) are given in the Synthetic Data User Guide and the technical report describing this dataset's creation.

Feedback

Please reach out to this dataset's data steward (listed below) to let us know how you are using this data and if you found it to be helpful. Please also provide any feedback on how to make this dataset more applicable to your work, any suggestions of future synthetic datasets, or any additional information that would make this more useful. Also, please copy wprdc@pitt.edu on any such feedback (as the WPRDC always loves to hear about how people use the data that they publish and how the data could be improved).

Further Documentation and Resources

1) A high-level overview of synthetic data generation as a method for protecting privacy can be found in the Understanding Synthetic Data white paper.
2) The Synthetic Data User Guide provides high-level information to help users understand the motivation, evaluation process, and limitations of the synthetic version of Allegheny County DHS's Human Services data published here.
3) Generating a Fully Synthetic Human Services Dataset: A Technical Report on Synthesis and Evaluation Methodologies describes the full technical methodology used for generating the synthetic data, evaluating the various options, and selecting the final candidate for publication.
4) The WPRDC also hosts the Allegheny County Human Services Community Profiles dataset, which provides annual updates on human-services usage, aggregated by neighborhood/municipality. That data can be explored using the County's Human Services Community Profile web site.

Dataset Card for synthetic-data-generation-with-llama3-405B

This dataset has been created with distilabel.

  Dataset Summary

This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI: distilabel pipeline run --config "https://huggingface.co/datasets/lukmanaj/synthetic-data-generation-with-llama3-405B/raw/main/pipeline.yaml"

or explore the configuration: distilabel pipeline info… See the full description on the dataset page: https://huggingface.co/datasets/lukmanaj/synthetic-data-generation-with-llama3-405B.

BackgroundClinical data is instrumental to medical research, machine learning (ML) model development, and advancing surgical care, but access is often constrained by privacy regulations and missing data. Synthetic data offers a promising solution to preserve privacy while enabling broader data access. Recent advances in large language models (LLMs) provide an opportunity to generate synthetic data with reduced reliance on domain expertise, computational resources, and pre-training.ObjectiveThis study aims to assess the feasibility of generating realistic tabular clinical data with OpenAI’s GPT-4o using zero-shot prompting, and evaluate the fidelity of LLM-generated data by comparing its statistical properties to the Vital Signs DataBase (VitalDB), a real-world open-source perioperative dataset.MethodsIn Phase 1, GPT-4o was prompted to generate a dataset with qualitative descriptions of 13 clinical parameters. The resultant data was assessed for general errors, plausibility of outputs, and cross-verification of related parameters. In Phase 2, GPT-4o was prompted to generate a dataset using descriptive statistics of the VitalDB dataset. Fidelity was assessed using two-sample t-tests, two-sample proportion tests, and 95% confidence interval (CI) overlap.ResultsIn Phase 1, GPT-4o generated a complete and structured dataset comprising 6,166 case files. The dataset was plausible in range and correctly calculated body mass index for all case files based on respective heights and weights. Statistical comparison between the LLM-generated datasets and VitalDB revealed that Phase 2 data achieved significant fidelity. Phase 2 data demonstrated statistical similarity in 12/13 (92.31%) parameters, whereby no statistically significant differences were observed in 6/6 (100.0%) categorical/binary and 6/7 (85.71%) continuous parameters. Overlap of 95% CIs were observed in 6/7 (85.71%) continuous parameters.ConclusionZero-shot prompting with GPT-4o can generate realistic tabular synthetic datasets, which can replicate key statistical properties of real-world perioperative data. This study highlights the potential of LLMs as a novel and accessible modality for synthetic data generation, which may address critical barriers in clinical data access and eliminate the need for technical expertise, extensive computational resources, and pre-training. Further research is warranted to enhance fidelity and investigate the use of LLMs to amplify and augment datasets, preserve multivariate relationships, and train robust ML models.

Snapshot img

Synthetic Data Generation Market Size 2025-2029

The synthetic data generation market size is forecast to increase by USD 4.39 billion, at a CAGR of 61.1% between 2024 and 2029.

The market is experiencing significant growth, driven by the escalating demand for data privacy protection. With increasing concerns over data security and the potential risks associated with using real data, synthetic data is gaining traction as a viable alternative. Furthermore, the deployment of large language models is fueling market expansion, as these models can generate vast amounts of realistic and diverse data, reducing the reliance on real-world data sources. However, high costs associated with high-end generative models pose a challenge for market participants. These models require substantial computational resources and expertise to develop and implement effectively. Companies seeking to capitalize on market opportunities must navigate these challenges by investing in research and development to create more cost-effective solutions or partnering with specialists in the field. Overall, the market presents significant potential for innovation and growth, particularly in industries where data privacy is a priority and large language models can be effectively utilized.

What will be the Size of the Synthetic Data Generation Market during the forecast period?

Explore in-depth regional segment analysis with market size data - historical 2019-2023 and forecasts 2025-2029 - in the full report.
Request Free SampleThe market continues to evolve, driven by the increasing demand for data-driven insights across various sectors. Data processing is a crucial aspect of this market, with a focus on ensuring data integrity, privacy, and security. Data privacy-preserving techniques, such as data masking and anonymization, are essential in maintaining confidentiality while enabling data sharing. Real-time data processing and data simulation are key applications of synthetic data, enabling predictive modeling and data consistency. Data management and workflow automation are integral components of synthetic data platforms, with cloud computing and model deployment facilitating scalability and flexibility. Data governance frameworks and compliance regulations play a significant role in ensuring data quality and security. Deep learning models, variational autoencoders (VAEs), and neural networks are essential tools for model training and optimization, while API integration and batch data processing streamline the data pipeline. Machine learning models and data visualization provide valuable insights, while edge computing enables data processing at the source. Data augmentation and data transformation are essential techniques for enhancing the quality and quantity of synthetic data. Data warehousing and data analytics provide a centralized platform for managing and deriving insights from large datasets. Synthetic data generation continues to unfold, with ongoing research and development in areas such as federated learning, homomorphic encryption, statistical modeling, and software development. The market's dynamic nature reflects the evolving needs of businesses and the continuous advancements in data technology.

How is this Synthetic Data Generation Industry segmented?

The synthetic data generation industry research report provides comprehensive data (region-wise segment analysis), with forecasts and estimates in 'USD million' for the period 2025-2029, as well as historical data from 2019-2023 for the following segments. End-userHealthcare and life sciencesRetail and e-commerceTransportation and logisticsIT and telecommunicationBFSI and othersTypeAgent-based modellingDirect modellingApplicationAI and ML Model TrainingData privacySimulation and testingOthersProductTabular dataText dataImage and video dataOthersGeographyNorth AmericaUSCanadaMexicoEuropeFranceGermanyItalyUKAPACChinaIndiaJapanRest of World (ROW)

By End-user Insights

The healthcare and life sciences segment is estimated to witness significant growth during the forecast period.In the rapidly evolving data landscape, the market is gaining significant traction, particularly in the healthcare and life sciences sector. With a growing emphasis on data-driven decision-making and stringent data privacy regulations, synthetic data has emerged as a viable alternative to real data for various applications. This includes data processing, data preprocessing, data cleaning, data labeling, data augmentation, and predictive modeling, among others. Medical imaging data, such as MRI scans and X-rays, are essential for diagnosis and treatment planning. However, sharing real patient data for research purposes or training machine learning algorithms can pose significant privacy risks. Synthetic data generation addresses this challenge by producing realistic medical imaging data, ensuring data privacy while enabling research

Data Description

We release the synthetic data generated using the method described in the paper Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (ACL 2024 Findings). The external knowledge we use is based on LLM-generated topics and writing styles.

  Generated Datasets

The original train/validation/test data, and the generated synthetic training data are listed as follows. For each dataset, we generate 5000… See the full description on the dataset page: https://huggingface.co/datasets/ritaranx/clinical-synthetic-text-llm.

crate

Synthetic Data Generation Market Outlook 2032

The global synthetic data generation market size was USD 378.3 Billion in 2023 and is projected to reach USD 13,800 Billion by 2032, expanding at a CAGR of 31.1 % during 2024–2032. The market growth is attributed to the increasing demand for privacy-preserving synthetic data across the world.



Growing demand for privacy-preserving synthetic data is expected to boost the market. Synthetic data, being artificially generated, does not contain any personal or sensitive information, thereby ensuring data privacy. This has propelled organizations to adopt synthetic data generation methods, particularly in sectors where data privacy is paramount, such as healthcare and finance.

Impact of Artificial Intelligence (AI) in Synthetic Data Generation Market

Artificial Intelligence (AI) has significantly influenced the synthetic data generation market, transforming the way businesses operate and make decisions. The integration of AI in synthetic data generation has enhanced the efficiency and accuracy of data modeling, simulation, and analysis. AI algorithms, through machine learning and deep learning techniques, generate synthetic data that closely mimics real-world data, thereby providing a safe and effective alternative for data privacy concerns.



AI has led to the increased adoption of synthetic data in various sectors such as healthcare, finance, and retail, among others. Furthermore, AI-driven synthetic data generation aids in overcoming the challenges of data scarcity and bias, thereby improving the quality of predictive models and decision-making processes. The impact of AI on the synthetic data generation market is profound, fostering innovation, enhancing data security, and driving market growth. For instance,

• 
  

  In October 2023, K2view

Surveillance data play a vital role in estimating the burden of diseases, pathogens, exposures, behaviors, and susceptibility in populations, providing insights that can inform the design of policies and targeted public health interventions. The use of Health and Demographic Surveillance System (HDSS) collected from the Kilifi region of Kenya, has led to the collection of massive amounts of data on the demographics and health events of different populations. This has necessitated the adoption of tools and techniques to enhance data analysis to derive insights that will improve the accuracy and efficiency of decision-making. Machine Learning (ML) and artificial intelligence (AI) based techniques are promising for extracting insights from HDSS data, given their ability to capture complex relationships and interactions in data. However, broad utilization of HDSS datasets using AI/ML is currently challenging as most of these datasets are not AI-ready due to factors that include, but are not limited to, regulatory concerns around privacy and confidentiality, heterogeneity in data laws across countries limiting the accessibility of data, and a lack of sufficient datasets for training AI/ML models. Synthetic data generation offers a potential strategy to enhance accessibility of datasets by creating synthetic datasets that uphold privacy and confidentiality, suitable for training AI/ML models and can also augment existing AI datasets used to train the AI/ML models. These synthetic datasets, generated from two rounds of separate data collection periods, represent a version of the real data while retaining the relationships inherent in the data. For more information please visit The Aga Khan University Website.

In this research, we create synthetic data with features that are like data from IoT devices. We use an existing air quality dataset that includes temperature and gas sensor measurements. This real-time dataset includes component values for the Air Quality Index (AQI) and ppm concentrations for various polluting gas concentrations. We build a JavaScript Object Notation (JSON) model to capture the distribution of variables and structure of this real dataset to generate the synthetic data. Based on the synthetic dataset and original dataset, we create a comparative predictive model. Analysis of synthetic dataset predictive model shows that it can be successfully used for edge analytics purposes, replacing real-world datasets. There is no significant difference between the real-world dataset compared the synthetic dataset. The generated synthetic data requires no modification to suit the edge computing requirements. The framework can generate correct synthetic datasets based on JSON schema attributes. The accuracy, precision, and recall values for the real and synthetic datasets indicate that the logistic regression model is capable of successfully classifying data

Abstract

The dataset is a relational dataset of 8,000 households households, representing a sample of the population of an imaginary middle-income country. The dataset contains two data files: one with variables at the household level, the other one with variables at the individual level. It includes variables that are typically collected in population censuses (demography, education, occupation, dwelling characteristics, fertility, mortality, and migration) and in household surveys (household expenditure, anthropometric data for children, assets ownership). The data only includes ordinary households (no community households). The dataset was created using REaLTabFormer, a model that leverages deep learning methods. The dataset was created for the purpose of training and simulation and is not intended to be representative of any specific country.

The full-population dataset (with about 10 million individuals) is also distributed as open data.

Geographic coverage

The dataset is a synthetic dataset for an imaginary country. It was created to represent the population of this country by province (equivalent to admin1) and by urban/rural areas of residence.

Analysis unit

Household, Individual

Universe

The dataset is a fully-synthetic dataset representative of the resident population of ordinary households for an imaginary middle-income country.

Kind of data

ssd

Sampling procedure

The sample size was set to 8,000 households. The fixed number of households to be selected from each enumeration area was set to 25. In a first stage, the number of enumeration areas to be selected in each stratum was calculated, proportional to the size of each stratum (stratification by geo_1 and urban/rural). Then 25 households were randomly selected within each enumeration area. The R script used to draw the sample is provided as an external resource.

Mode of data collection

other

Research instrument

The dataset is a synthetic dataset. Although the variables it contains are variables typically collected from sample surveys or population censuses, no questionnaire is available for this dataset. A "fake" questionnaire was however created for the sample dataset extracted from this dataset, to be used as training material.

Cleaning operations

The synthetic data generation process included a set of "validators" (consistency checks, based on which synthetic observation were assessed and rejected/replaced when needed). Also, some post-processing was applied to the data to result in the distributed data files.

Response rate

This is a synthetic dataset; the "response rate" is 100%.

The Synthetic Data Generation Marketsize was valued at USD 288.5 USD Million in 2023 and is projected to reach USD 1920.28 USD Million by 2032, exhibiting a CAGR of 31.1 % during the forecast period.Synthetic data generation stands for the generation of fake datasets that resemble real datasets with reference to their data distribution and patterns. It refers to the process of creating synthetic data points utilizing algorithms or models instead of conducting observations or surveys. There is one of its core advantages: it can maintain the statistical characteristics of the original data and remove the privacy risk of using real data. Further, with synthetic data, there is no limitation to how much data can be created, and hence, it can be used for extensive testing and training of machine learning models, unlike the case with conventional data, which may be highly regulated or limited in availability. It also helps in the generation of datasets that are comprehensive and include many examples of specific situations or contexts that may occur in practice for improving the AI system’s performance. The use of SDG significantly shortens the process of the development cycle, requiring less time and effort for data collection as well as annotation. It basically allows researchers and developers to be highly efficient in their discovery and development in specific domains like healthcare, finance, etc. Key drivers for this market are: Growing Demand for Data Privacy and Security to Fuel Market Growth. Potential restraints include: Lack of Data Accuracy and Realism Hinders Market Growth. Notable trends are: Growing Implementation of Touch-based and Voice-based Infotainment Systems to Increase Adoption of Intelligent Cars.

AI-Generated Synthetic Tabular Dataset Market Outlook

According to our latest research, the AI-Generated Synthetic Tabular Dataset market size reached USD 1.42 billion in 2024 globally, reflecting the rapid adoption of artificial intelligence-driven data generation solutions across numerous industries. The market is expected to expand at a robust CAGR of 34.7% from 2025 to 2033, reaching a forecasted value of USD 19.17 billion by 2033. This exceptional growth is primarily driven by the increasing need for high-quality, privacy-preserving datasets for analytics, model training, and regulatory compliance, particularly in sectors with stringent data privacy requirements.

One of the principal growth factors propelling the AI-Generated Synthetic Tabular Dataset market is the escalating demand for data-driven innovation amidst tightening data privacy regulations. Organizations across healthcare, finance, and government sectors are facing mounting challenges in accessing and sharing real-world data due to GDPR, HIPAA, and other global privacy laws. Synthetic data, generated by advanced AI algorithms, offers a solution by mimicking the statistical properties of real datasets without exposing sensitive information. This enables organizations to accelerate AI and machine learning development, conduct robust analytics, and facilitate collaborative research without risking data breaches or non-compliance. The growing sophistication of generative models, such as GANs and VAEs, has further increased confidence in the utility and realism of synthetic tabular data, fueling adoption across both large enterprises and research institutions.

Another significant driver is the surge in digital transformation initiatives and the proliferation of AI and machine learning applications across industries. As businesses strive to leverage predictive analytics, automation, and intelligent decision-making, the need for large, diverse, and high-quality datasets has become paramount. However, real-world data is often siloed, incomplete, or inaccessible due to privacy concerns. AI-generated synthetic tabular datasets bridge this gap by providing scalable, customizable, and bias-mitigated data for model training and validation. This not only accelerates AI deployment but also enhances model robustness and generalizability. The flexibility of synthetic data generation platforms, which can simulate rare events and edge cases, is particularly valuable in sectors like finance and healthcare, where such scenarios are underrepresented in real datasets but critical for risk assessment and decision support.

The rapid evolution of the AI-Generated Synthetic Tabular Dataset market is also underpinned by technological advancements and growing investments in AI infrastructure. The availability of cloud-based synthetic data generation platforms, coupled with advancements in natural language processing and tabular data modeling, has democratized access to synthetic datasets for organizations of all sizes. Strategic partnerships between technology providers, research institutions, and regulatory bodies are fostering innovation and establishing best practices for synthetic data quality, utility, and governance. Furthermore, the integration of synthetic data solutions with existing data management and analytics ecosystems is streamlining workflows and reducing barriers to adoption, thereby accelerating market growth.

Regionally, North America dominates the AI-Generated Synthetic Tabular Dataset market, accounting for the largest share in 2024 due to the presence of leading AI technology firms, strong regulatory frameworks, and early adoption across industries. Europe follows closely, driven by stringent data protection laws and a vibrant research ecosystem. The Asia Pacific region is emerging as a high-growth market, fueled by rapid digitalization, government initiatives, and increasing investments in AI research and development. Latin America and the Middle East & Africa are also witnessing growing interest, particularly in sectors like finance and government, though market maturity varies across countries. The regional landscape is expected to evolve dynamically as regulatory harmonization, cross-border data collaboration, and technological advancements continue to shape market trajectories globally.

Synthetic data generated for the ECHILD training course (code available at https://github.com/UCL-CHIG/ECHILD_Synthetic), along with suggested solutions for practical exercises in R.

Title: Rule-based Synthetic Data for Japanese GEC. Dataset Contents:This dataset contains two parallel corpora intended for the training and evaluating of models for the NLP (natural language processing) subtask of Japanese GEC (grammatical error correction). These are as follows:Synthetic Corpus - synthesized_data.tsv. This corpus file contains 2,179,130 parallel sentence pairs synthesized using the process described in [1]. Each line of the file consists of two sentences delimited by a tab. The first sentence is the erroneous sentence while the second is the corresponding correction.These paired sentences are derived from data scraped from the keyword-lookup site

The data we used to evaluate Louvain Method in the study Benchmarking Graph Databases on the Problem of Community Detection. These data werw synthetically generated using the LFR-Benchmark (3rd link). There are two type of files, networkX.dat and communityX.dat. The networkX.dat file contains the list of edges (nodes are labelled from 1 to the number of nodes; the edges are ordered and repeated twice, i.e. source-target and target-source). The first four lines of the networkX.dat file list the parameters we used to generate the data. The communityX.dat file contains a list of the nodes and their membership (memberships are labelled by integer numbers >=1). Note X correspond to the number of nodes each dataset contains.

Synthetic dataset:

generated.csv - synthetic datasets containing 41,185 clinical note samples spanning 219 ICD-10 codes.

Data field Description

idx Unique sample identifier.

ICD-10 The targeted ICD-10 code used for prior data sampling.

generation_model The model used for sample generation (GTP-3.5, GPT-4, LLaMA-7b, LLaMA-13b)

prompt Prompt used for sample generation.

prior Type of prior data used for sample generation.

example Bool variable for the presence or… See the full description on the dataset page: https://huggingface.co/datasets/Glebkaa/MedSyn-synthetic.

This dataset is focuses on cardiovascular diseases. It is generated using a hybrid machine learning model that combines diffusion models with Transformers, emphasizing data privacy. The dataset has been meticulously validated for quality and utility, yielding auspicious results.Validation and Metrics:The dataset has undergone rigorous validation processes to ensure quality, utility, and privacy. These validations involved:

Distance to the Closest Record (DCR): The dataset achieved a DCR of 1.2879. The DCR is a metric that measures the distance of the generated data to the closest record in the original dataset. A higher DCR indicates that the synthetic data closely mirrors the real data in terms of statistical properties, making it reliable for further analysis and research.

Membership Inference Attack Accuracy: The dataset scored 0.6780 in this metric. Membership inference attack accuracy measures the likelihood of correctly inferring whether a particular data point was part of the training dataset. An accuracy of 0.6780 suggests that the model maintains a strong level of privacy. It is important to note that a score of 0.5 would indicate random guessing, hence the achieved score demonstrates significantly better privacy protection than random predictions.

Statistical Tests: Comprehensive statistical tests were conducted to compare the synthetic data with real data. These tests ensure that the synthetic data has similar statistical properties and distributions to the original data.

Machine Learning Efficiency: The utility of the dataset was also validated using machine learning models to ensure that the synthetic data is effective for training and can produce reliable predictive models. The results showed that models trained on this dataset performed well, reinforcing the practical utility of the data.

The high DCR value and the membership inference attack accuracy highlight the balance between data utility and privacy, making this dataset an invaluable resource for researchers and practitioners focusing on cardiovascular diseases and machine learning.

Quantum-AI Synthetic Data Generator Market Outlook

According to our latest research, the global Quantum-AI Synthetic Data Generator market size reached USD 1.82 billion in 2024, reflecting a robust expansion driven by technological advancements and increasing adoption across multiple industries. The market is projected to grow at a CAGR of 32.7% from 2025 to 2033, reaching a forecasted market size of USD 21.69 billion by 2033. This growth trajectory is primarily fueled by the rising demand for high-quality synthetic data to train artificial intelligence models, address data privacy concerns, and accelerate digital transformation initiatives across sectors such as healthcare, finance, and retail.

One of the most significant growth factors for the Quantum-AI Synthetic Data Generator market is the escalating need for vast, diverse, and privacy-compliant datasets to train advanced AI and machine learning models. As organizations increasingly recognize the limitations and risks associated with using real-world data, particularly regarding data privacy regulations like GDPR and CCPA, the adoption of synthetic data generation technologies has surged. Quantum computing, when integrated with artificial intelligence, enables the rapid and efficient creation of highly realistic synthetic datasets that closely mimic real-world data distributions while ensuring complete anonymity. This capability is proving invaluable for sectors like healthcare and finance, where data sensitivity is paramount and regulatory compliance is non-negotiable. As a result, organizations are investing heavily in Quantum-AI synthetic data solutions to enhance model accuracy, reduce bias, and streamline data sharing without compromising privacy.

Another key driver propelling the market is the growing complexity and volume of data generated by emerging technologies such as IoT, autonomous vehicles, and smart devices. Traditional data collection methods are often insufficient to keep pace with the data requirements of modern AI applications, leading to gaps in data availability and quality. Quantum-AI Synthetic Data Generators address these challenges by producing large-scale, high-fidelity synthetic datasets on demand, enabling organizations to simulate rare events, test edge cases, and improve model robustness. Additionally, the capability to generate structured, semi-structured, and unstructured data allows businesses to meet the specific needs of diverse applications, ranging from fraud detection in banking to predictive maintenance in manufacturing. This versatility is further accelerating market adoption, as enterprises seek to future-proof their AI initiatives and gain a competitive edge.

The integration of Quantum-AI Synthetic Data Generators into cloud-based platforms and enterprise IT ecosystems is also catalyzing market growth. Cloud deployment models offer scalability, flexibility, and cost-effectiveness, making synthetic data generation accessible to organizations of all sizes, including small and medium enterprises. Furthermore, the proliferation of AI-driven analytics in sectors such as retail, e-commerce, and telecommunications is creating new opportunities for synthetic data applications, from enhancing customer experience to optimizing supply chain operations. As vendors continue to innovate and expand their service offerings, the market is expected to witness sustained growth, with new entrants and established players alike vying for market share through strategic partnerships, product launches, and investments in R&D.

From a regional perspective, North America currently dominates the Quantum-AI Synthetic Data Generator market, accounting for over 38% of the global revenue in 2024, followed by Europe and Asia Pacific. The strong presence of leading technology companies, robust investment in AI research, and favorable regulatory environment contribute to North America's leadership position. Europe is also witnessing significant growth, driven by stringent data privacy regulations and increasing adoption of AI across industries. Meanwhile, the Asia Pacific region is emerging as a high-growth market, fueled by rapid digitalization, expanding IT infrastructure, and government initiatives promoting AI innovation. As regional markets continue to evolve, strategic collaborations and cross-border partnerships are expected to play a pivotal role in shaping the global landscape of the Quantum-AI Synthetic Data Generator market.

Component Analysis

Synthetic datasets (training/validation) for end-to-end Relation Extraction of relationships between Organisms and Natural-Products. The datasets are provided for reproducibility purposes, but, can also be used to train new models. As in the corresponding article, 3 subtypes of synthetic datasets are provided:

Diversity-synt: The seed literature references used in the generation process correspond to the top-500 extracted items per biological kingdoms using the GME-sampler. Random-synt: 5 datasets of equivalent sizes as Diversity-synt, but using randomly sampled seed literature references. Extended-synt: A merge of Diversity-synt and the 5 Random-synt datasets. All datasets were produced with Vicuna-13b-v1.3. Like the model, the produced synthetic data are also submitted to the License of the model used for generation, see the original LLaMA model card. LLaMA is licensed under the LLaMA License, Copyright (c) Meta Platforms, Inc. All Rights Reserved.

This repository provides the necessary data and Python code to replicate the experiments and generate the figures presented in our manuscript: "Supporting data and code: Beyond Economic Dispatch: Modeling Renewable Purchase Agreements in Production Cost Models".

Contents:

• pownet.zip: Contains PowNet version 3.2, the specific version of the simulation software used in this study.
• inputs.zip: Contains essential modeling inputs required by PowNet for the experiments, including network data, and pre-generated synthetic load and solar time series.
• scripts.zip: Contains the Python scripts used for installing PowNet, optionally regenerating synthetic data, running simulation experiments, processing results, and generating figures.
• thai_data.zip (Reference Only): Contains raw data related to the 2023 Thai power system. This data served as a reference during the creation of the PowNet inputs for this study but is not required to run the replication experiments themselves. Code to process the raw data is also provided.

System Requirements:

• Python version 3.10+
• pip package manager

Setup Instructions:

1. Download and Unzip Core Files: Download pownet.zip, inputs.zip, scripts.zip, and thai_data.zip. Extract their contents into the same parent folder. Your directory structure should look like this:

   Parent_Folder/
   ├── pownet/    # from pownet.zip
   ├── inputs/    # from inputs.zip
   ├── scripts/    # from scripts.zip
   ├── thai_data.zip/    # from scripts.zip
   ├── figures/    # Created by scripts later
   ├── outputs/    # Created by scripts later
   

2. Install PowNet:

   • Open your terminal or command prompt.
   • Navigate into the pownet directory that you just extracted:

cd path/to/Parent_Folder/pownet

pip install -e .

• • These commands install PowNet and its required dependencies into your active Python environment.

Workflow and Usage:

Note: All subsequent Python script commands should be run from the scripts directory. Navigate to it first:

cd path/to/Parent_Folder/scripts

1. Generate Synthetic Time Series (Optional):

• This step is optional as the required time series files are already provided within the inputs directory (extracted from inputs.zip). If you wish to regenerate them:
• Run the generation scripts:

  python create_synthetic_load.py
  
  python create_synthetic_solar.py
  

• Evaluate the generated time series (optional):

  python eval_synthetic_load.py
  
  python eval_synthetic_solar.py
  

2. Calculate Total Solar Availability:

• Process solar scenarios using data from the inputs directory:

  python process_scenario_solar.py
  

3. Experiment 1: Compare Strategies for Modeling Purchase Obligations:

• Run the base case simulations for different modeling strategies:
  • No Must-Take (NoMT):

    python run_basecase.py --model_name "TH23NMT"
    

  • Zero-Cost Renewables (ZCR):

    python run_basecase.py --model_name "TH23ZC"
    

  • Penalized Curtailment (Proposed Method):

    python run_basecase.py --model_name "TH23"
    

• Run the base case simulation for the Minimum Capacity (MinCap) strategy:

  python run_min_cap.py
  
  This is a new script because we need to modify the objective function and add constraints.

4. Experiment 2: Simulate Partial-Firm Contract Switching:

• Run simulations comparing the base case with the partial-firm contract scenario:
  • Base Case Scenario:

    python run_scenarios.py --model_name "TH23"
    

  • Partial-Firm Contract Scenario:

    python run_scenarios.py --model_name "TH23ESB"
    

5. Visualize Results:

• Generate all figures presented in the manuscript:

  python run_viz.py
  

• Figures will typically be saved in afigures directory within the Parent_Folder.