The synthetic data generation market is booming, projected to reach $10 billion by 2033 with a 25% CAGR. Learn about key drivers, trends, and major players shaping this rapidly expanding sector, including AI model training, data privacy, and software testing solutions. Discover market analysis and forecasts for synthetic data generation.

The Synthetic Data Generation market is booming, projected to reach $11.9 billion by 2033 with a 25% CAGR. Learn about key drivers, trends, and top companies shaping this rapidly expanding sector, addressing data privacy and AI model training needs. Explore market segmentation and regional analysis for a comprehensive overview.

Context

There is a lack of public available datasets on financial services and specially in the emerging mobile money transactions domain. Financial datasets are important to many researchers and in particular to us performing research in the domain of fraud detection. Part of the problem is the intrinsically private nature of financial transactions, that leads to no publicly available datasets.

We present a synthetic dataset generated using the simulator called PaySim as an approach to such a problem. PaySim uses aggregated data from the private dataset to generate a synthetic dataset that resembles the normal operation of transactions and injects malicious behaviour to later evaluate the performance of fraud detection methods.

Content

PaySim simulates mobile money transactions based on a sample of real transactions extracted from one month of financial logs from a mobile money service implemented in an African country. The original logs were provided by a multinational company, who is the provider of the mobile financial service which is currently running in more than 14 countries all around the world.

This synthetic dataset is scaled down 1/4 of the original dataset and it is created just for Kaggle.

NOTE: Transactions which are detected as fraud are cancelled, so for fraud detection these columns (oldbalanceOrg, newbalanceOrig, oldbalanceDest, newbalanceDest ) must not be used.

Headers

This is a sample of 1 row with headers explanation:

1,PAYMENT,1060.31,C429214117,1089.0,28.69,M1591654462,0.0,0.0,0,0

step - maps a unit of time in the real world. In this case 1 step is 1 hour of time. Total steps 744 (30 days simulation).

type - CASH-IN, CASH-OUT, DEBIT, PAYMENT and TRANSFER.

amount - amount of the transaction in local currency.

nameOrig - customer who started the transaction

oldbalanceOrg - initial balance before the transaction

newbalanceOrig - new balance after the transaction.

nameDest - customer who is the recipient of the transaction

oldbalanceDest - initial balance recipient before the transaction. Note that there is not information for customers that start with M (Merchants).

newbalanceDest - new balance recipient after the transaction. Note that there is not information for customers that start with M (Merchants).

isFraud - This is the transactions made by the fraudulent agents inside the simulation. In this specific dataset the fraudulent behavior of the agents aims to profit by taking control or customers accounts and try to empty the funds by transferring to another account and then cashing out of the system.

isFlaggedFraud - The business model aims to control massive transfers from one account to another and flags illegal attempts. An illegal attempt in this dataset is an attempt to transfer more than 200.000 in a single transaction.

Past Research

There are 5 similar files that contain the run of 5 different scenarios. These files are better explained at my PhD thesis chapter 7 (PhD Thesis Available here http://urn.kb.se/resolve?urn=urn:nbn:se:bth-12932.

We ran PaySim several times using random seeds for 744 steps, representing each hour of one month of real time, which matches the original logs. Each run took around 45 minutes on an i7 intel processor with 16GB of RAM. The final result of a run contains approximately 24 million of financial records divided into the 5 types of categories: CASH-IN, CASH-OUT, DEBIT, PAYMENT and TRANSFER.

Acknowledgements

This work is part of the research project ”Scalable resource-efficient systems for big data analytics” funded by the Knowledge Foundation (grant: 20140032) in Sweden.

Please refer to this dataset using the following citations:

PaySim first paper of the simulator:

E. A. Lopez-Rojas , A. Elmir, and S. Axelsson. "PaySim: A financial mobile money simulator for fraud detection". In: The 28th European Modeling and Simulation Symposium-EMSS, Larnaca, Cyprus. 2016

The Test Data Generation Tools market is poised for significant expansion, projected to reach an estimated USD 1.5 billion in 2025 and exhibit a robust Compound Annual Growth Rate (CAGR) of approximately 15% through 2033. This growth is primarily fueled by the escalating complexity of software applications, the increasing demand for agile development methodologies, and the critical need for comprehensive and realistic test data to ensure application quality and performance. Enterprises across all sizes, from large corporations to Small and Medium-sized Enterprises (SMEs), are recognizing the indispensable role of effective test data management in mitigating risks, accelerating time-to-market, and enhancing user experience. The drive for cost optimization and regulatory compliance further propels the adoption of advanced test data generation solutions, as manual data creation is often time-consuming, error-prone, and unsustainable in today's fast-paced development cycles. The market is witnessing a paradigm shift towards intelligent and automated data generation, moving beyond basic random or pathwise techniques to more sophisticated goal-oriented and AI-driven approaches that can generate highly relevant and production-like data. The market landscape is characterized by a dynamic interplay of established technology giants and specialized players, all vying for market share by offering innovative features and tailored solutions. Prominent companies like IBM, Informatica, Microsoft, and Broadcom are leveraging their extensive portfolios and cloud infrastructure to provide integrated data management and testing solutions. Simultaneously, specialized vendors such as DATPROF, Delphix Corporation, and Solix Technologies are carving out niches by focusing on advanced synthetic data generation, data masking, and data subsetting capabilities. The evolution of cloud-native architectures and microservices has created a new set of challenges and opportunities, with a growing emphasis on generating diverse and high-volume test data for distributed systems. Asia Pacific, particularly China and India, is emerging as a significant growth region due to the burgeoning IT sector and increasing investments in digital transformation initiatives. North America and Europe continue to be mature markets, driven by strong R&D investments and a high level of digital adoption. The market's trajectory indicates a sustained upward trend, driven by the continuous pursuit of software excellence and the critical need for robust testing strategies. This report provides an in-depth analysis of the global Test Data Generation Tools market, examining its evolution, current landscape, and future trajectory from 2019 to 2033. The Base Year for analysis is 2025, with the Estimated Year also being 2025, and the Forecast Period extending from 2025 to 2033. The Historical Period covered is 2019-2024. We delve into the critical aspects of this rapidly growing industry, offering insights into market dynamics, key players, emerging trends, and growth opportunities. The market is projected to witness substantial growth, with an estimated value reaching several million by the end of the forecast period.

Discover the booming Test Data Generation Tools market! This in-depth analysis reveals key trends, growth drivers, and leading companies shaping this dynamic sector. Explore market size projections, regional breakdowns, and future opportunities for 2025-2033.

CONTEXT

========================================

• [ Paper describing generation of data and uses of it -- Also to appear at Neurips'2023 ]
• [ Github Site with GNN Models to Predict Laundering ]
• [ Provably Powerful Graph Neural Networks for Directed Multigraphs ] Paper with more detailed description of the GNN models.
• If use the datasets and code and publish papers, we would appreciate if you cite these works.

========================================

Money laundering is a multi-billion dollar issue. Detection of laundering is very difficult. Most automated algorithms have a high false positive rate: legitimate transactions incorrectly flagged as laundering. The converse is also a major problem -- false negatives, i.e. undetected laundering transactions. Naturally, criminals work hard to cover their tracks.

Access to real financial transaction data is highly restricted -- for both proprietary and privacy reasons. Even when access is possible, it is problematic to provide a correct tag (laundering or legitimate) to each transaction -- as noted above. This synthetic transaction data from IBM avoids these problems.

The data provided here is based on a virtual world inhabited by individuals, companies, and banks. Individuals interact with other individuals and companies. Likewise, companies interact with other companies and with individuals. These interactions can take many forms, e.g. purchase of consumer goods and services, purchase orders for industrial supplies, payment of salaries, repayment of loans, and more. These financial transactions are generally conducted via banks, i.e. the payer and receiver both have accounts, with accounts taking multiple forms from checking to credit cards to bitcoin.

Some (small) fraction of the individuals and companies in the generator model engage in criminal behavior -- such as smuggling, illegal gambling, extortion, and more. Criminals obtain funds from these illicit activities, and then try to hide the source of these illicit funds via a series of financial transactions. Such financial transactions to hide illicit funds constitute laundering. Thus, the data available here is labelled and can be used for training and testing AML (Anti Money Laundering) models and for other purposes.

The data generator that created the data here not only models illicit activity, but also tracks funds derived from illicit activity through arbitrarily many transactions -- thus creating the ability to label laundering transactions many steps removed from their illicit source. With this foundation, it is straightforward for the generator to label individual transactions as laundering or legitimate.

Note that this IBM generator models the entire money laundering cycle: - Placement: Sources like smuggling of illicit funds. - Layering: Mixing the illicit funds into the financial system. - Integration: Spending the illicit funds.

As another capability possible only with synthetic data, note that a real bank or other institution typically has access to only a portion of the transactions involved in laundering: the transactions involving that bank. Transactions happening at other banks or between other banks are not seen. Thus, models built on real transactions from one institution can have only a limited view of the world.

By contrast these synthetic transactions contain an entire financial ecosystem. Thus it may be possible to create laundering detection models that undertand the broad sweep of transactions across institutions, but apply those models to make inferences only about transactions at a particular bank.

As another point of reference, IBM previously released data from a very early version of this data generator: https://ibm.box.com/v/AML-Anti-Money-Laundering-Data

The generator has been made significantly more robust since that previous data was released, and these transactions reflect improved realism, bug fixes, and other improvements compared to the previous release.

Credit card transaction data labeled for fraud and built using a related generator is also available on Kaggle: https://www.kaggle.com/datasets/ealtman2019/credit-card-transactions

CONTENT

We release 6 datasets here divided into two groups of three: - Group HI has a relatively higher illicit ratio (more laundering). - Group LI has a relatively lower illicit ratio (less laundering).

Both HI and LI internally have three sets of data: small, medium, and large. The goal is to support a broad degree of modeling and computational resources. All of these datasets are independent, e.g. the small datasets are not ...

T10I4D100K is a renowned synthetic database generated using the IBM Quest generator. This database is widely used to evaluate various frequent and correlated pattern mining algorithms.

StructText — SEC_WikiDB & SEC_WikiDB_subset

Dataset card for the VLDB 2025 TaDA-workshop submission “StructText: A Synthetic Table-to-Text Approach for Benchmark Generation with Multi-Dimensional Evaluation” (under review). from datasets import load_dataset

default = SEC_WikiDB_unfiltered_all

ds = load_dataset( "ibm-research/struct-text", trust_remote_code=True)

a specific configuration

subset = load_dataset( "ibm-research/struct-text"… See the full description on the dataset page: https://huggingface.co/datasets/ibm-research/struct-text.

The Test Data Management (TDM) market is poised for substantial growth, projected to reach a market size of $912 million by 2025, expanding at a robust Compound Annual Growth Rate (CAGR) of 10.4% through 2033. This significant upward trajectory is driven by an increasing demand for agile and efficient software development lifecycles, coupled with the growing complexity of data across industries. Organizations are increasingly recognizing the critical role of high-quality, relevant, and secure test data in ensuring the reliability, performance, and security of their applications. The widespread adoption of DevOps practices, continuous integration/continuous deployment (CI/CD) pipelines, and the rise of data-intensive applications in sectors like BFSI, Healthcare, and IT are primary accelerators for TDM solutions. Furthermore, stringent data privacy regulations such as GDPR and CCPA are compelling businesses to invest in TDM to anonymize and mask sensitive data, thus mitigating compliance risks and maintaining customer trust. The market is characterized by a shift towards cloud-based TDM solutions, offering greater scalability, flexibility, and cost-effectiveness compared to traditional on-premises deployments. The TDM market encompasses a wide array of applications, with Information Technology (IT) and Telecom sectors leading the adoption due to their rapid development cycles and extensive testing needs. BFSI and Healthcare & Life Sciences are also significant contributors, driven by regulatory compliance and the need for secure handling of sensitive patient and financial data. The "Others" segment, encompassing emerging industries and niche applications, is expected to witness considerable growth as more businesses realize the value of effective test data. Key players like Broadcom, IBM, Informatica, and Infosys are continuously innovating, offering advanced features such as synthetic data generation, data masking, subsetting, and automated data provisioning. The market's expansion is further supported by strategic partnerships and mergers & acquisitions aimed at broadening product portfolios and geographic reach. While the growth is strong, challenges such as the initial investment cost for comprehensive TDM solutions and the need for skilled personnel to manage them, alongside the inherent complexity of integrating TDM into existing workflows, represent areas that vendors are actively addressing to ensure seamless adoption and maximize market penetration. This comprehensive report provides an in-depth analysis of the Test Data Management (TDM) market, offering insights into its evolution, key drivers, challenges, and future trajectory. The study encompasses a detailed examination of the market dynamics from the Historical Period (2019-2024), establishing the Base Year (2025) for detailed analysis, and projecting growth through the Forecast Period (2025-2033), with an emphasis on the Study Period (2019-2033). The report is designed to equip stakeholders with actionable intelligence to navigate this rapidly evolving landscape. The global Test Data Management market is projected to reach a valuation in the millions of US dollars, indicating significant economic activity and investment within this domain.

Se prevé que el tamaño del mercado global de generación de datos sintéticos aumente de USD 376,02 millones en 2025 a USD 7120 millones en 2035, lo que refleja una tasa de crecimiento anual compuesta (TCAC) superior al 34,2 %. Los principales actores del sector son Nvidia, Microsoft, IBM, Databricks y Mostly AI, que lideran la innovación y establecen los estándares del sector.