Test Data Generation Tools Market Outlook

The global market size for Test Data Generation Tools was valued at USD 800 million in 2023 and is projected to reach USD 2.2 billion by 2032, growing at a CAGR of 12.1% during the forecast period. The surge in the adoption of agile and DevOps practices, along with the increasing complexity of software applications, is driving the growth of this market.

One of the primary growth factors for the Test Data Generation Tools market is the increasing need for high-quality test data in software development. As businesses shift towards more agile and DevOps methodologies, the demand for automated and efficient test data generation solutions has surged. These tools help in reducing the time required for test data creation, thereby accelerating the overall software development lifecycle. Additionally, the rise in digital transformation across various industries has necessitated the need for robust testing frameworks, further propelling the market growth.

The proliferation of big data and the growing emphasis on data privacy and security are also significant contributors to market expansion. With the introduction of stringent regulations like GDPR and CCPA, organizations are compelled to ensure that their test data is compliant with these laws. Test Data Generation Tools that offer features like data masking and data subsetting are increasingly being adopted to address these compliance requirements. Furthermore, the increasing instances of data breaches have underscored the importance of using synthetic data for testing purposes, thereby driving the demand for these tools.

Another critical growth factor is the technological advancements in artificial intelligence and machine learning. These technologies have revolutionized the field of test data generation by enabling the creation of more realistic and comprehensive test data sets. Machine learning algorithms can analyze large datasets to generate synthetic data that closely mimics real-world data, thus enhancing the effectiveness of software testing. This aspect has made AI and ML-powered test data generation tools highly sought after in the market.

Regional outlook for the Test Data Generation Tools market shows promising growth across various regions. North America is expected to hold the largest market share due to the early adoption of advanced technologies and the presence of major software companies. Europe is also anticipated to witness significant growth owing to strict regulatory requirements and increased focus on data security. The Asia Pacific region is projected to grow at the highest CAGR, driven by rapid industrialization and the growing IT sector in countries like India and China.

Synthetic Data Generation has emerged as a pivotal component in the realm of test data generation tools. This process involves creating artificial data that closely resembles real-world data, without compromising on privacy or security. The ability to generate synthetic data is particularly beneficial in scenarios where access to real data is restricted due to privacy concerns or regulatory constraints. By leveraging synthetic data, organizations can perform comprehensive testing without the risk of exposing sensitive information. This not only ensures compliance with data protection regulations but also enhances the overall quality and reliability of software applications. As the demand for privacy-compliant testing solutions grows, synthetic data generation is becoming an indispensable tool in the software development lifecycle.

Component Analysis

The Test Data Generation Tools market is segmented into software and services. The software segment is expected to dominate the market throughout the forecast period. This dominance can be attributed to the increasing adoption of automated testing tools and the growing need for robust test data management solutions. Software tools offer a wide range of functionalities, including data profiling, data masking, and data subsetting, which are essential for effective software testing. The continuous advancements in software capabilities also contribute to the growth of this segment.

In contrast, the services segment, although smaller in market share, is expected to grow at a substantial rate. Services include consulting, implementation, and support services, which are crucial for the successful deployment and management of test data generation tools. The increasing complexity of IT inf

Dataset used in the article entitled 'Synthetic Datasets Generator for Testing Information Visualization and Machine Learning Techniques and Tools'. These datasets can be used to test several characteristics in machine learning and data processing algorithms.

The Test Data Generation Tools market is experiencing robust growth, driven by the increasing demand for efficient and reliable software testing in a rapidly evolving digital landscape. The market's expansion is fueled by several key factors: the escalating complexity of software applications, the growing adoption of agile and DevOps methodologies which necessitate faster test cycles, and the rising need for high-quality software releases to meet stringent customer expectations. Organizations across various sectors, including finance, healthcare, and technology, are increasingly adopting test data generation tools to automate the creation of realistic and representative test data, thereby reducing testing time and costs while enhancing the overall quality of software products. This shift is particularly evident in the adoption of cloud-based solutions, offering scalability and accessibility benefits. The competitive landscape is marked by a mix of established players like IBM and Microsoft, alongside specialized vendors like Broadcom and Informatica, and emerging innovative startups. The market is witnessing increased mergers and acquisitions as larger players seek to expand their market share and product portfolios. Future growth will be influenced by advancements in artificial intelligence (AI) and machine learning (ML), enabling the generation of even more realistic and sophisticated test data, further accelerating market expansion. The market's projected Compound Annual Growth Rate (CAGR) suggests a substantial increase in market value over the forecast period (2025-2033). While precise figures were not provided, a reasonable estimation based on current market trends indicates a significant expansion. Market segmentation will likely see continued growth across various sectors, with cloud-based solutions gaining traction. Geographic expansion will also contribute to overall growth, particularly in regions with rapidly developing software industries. However, challenges remain, such as the need for skilled professionals to manage and utilize these tools effectively and the potential security concerns related to managing large datasets. Addressing these challenges will be crucial for sustained market growth and wider adoption. The overall outlook for the Test Data Generation Tools market remains positive, driven by the persistent need for efficient and robust software testing processes in a continuously evolving technological environment.

Test Data Management Market size was valued at USD 1.54 Billion in 2024 and is projected to reach USD 2.97 Billion by 2032, growing at a CAGR of 11.19% from 2026 to 2032.

Test Data Management Market Drivers

Increasing Data Volumes: The exponential growth in data generated by businesses necessitates efficient management of test data. Effective TDM solutions help organizations handle large volumes of data, ensuring accurate and reliable testing processes.

Need for Regulatory Compliance: Stringent data privacy regulations, such as GDPR, HIPAA, and CCPA, require organizations to protect sensitive data. TDM solutions help ensure compliance by masking or anonymizing sensitive data used in testing environments.

Synthetic Data Generation Market size was valued at USD 0.4 Billion in 2024 and is projected to reach USD 9.3 Billion by 2032, growing at a CAGR of 46.5 % from 2026 to 2032.

The Synthetic Data Generation Market is driven by the rising demand for AI and machine learning, where high-quality, privacy-compliant data is crucial for model training. Businesses seek synthetic data to overcome real-data limitations, ensuring security, diversity, and scalability without regulatory concerns. Industries like healthcare, finance, and autonomous vehicles increasingly adopt synthetic data to enhance AI accuracy while complying with stringent privacy laws.

Additionally, cost efficiency and faster data availability fuel market growth, reducing dependency on expensive, time-consuming real-world data collection. Advancements in generative AI, deep learning, and simulation technologies further accelerate adoption, enabling realistic synthetic datasets for robust AI model development.

AI-Generated Test Data Market Outlook

According to our latest research, the global AI-Generated Test Data market size reached USD 1.24 billion in 2024, with a robust year-on-year growth rate. The market is poised to expand at a CAGR of 32.8% from 2025 to 2033, driven by the increasing demand for automated software quality assurance and the rapid adoption of AI-powered solutions across industries. By 2033, the AI-Generated Test Data market is forecasted to reach USD 16.62 billion, reflecting its critical role in modern software development and digital transformation initiatives worldwide.

One of the primary growth factors fueling the AI-Generated Test Data market is the escalating complexity of software systems, which necessitates more advanced, scalable, and realistic test data generation. Traditional manual and rule-based test data creation methods are increasingly inadequate in meeting the dynamic requirements of continuous integration and deployment pipelines. AI-driven test data solutions offer unparalleled efficiency by automating the generation of diverse, high-quality test datasets that closely mimic real-world scenarios. This not only accelerates the software development lifecycle but also significantly improves the accuracy and reliability of testing outcomes, thereby reducing the risk of defects in production environments.

Another significant driver is the growing emphasis on data privacy and compliance with global regulations such as GDPR, HIPAA, and CCPA. Organizations are under immense pressure to ensure that sensitive customer data is not exposed during software testing. AI-Generated Test Data tools address this challenge by creating synthetic datasets that preserve statistical fidelity without compromising privacy. This approach enables organizations to conduct robust testing while adhering to stringent data protection standards, thus fostering trust among stakeholders and regulators. The increasing adoption of these tools in regulated industries such as banking, healthcare, and telecommunications is a testament to their value proposition.

The surge in machine learning and artificial intelligence applications across various industries is also contributing to the expansion of the AI-Generated Test Data market. High-quality, representative data is the cornerstone of effective AI model training and validation. AI-powered test data generation platforms can synthesize complex datasets tailored to specific use cases, enhancing the performance and generalizability of machine learning models. As enterprises invest heavily in AI-driven innovation, the demand for sophisticated test data generation capabilities is expected to grow exponentially, further propelling market growth.

Regionally, North America continues to dominate the AI-Generated Test Data market, accounting for the largest share in 2024, followed closely by Europe and Asia Pacific. The presence of major technology companies, advanced IT infrastructure, and a strong focus on software quality assurance are key factors supporting market leadership in these regions. Asia Pacific, in particular, is witnessing the fastest growth, driven by rapid digitalization, expanding IT and telecom sectors, and increasing investments in AI research and development. The regional landscape is expected to evolve rapidly over the forecast period, with emerging economies playing a pivotal role in market expansion.

Component Analysis

The Component segment of the AI-Generated Test Data market is bifurcated into Software and Services, each playing a distinct yet complementary role in the ecosystem. Software solutions constitute the backbone of the market, providing the core functionalities required for automated test data generation, management, and integration with existing DevOps pipelines. These platforms leverage advanced AI algorithms to analyze application requirements, generate synthetic datasets, and support a wide range of testing scenarios, from functional and regression testing to performance and security assessments. The continuous evolution of software platforms, with features such as self-learning, adaptive data generation, and seamless integration with popular development tools, is driving their adoption across enterprises of all sizes.

Services, on the other hand, encompass a broad spectrum of offerings, including consulting, implementation, training, and support. As organizations emb

The global test data management market size was worth around USD 1.50 billion in 2023 and is predicted to grow to around USD 3.87 billion by 2032

Overview of Data

The site includes data only for the two subjects: Ceu-pacific and JBilling. For both the subjects, the “.model” shows the model created from the business rules obtained from respective websites, and “_HighLevelTests.csv” shows the tests generated. Among csv files, we show tests generated by both BUSTER and Exhaust as well.

Paper Abstract

Test cases that drive an application under test via its graphical user interface (GUI) consist of sequences of steps that perform actions on, or verify the state of, the application user interface. Such tests can be hard to maintain, especially if they are not properly modularized—that is, common steps occur in many test cases, which can make test maintenance cumbersome and expensive. Performing modularization manually can take up considerable human effort. To address this, we present an automated approach for modularizing GUI test cases. Our approach consists of multiple phases. In the first phase, it analyzes individual test cases to partition test steps into candidate subroutines, based on how user-interface elements are accessed in the steps. This phase can analyze the test cases only or also leverage execution traces of the tests, which involves a cost-accuracy tradeoff. In the second phase, the technique compares candidate subroutines across test cases, and refines them to compute the final set of subroutines. In the last phase, it creates callable subroutines, with parameterized data and control flow, and refactors the original tests to call the subroutines with context-specific data and control parameters. Our empirical results, collected using open-source applications, illustrate the effectiveness of the approach.

Dataset Card for test-data-generator

This dataset has been created with distilabel.

  Dataset Summary

This dataset contains a pipeline.yaml which can be used to reproduce the pipeline that generated it in distilabel using the distilabel CLI: distilabel pipeline run --config "https://huggingface.co/datasets/franciscoflorencio/test-data-generator/raw/main/pipeline.yaml"

or explore the configuration: distilabel pipeline info --config… See the full description on the dataset page: https://huggingface.co/datasets/franciscoflorencio/test-data-generator.

AI-Generated Test Data Market Outlook

According to our latest research, the global AI-Generated Test Data market size reached USD 1.12 billion in 2024, driven by the rapid adoption of artificial intelligence across software development and testing environments. The market is exhibiting a robust growth trajectory, registering a CAGR of 28.6% from 2025 to 2033. By 2033, the market is forecasted to achieve a value of USD 10.23 billion, reflecting the increasing reliance on AI-driven solutions for efficient, scalable, and accurate test data generation. This growth is primarily fueled by the rising complexity of software systems, stringent compliance requirements, and the need for enhanced data privacy across industries.

One of the primary growth factors for the AI-Generated Test Data market is the escalating demand for automation in software development lifecycles. As organizations strive to accelerate release cycles and improve software quality, traditional manual test data generation methods are proving inadequate. AI-generated test data solutions offer a compelling alternative by enabling rapid, scalable, and highly accurate data creation, which not only reduces time-to-market but also minimizes human error. This automation is particularly crucial in DevOps and Agile environments, where continuous integration and delivery necessitate fast and reliable testing processes. The ability of AI-driven tools to mimic real-world data scenarios and generate vast datasets on demand is revolutionizing the way enterprises approach software testing and quality assurance.

Another significant driver is the growing emphasis on data privacy and regulatory compliance, especially in sectors such as BFSI, healthcare, and government. With regulations like GDPR, HIPAA, and CCPA imposing strict controls on the use and sharing of real customer data, organizations are increasingly turning to AI-generated synthetic data for testing purposes. This not only ensures compliance but also protects sensitive information from potential breaches during the software development and testing phases. AI-generated test data tools can create anonymized yet realistic datasets that closely replicate production data, allowing organizations to rigorously test their systems without exposing confidential information. This capability is becoming a critical differentiator for vendors in the AI-generated test data market.

The proliferation of complex, data-intensive applications across industries further amplifies the need for sophisticated test data generation solutions. Sectors such as IT and telecommunications, retail and e-commerce, and manufacturing are witnessing a surge in digital transformation initiatives, resulting in intricate software architectures and interconnected systems. AI-generated test data solutions are uniquely positioned to address the challenges posed by these environments, enabling organizations to simulate diverse scenarios, validate system performance, and identify vulnerabilities with unprecedented accuracy. As digital ecosystems continue to evolve, the demand for advanced AI-powered test data generation tools is expected to rise exponentially, driving sustained market growth.

From a regional perspective, North America currently leads the AI-Generated Test Data market, accounting for the largest share in 2024, followed closely by Europe and Asia Pacific. The dominance of North America can be attributed to the high concentration of technology giants, early adoption of AI technologies, and a mature regulatory landscape. Meanwhile, Asia Pacific is emerging as a high-growth region, propelled by rapid digitalization, expanding IT infrastructure, and increasing investments in AI research and development. Europe maintains a steady growth trajectory, bolstered by stringent data privacy regulations and a strong focus on innovation. As global enterprises continue to invest in digital transformation, the regional dynamics of the AI-generated test data market are expected to evolve, with significant opportunities emerging across developing economies.

Componen

The Synthetic Data Generation Market size is expected to reach a valuation of USD 36.09 Billion in 2033 growing at a CAGR of 39.45%. The research report classifies market by share, trend, demand and based on segmentation by Data Type, Modeling Type, Offering, Application, End Use and Regional Outloo...

The appendix of our ICSE 2018 paper "Search-Based Test Data Generation for SQL Queries: Appendix".

The appendix contains:

• The queries from the three open source systems we used in the evaluation of our tool (the industry software system is not part of this appendix, due to privacy reasons)
• The results of our evaluation.
• The source code of the tool. Most recent version can be found at https://github.com/SERG-Delft/evosql.
• The results of the tuning procedure we conducted before running the final evaluation.

Synthetic Data Generation Market Outlook

According to our latest research, the global synthetic data generation market size reached USD 1.6 billion in 2024, demonstrating robust expansion driven by increasing demand for high-quality, privacy-preserving datasets. The market is projected to grow at a CAGR of 38.2% over the forecast period, reaching USD 19.2 billion by 2033. This remarkable growth trajectory is fueled by the growing adoption of artificial intelligence (AI) and machine learning (ML) technologies across industries, coupled with stringent data privacy regulations that necessitate innovative data solutions. As per our latest research, organizations worldwide are increasingly leveraging synthetic data to address data scarcity, enhance AI model training, and ensure compliance with evolving privacy standards.

One of the primary growth factors for the synthetic data generation market is the rising emphasis on data privacy and regulatory compliance. With the implementation of stringent data protection laws such as the General Data Protection Regulation (GDPR) in Europe and the California Consumer Privacy Act (CCPA) in the United States, enterprises are under immense pressure to safeguard sensitive information. Synthetic data offers a compelling solution by enabling organizations to generate artificial datasets that mirror the statistical properties of real data without exposing personally identifiable information. This not only facilitates regulatory compliance but also empowers organizations to innovate without the risk of data breaches or privacy violations. As businesses increasingly recognize the value of privacy-preserving data, the demand for advanced synthetic data generation solutions is set to surge.

Another significant driver is the exponential growth in AI and ML adoption across various sectors, including healthcare, finance, automotive, and retail. High-quality, diverse, and unbiased data is the cornerstone of effective AI model development. However, acquiring such data is often challenging due to privacy concerns, limited availability, or high acquisition costs. Synthetic data generation bridges this gap by providing scalable, customizable datasets tailored to specific use cases, thereby accelerating AI training and reducing dependency on real-world data. Organizations are leveraging synthetic data to enhance algorithm performance, mitigate data bias, and simulate rare events, which are otherwise difficult to capture in real datasets. This capability is particularly valuable in sectors like autonomous vehicles, where training models on rare but critical scenarios is essential for safety and reliability.

Furthermore, the growing complexity of data types—ranging from tabular and image data to text, audio, and video—has amplified the need for versatile synthetic data generation tools. Enterprises are increasingly seeking solutions that can generate multi-modal synthetic datasets to support diverse applications such as fraud detection, product testing, and quality assurance. The flexibility offered by synthetic data generation platforms enables organizations to simulate a wide array of scenarios, test software systems, and validate AI models in controlled environments. This not only enhances operational efficiency but also drives innovation by enabling rapid prototyping and experimentation. As the digital ecosystem continues to evolve, the ability to generate synthetic data across various formats will be a critical differentiator for businesses striving to maintain a competitive edge.

Regionally, North America leads the synthetic data generation market, accounting for the largest revenue share in 2024, followed closely by Europe and Asia Pacific. The dominance of North America can be attributed to the strong presence of technology giants, advanced research institutions, and a favorable regulatory environment that encourages AI innovation. Europe is witnessing rapid growth due to proactive data privacy regulations and increasing investments in digital transformation initiatives. Meanwhile, Asia Pacific is emerging as a high-growth region, driven by the proliferation of digital technologies and rising adoption of AI-powered solutions across industries. Latin America and the Middle East & Africa are also expected to experience steady growth, supported by government-led digitalization programs and expanding IT infrastructure.

Comparison of results between NSA-GA, NSA, and random testing on all programs using integer data type and different range.

North America Synthetic Data Generation Market is expected to grow during 2025-2031

Data Description

We release the synthetic data generated using the method described in the paper Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (ACL 2024 Findings). The external knowledge we use is based on LLM-generated topics and writing styles.

  Generated Datasets

The original train/validation/test data, and the generated synthetic training data are listed as follows. For each dataset, we generate 5000… See the full description on the dataset page: https://huggingface.co/datasets/ritaranx/clinical-synthetic-text-llm.

Introduction

This repository hosts the Testing Roads for Autonomous VEhicLes (TRAVEL) dataset. TRAVEL is an extensive collection of virtual roads that have been used for testing lane assist/keeping systems (i.e., driving agents) and data from their execution in state of the art, physically accurate driving simulator, called BeamNG.tech. Virtual roads consist of sequences of road points interpolated using Cubic splines.

Along with the data, this repository contains instructions on how to install the tooling necessary to generate new data (i.e., test cases) and analyze them in the context of test regression. We focus on test selection and test prioritization, given their importance for developing high-quality software following the DevOps paradigms.

This dataset builds on top of our previous work in this area, including work on

test generation (e.g., AsFault, DeepJanus, and DeepHyperion) and the SBST CPS tool competition (SBST2021),

test selection: SDC-Scissor and related tool

test prioritization: automated test cases prioritization work for SDCs.

Dataset Overview

The TRAVEL dataset is available under the data folder and is organized as a set of experiments folders. Each of these folders is generated by running the test-generator (see below) and contains the configuration used for generating the data (experiment_description.csv), various statistics on generated tests (generation_stats.csv) and found faults (oob_stats.csv). Additionally, the folders contain the raw test cases generated and executed during each experiment (test..json).

The following sections describe what each of those files contains.

Experiment Description

The experiment_description.csv contains the settings used to generate the data, including:

Time budget. The overall generation budget in hours. This budget includes both the time to generate and execute the tests as driving simulations.

The size of the map. The size of the squared map defines the boundaries inside which the virtual roads develop in meters.

The test subject. The driving agent that implements the lane-keeping system under test. The TRAVEL dataset contains data generated testing the BeamNG.AI and the end-to-end Dave2 systems.

The test generator. The algorithm that generated the test cases. The TRAVEL dataset contains data obtained using various algorithms, ranging from naive and advanced random generators to complex evolutionary algorithms, for generating tests.

The speed limit. The maximum speed at which the driving agent under test can travel.

Out of Bound (OOB) tolerance. The test cases' oracle that defines the tolerable amount of the ego-car that can lie outside the lane boundaries. This parameter ranges between 0.0 and 1.0. In the former case, a test failure triggers as soon as any part of the ego-vehicle goes out of the lane boundary; in the latter case, a test failure triggers only if the entire body of the ego-car falls outside the lane.

Experiment Statistics

The generation_stats.csv contains statistics about the test generation, including:

Total number of generated tests. The number of tests generated during an experiment. This number is broken down into the number of valid tests and invalid tests. Valid tests contain virtual roads that do not self-intersect and contain turns that are not too sharp.

Test outcome. The test outcome contains the number of passed tests, failed tests, and test in error. Passed and failed tests are defined by the OOB Tolerance and an additional (implicit) oracle that checks whether the ego-car is moving or standing. Tests that did not pass because of other errors (e.g., the simulator crashed) are reported in a separated category.

The TRAVEL dataset also contains statistics about the failed tests, including the overall number of failed tests (total oob) and its breakdown into OOB that happened while driving left or right. Further statistics about the diversity (i.e., sparseness) of the failures are also reported.

Test Cases and Executions

Each test..json contains information about a test case and, if the test case is valid, the data observed during its execution as driving simulation.

The data about the test case definition include:

The road points. The list of points in a 2D space that identifies the center of the virtual road, and their interpolation using cubic splines (interpolated_points)

The test ID. The unique identifier of the test in the experiment.

Validity flag and explanation. A flag that indicates whether the test is valid or not, and a brief message describing why the test is not considered valid (e.g., the road contains sharp turns or the road self intersects)

The test data are organized according to the following JSON Schema and can be interpreted as RoadTest objects provided by the tests_generation.py module.

{ "type": "object", "properties": { "id": { "type": "integer" }, "is_valid": { "type": "boolean" }, "validation_message": { "type": "string" }, "road_points": { §\label{line:road-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "interpolated_points": { §\label{line:interpolated-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "test_outcome": { "type": "string" }, §\label{line:test-outcome}§ "description": { "type": "string" }, "execution_data": { "type": "array", "items": { "$ref" : "schemas/simulationdata" } } }, "required": [ "id", "is_valid", "validation_message", "road_points", "interpolated_points" ] }

Finally, the execution data contain a list of timestamped state information recorded by the driving simulation. State information is collected at constant frequency and includes absolute position, rotation, and velocity of the ego-car, its speed in Km/h, and control inputs from the driving agent (steering, throttle, and braking). Additionally, execution data contain OOB-related data, such as the lateral distance between the car and the lane center and the OOB percentage (i.e., how much the car is outside the lane).

The simulation data adhere to the following (simplified) JSON Schema and can be interpreted as Python objects using the simulation_data.py module.

{ "$id": "schemas/simulationdata", "type": "object", "properties": { "timer" : { "type": "number" }, "pos" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel_kmh" : { "type": "number" }, "steering" : { "type": "number" }, "brake" : { "type": "number" }, "throttle" : { "type": "number" }, "is_oob" : { "type": "number" }, "oob_percentage" : { "type": "number" } §\label{line:oob-percentage}§ }, "required": [ "timer", "pos", "vel", "vel_kmh", "steering", "brake", "throttle", "is_oob", "oob_percentage" ] }

Dataset Content

The TRAVEL dataset is a lively initiative so the content of the dataset is subject to change. Currently, the dataset contains the data collected during the SBST CPS tool competition, and data collected in the context of our recent work on test selection (SDC-Scissor work and tool) and test prioritization (automated test cases prioritization work for SDCs).

SBST CPS Tool Competition Data

The data collected during the SBST CPS tool competition are stored inside data/competition.tar.gz. The file contains the test cases generated by Deeper, Frenetic, AdaFrenetic, and Swat, the open-source test generators submitted to the competition and executed against BeamNG.AI with an aggression factor of 0.7 (i.e., conservative driver).

    Name
    Map Size (m x m)
    Max Speed (Km/h)
    Budget (h)
    OOB Tolerance (%)
    Test Subject




    DEFAULT
    200 × 200
    120
    5 (real time)
    0.95
    BeamNG.AI - 0.7


    SBST
    200 × 200
    70
    2 (real time)
    0.5
    BeamNG.AI - 0.7

Specifically, the TRAVEL dataset contains 8 repetitions for each of the above configurations for each test generator totaling 64 experiments.

SDC Scissor

With SDC-Scissor we collected data based on the Frenetic test generator. The data is stored inside data/sdc-scissor.tar.gz. The following table summarizes the used parameters.

    Name
    Map Size (m x m)
    Max Speed (Km/h)
    Budget (h)
    OOB Tolerance (%)
    Test Subject




    SDC-SCISSOR
    200 × 200
    120
    16 (real time)
    0.5
    BeamNG.AI - 1.5

The dataset contains 9 experiments with the above configuration. For generating your own data with SDC-Scissor follow the instructions in its repository.

Dataset Statistics

Here is an overview of the TRAVEL dataset: generated tests, executed tests, and faults found by all the test generators grouped by experiment configuration. Some 25,845 test cases are generated by running 4 test generators 8 times in 2 configurations using the SBST CPS Tool Competition code pipeline (SBST in the table). We ran the test generators for 5 hours, allowing the ego-car a generous speed limit (120 Km/h) and defining a high OOB tolerance (i.e., 0.95), and we also ran the test generators using a smaller generation budget (i.e., 2 hours) and speed limit (i.e., 70 Km/h) while setting the OOB tolerance to a lower value (i.e., 0.85). We also collected some 5, 971 additional tests with SDC-Scissor (SDC-Scissor in the table) by running it 9 times for 16 hours using Frenetic as a test generator and defining a more realistic OOB tolerance (i.e., 0.50).

Generating new Data

Generating new data, i.e., test cases, can be done using the SBST CPS Tool Competition pipeline and the driving simulator BeamNG.tech.

Extensive instructions on how to install both software are reported inside the SBST CPS Tool Competition pipeline Documentation;

The benchmark programs description.

About

IUST-PDFCorpus is a large set of various PDF files, aimed at building and manipulating new PDF files, to test, debug, and improve the qualification of real-world PDF readers such as Adobe Acrobat Reader, Foxit Reader, Nitro Reader, MuPDF. IUST-PDFCorpus contains 6,141 PDF complete files in various sizes and contents. The corpus includes 507,299 PDF data objects and 151,132 PDF streams extracted from the set of complete files. Data objects are in the textual format while streams have a binary format and together they make PDF files. In addition, we attached the code coverage of each PDF file when it used as test data in testing MuPDF. The coverage info is available in both binary and XML formats. PDF data objects are organized into three categories. The first category contains all objects in the corpus. Each file in this category holds all PDF objects extracted from one PDF file without any preprocessing. The second category is a dataset made by merging all files in the first category with some preprocessing. The dataset is spilled into train, test and validation set which is useful for using in the machine learning tasks. The third category is the same as the second category but in a smaller size for using in the developing stage of different algorithms. IUST-PDFCorpus is collected from various sources including the Mozilla PDF.js open test corpus, some PDFs which are used in AFL as initial seed, and PDFs gathered from existing e-books, software documents, and public web in different languages. We first introduced IUST-PDFCorpus in our paper “Format-aware learn&fuzz: deep test data generation for efficient fuzzing” where we used it to build an intelligent file format fuzzer, called IUST-DeepFuzz. For the time being, we are gathering other file formats to automate testing of related applications.

Citing IUST-PDFCorpus

If IUST-PDFCorpus is used in your work in any form please cite the relevant paper: https://arxiv.org/abs/1812.09961v2

The Synthetic Data Generation Market is estimated to reach USD 6,637.9 Mn By 2034, Riding on a Strong 35.9% CAGR during forecast period.