Test Data Generation Tools Market Outlook

The global market size for Test Data Generation Tools was valued at USD 800 million in 2023 and is projected to reach USD 2.2 billion by 2032, growing at a CAGR of 12.1% during the forecast period. The surge in the adoption of agile and DevOps practices, along with the increasing complexity of software applications, is driving the growth of this market.

One of the primary growth factors for the Test Data Generation Tools market is the increasing need for high-quality test data in software development. As businesses shift towards more agile and DevOps methodologies, the demand for automated and efficient test data generation solutions has surged. These tools help in reducing the time required for test data creation, thereby accelerating the overall software development lifecycle. Additionally, the rise in digital transformation across various industries has necessitated the need for robust testing frameworks, further propelling the market growth.

The proliferation of big data and the growing emphasis on data privacy and security are also significant contributors to market expansion. With the introduction of stringent regulations like GDPR and CCPA, organizations are compelled to ensure that their test data is compliant with these laws. Test Data Generation Tools that offer features like data masking and data subsetting are increasingly being adopted to address these compliance requirements. Furthermore, the increasing instances of data breaches have underscored the importance of using synthetic data for testing purposes, thereby driving the demand for these tools.

Another critical growth factor is the technological advancements in artificial intelligence and machine learning. These technologies have revolutionized the field of test data generation by enabling the creation of more realistic and comprehensive test data sets. Machine learning algorithms can analyze large datasets to generate synthetic data that closely mimics real-world data, thus enhancing the effectiveness of software testing. This aspect has made AI and ML-powered test data generation tools highly sought after in the market.

Regional outlook for the Test Data Generation Tools market shows promising growth across various regions. North America is expected to hold the largest market share due to the early adoption of advanced technologies and the presence of major software companies. Europe is also anticipated to witness significant growth owing to strict regulatory requirements and increased focus on data security. The Asia Pacific region is projected to grow at the highest CAGR, driven by rapid industrialization and the growing IT sector in countries like India and China.

Synthetic Data Generation has emerged as a pivotal component in the realm of test data generation tools. This process involves creating artificial data that closely resembles real-world data, without compromising on privacy or security. The ability to generate synthetic data is particularly beneficial in scenarios where access to real data is restricted due to privacy concerns or regulatory constraints. By leveraging synthetic data, organizations can perform comprehensive testing without the risk of exposing sensitive information. This not only ensures compliance with data protection regulations but also enhances the overall quality and reliability of software applications. As the demand for privacy-compliant testing solutions grows, synthetic data generation is becoming an indispensable tool in the software development lifecycle.

Component Analysis

The Test Data Generation Tools market is segmented into software and services. The software segment is expected to dominate the market throughout the forecast period. This dominance can be attributed to the increasing adoption of automated testing tools and the growing need for robust test data management solutions. Software tools offer a wide range of functionalities, including data profiling, data masking, and data subsetting, which are essential for effective software testing. The continuous advancements in software capabilities also contribute to the growth of this segment.

In contrast, the services segment, although smaller in market share, is expected to grow at a substantial rate. Services include consulting, implementation, and support services, which are crucial for the successful deployment and management of test data generation tools. The increasing complexity of IT inf

AI-Generated Test Data Market Outlook

According to our latest research, the global AI-Generated Test Data market size reached USD 1.24 billion in 2024, with a robust year-on-year growth rate. The market is poised to expand at a CAGR of 32.8% from 2025 to 2033, driven by the increasing demand for automated software quality assurance and the rapid adoption of AI-powered solutions across industries. By 2033, the AI-Generated Test Data market is forecasted to reach USD 16.62 billion, reflecting its critical role in modern software development and digital transformation initiatives worldwide.

One of the primary growth factors fueling the AI-Generated Test Data market is the escalating complexity of software systems, which necessitates more advanced, scalable, and realistic test data generation. Traditional manual and rule-based test data creation methods are increasingly inadequate in meeting the dynamic requirements of continuous integration and deployment pipelines. AI-driven test data solutions offer unparalleled efficiency by automating the generation of diverse, high-quality test datasets that closely mimic real-world scenarios. This not only accelerates the software development lifecycle but also significantly improves the accuracy and reliability of testing outcomes, thereby reducing the risk of defects in production environments.

Another significant driver is the growing emphasis on data privacy and compliance with global regulations such as GDPR, HIPAA, and CCPA. Organizations are under immense pressure to ensure that sensitive customer data is not exposed during software testing. AI-Generated Test Data tools address this challenge by creating synthetic datasets that preserve statistical fidelity without compromising privacy. This approach enables organizations to conduct robust testing while adhering to stringent data protection standards, thus fostering trust among stakeholders and regulators. The increasing adoption of these tools in regulated industries such as banking, healthcare, and telecommunications is a testament to their value proposition.

The surge in machine learning and artificial intelligence applications across various industries is also contributing to the expansion of the AI-Generated Test Data market. High-quality, representative data is the cornerstone of effective AI model training and validation. AI-powered test data generation platforms can synthesize complex datasets tailored to specific use cases, enhancing the performance and generalizability of machine learning models. As enterprises invest heavily in AI-driven innovation, the demand for sophisticated test data generation capabilities is expected to grow exponentially, further propelling market growth.

Regionally, North America continues to dominate the AI-Generated Test Data market, accounting for the largest share in 2024, followed closely by Europe and Asia Pacific. The presence of major technology companies, advanced IT infrastructure, and a strong focus on software quality assurance are key factors supporting market leadership in these regions. Asia Pacific, in particular, is witnessing the fastest growth, driven by rapid digitalization, expanding IT and telecom sectors, and increasing investments in AI research and development. The regional landscape is expected to evolve rapidly over the forecast period, with emerging economies playing a pivotal role in market expansion.

Component Analysis

The Component segment of the AI-Generated Test Data market is bifurcated into Software and Services, each playing a distinct yet complementary role in the ecosystem. Software solutions constitute the backbone of the market, providing the core functionalities required for automated test data generation, management, and integration with existing DevOps pipelines. These platforms leverage advanced AI algorithms to analyze application requirements, generate synthetic datasets, and support a wide range of testing scenarios, from functional and regression testing to performance and security assessments. The continuous evolution of software platforms, with features such as self-learning, adaptive data generation, and seamless integration with popular development tools, is driving their adoption across enterprises of all sizes.

Services, on the other hand, encompass a broad spectrum of offerings, including consulting, implementation, training, and support. As organizations emb

AI-Generated Test Data Market Outlook

According to our latest research, the global AI-Generated Test Data market size reached USD 1.12 billion in 2024, driven by the rapid adoption of artificial intelligence across software development and testing environments. The market is exhibiting a robust growth trajectory, registering a CAGR of 28.6% from 2025 to 2033. By 2033, the market is forecasted to achieve a value of USD 10.23 billion, reflecting the increasing reliance on AI-driven solutions for efficient, scalable, and accurate test data generation. This growth is primarily fueled by the rising complexity of software systems, stringent compliance requirements, and the need for enhanced data privacy across industries.

One of the primary growth factors for the AI-Generated Test Data market is the escalating demand for automation in software development lifecycles. As organizations strive to accelerate release cycles and improve software quality, traditional manual test data generation methods are proving inadequate. AI-generated test data solutions offer a compelling alternative by enabling rapid, scalable, and highly accurate data creation, which not only reduces time-to-market but also minimizes human error. This automation is particularly crucial in DevOps and Agile environments, where continuous integration and delivery necessitate fast and reliable testing processes. The ability of AI-driven tools to mimic real-world data scenarios and generate vast datasets on demand is revolutionizing the way enterprises approach software testing and quality assurance.

Another significant driver is the growing emphasis on data privacy and regulatory compliance, especially in sectors such as BFSI, healthcare, and government. With regulations like GDPR, HIPAA, and CCPA imposing strict controls on the use and sharing of real customer data, organizations are increasingly turning to AI-generated synthetic data for testing purposes. This not only ensures compliance but also protects sensitive information from potential breaches during the software development and testing phases. AI-generated test data tools can create anonymized yet realistic datasets that closely replicate production data, allowing organizations to rigorously test their systems without exposing confidential information. This capability is becoming a critical differentiator for vendors in the AI-generated test data market.

The proliferation of complex, data-intensive applications across industries further amplifies the need for sophisticated test data generation solutions. Sectors such as IT and telecommunications, retail and e-commerce, and manufacturing are witnessing a surge in digital transformation initiatives, resulting in intricate software architectures and interconnected systems. AI-generated test data solutions are uniquely positioned to address the challenges posed by these environments, enabling organizations to simulate diverse scenarios, validate system performance, and identify vulnerabilities with unprecedented accuracy. As digital ecosystems continue to evolve, the demand for advanced AI-powered test data generation tools is expected to rise exponentially, driving sustained market growth.

From a regional perspective, North America currently leads the AI-Generated Test Data market, accounting for the largest share in 2024, followed closely by Europe and Asia Pacific. The dominance of North America can be attributed to the high concentration of technology giants, early adoption of AI technologies, and a mature regulatory landscape. Meanwhile, Asia Pacific is emerging as a high-growth region, propelled by rapid digitalization, expanding IT infrastructure, and increasing investments in AI research and development. Europe maintains a steady growth trajectory, bolstered by stringent data privacy regulations and a strong focus on innovation. As global enterprises continue to invest in digital transformation, the regional dynamics of the AI-generated test data market are expected to evolve, with significant opportunities emerging across developing economies.

Componen

Dataset used in the article entitled 'Synthetic Datasets Generator for Testing Information Visualization and Machine Learning Techniques and Tools'. These datasets can be used to test several characteristics in machine learning and data processing algorithms.

Introduction

This repository hosts the Testing Roads for Autonomous VEhicLes (TRAVEL) dataset. TRAVEL is an extensive collection of virtual roads that have been used for testing lane assist/keeping systems (i.e., driving agents) and data from their execution in state of the art, physically accurate driving simulator, called BeamNG.tech. Virtual roads consist of sequences of road points interpolated using Cubic splines.

Along with the data, this repository contains instructions on how to install the tooling necessary to generate new data (i.e., test cases) and analyze them in the context of test regression. We focus on test selection and test prioritization, given their importance for developing high-quality software following the DevOps paradigms.

This dataset builds on top of our previous work in this area, including work on

test generation (e.g., AsFault, DeepJanus, and DeepHyperion) and the SBST CPS tool competition (SBST2021),

test selection: SDC-Scissor and related tool

test prioritization: automated test cases prioritization work for SDCs.

Dataset Overview

The TRAVEL dataset is available under the data folder and is organized as a set of experiments folders. Each of these folders is generated by running the test-generator (see below) and contains the configuration used for generating the data (experiment_description.csv), various statistics on generated tests (generation_stats.csv) and found faults (oob_stats.csv). Additionally, the folders contain the raw test cases generated and executed during each experiment (test..json).

The following sections describe what each of those files contains.

Experiment Description

The experiment_description.csv contains the settings used to generate the data, including:

Time budget. The overall generation budget in hours. This budget includes both the time to generate and execute the tests as driving simulations.

The size of the map. The size of the squared map defines the boundaries inside which the virtual roads develop in meters.

The test subject. The driving agent that implements the lane-keeping system under test. The TRAVEL dataset contains data generated testing the BeamNG.AI and the end-to-end Dave2 systems.

The test generator. The algorithm that generated the test cases. The TRAVEL dataset contains data obtained using various algorithms, ranging from naive and advanced random generators to complex evolutionary algorithms, for generating tests.

The speed limit. The maximum speed at which the driving agent under test can travel.

Out of Bound (OOB) tolerance. The test cases' oracle that defines the tolerable amount of the ego-car that can lie outside the lane boundaries. This parameter ranges between 0.0 and 1.0. In the former case, a test failure triggers as soon as any part of the ego-vehicle goes out of the lane boundary; in the latter case, a test failure triggers only if the entire body of the ego-car falls outside the lane.

Experiment Statistics

The generation_stats.csv contains statistics about the test generation, including:

Total number of generated tests. The number of tests generated during an experiment. This number is broken down into the number of valid tests and invalid tests. Valid tests contain virtual roads that do not self-intersect and contain turns that are not too sharp.

Test outcome. The test outcome contains the number of passed tests, failed tests, and test in error. Passed and failed tests are defined by the OOB Tolerance and an additional (implicit) oracle that checks whether the ego-car is moving or standing. Tests that did not pass because of other errors (e.g., the simulator crashed) are reported in a separated category.

The TRAVEL dataset also contains statistics about the failed tests, including the overall number of failed tests (total oob) and its breakdown into OOB that happened while driving left or right. Further statistics about the diversity (i.e., sparseness) of the failures are also reported.

Test Cases and Executions

Each test..json contains information about a test case and, if the test case is valid, the data observed during its execution as driving simulation.

The data about the test case definition include:

The road points. The list of points in a 2D space that identifies the center of the virtual road, and their interpolation using cubic splines (interpolated_points)

The test ID. The unique identifier of the test in the experiment.

Validity flag and explanation. A flag that indicates whether the test is valid or not, and a brief message describing why the test is not considered valid (e.g., the road contains sharp turns or the road self intersects)

The test data are organized according to the following JSON Schema and can be interpreted as RoadTest objects provided by the tests_generation.py module.

{ "type": "object", "properties": { "id": { "type": "integer" }, "is_valid": { "type": "boolean" }, "validation_message": { "type": "string" }, "road_points": { §\label{line:road-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "interpolated_points": { §\label{line:interpolated-points}§ "type": "array", "items": { "$ref": "schemas/pair" }, }, "test_outcome": { "type": "string" }, §\label{line:test-outcome}§ "description": { "type": "string" }, "execution_data": { "type": "array", "items": { "$ref" : "schemas/simulationdata" } } }, "required": [ "id", "is_valid", "validation_message", "road_points", "interpolated_points" ] }

Finally, the execution data contain a list of timestamped state information recorded by the driving simulation. State information is collected at constant frequency and includes absolute position, rotation, and velocity of the ego-car, its speed in Km/h, and control inputs from the driving agent (steering, throttle, and braking). Additionally, execution data contain OOB-related data, such as the lateral distance between the car and the lane center and the OOB percentage (i.e., how much the car is outside the lane).

The simulation data adhere to the following (simplified) JSON Schema and can be interpreted as Python objects using the simulation_data.py module.

{ "$id": "schemas/simulationdata", "type": "object", "properties": { "timer" : { "type": "number" }, "pos" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel" : { "type": "array", "items":{ "$ref" : "schemas/triple" } } "vel_kmh" : { "type": "number" }, "steering" : { "type": "number" }, "brake" : { "type": "number" }, "throttle" : { "type": "number" }, "is_oob" : { "type": "number" }, "oob_percentage" : { "type": "number" } §\label{line:oob-percentage}§ }, "required": [ "timer", "pos", "vel", "vel_kmh", "steering", "brake", "throttle", "is_oob", "oob_percentage" ] }

Dataset Content

The TRAVEL dataset is a lively initiative so the content of the dataset is subject to change. Currently, the dataset contains the data collected during the SBST CPS tool competition, and data collected in the context of our recent work on test selection (SDC-Scissor work and tool) and test prioritization (automated test cases prioritization work for SDCs).

SBST CPS Tool Competition Data

The data collected during the SBST CPS tool competition are stored inside data/competition.tar.gz. The file contains the test cases generated by Deeper, Frenetic, AdaFrenetic, and Swat, the open-source test generators submitted to the competition and executed against BeamNG.AI with an aggression factor of 0.7 (i.e., conservative driver).

    Name
    Map Size (m x m)
    Max Speed (Km/h)
    Budget (h)
    OOB Tolerance (%)
    Test Subject




    DEFAULT
    200 × 200
    120
    5 (real time)
    0.95
    BeamNG.AI - 0.7


    SBST
    200 × 200
    70
    2 (real time)
    0.5
    BeamNG.AI - 0.7

Specifically, the TRAVEL dataset contains 8 repetitions for each of the above configurations for each test generator totaling 64 experiments.

SDC Scissor

With SDC-Scissor we collected data based on the Frenetic test generator. The data is stored inside data/sdc-scissor.tar.gz. The following table summarizes the used parameters.

    Name
    Map Size (m x m)
    Max Speed (Km/h)
    Budget (h)
    OOB Tolerance (%)
    Test Subject




    SDC-SCISSOR
    200 × 200
    120
    16 (real time)
    0.5
    BeamNG.AI - 1.5

The dataset contains 9 experiments with the above configuration. For generating your own data with SDC-Scissor follow the instructions in its repository.

Dataset Statistics

Here is an overview of the TRAVEL dataset: generated tests, executed tests, and faults found by all the test generators grouped by experiment configuration. Some 25,845 test cases are generated by running 4 test generators 8 times in 2 configurations using the SBST CPS Tool Competition code pipeline (SBST in the table). We ran the test generators for 5 hours, allowing the ego-car a generous speed limit (120 Km/h) and defining a high OOB tolerance (i.e., 0.95), and we also ran the test generators using a smaller generation budget (i.e., 2 hours) and speed limit (i.e., 70 Km/h) while setting the OOB tolerance to a lower value (i.e., 0.85). We also collected some 5, 971 additional tests with SDC-Scissor (SDC-Scissor in the table) by running it 9 times for 16 hours using Frenetic as a test generator and defining a more realistic OOB tolerance (i.e., 0.50).

Generating new Data

Generating new data, i.e., test cases, can be done using the SBST CPS Tool Competition pipeline and the driving simulator BeamNG.tech.

Extensive instructions on how to install both software are reported inside the SBST CPS Tool Competition pipeline Documentation;

Software testing is one of the most crucial tasks in the typical development process. Developers are usually required to write unit test cases for the code they implement. Since this is a time-consuming task, in last years many approaches and tools for automatic test case generation — such as EvoSuite — have been introduced. Nevertheless, developers have to maintain and evolve tests to sustain the changes in the source code; therefore, having readable test cases is important to ease such a process.However, it is still not clear whether developers make an effort in writing readable unit tests. Therefore, in this paper, we conduct an explorative study comparing the readability of manually written test cases with the classes they test. Moreover, we deepen such analysis looking at the readability of automatically generated test cases. Our results suggest that developers tend to neglect the readability of test cases and that automatically generated test cases are generally even less readable than manually written ones.

humidity

Overview of Data

The site includes data only for the two subjects: Ceu-pacific and JBilling. For both the subjects, the “.model” shows the model created from the business rules obtained from respective websites, and “_HighLevelTests.csv” shows the tests generated. Among csv files, we show tests generated by both BUSTER and Exhaust as well.

Paper Abstract

Test cases that drive an application under test via its graphical user interface (GUI) consist of sequences of steps that perform actions on, or verify the state of, the application user interface. Such tests can be hard to maintain, especially if they are not properly modularized—that is, common steps occur in many test cases, which can make test maintenance cumbersome and expensive. Performing modularization manually can take up considerable human effort. To address this, we present an automated approach for modularizing GUI test cases. Our approach consists of multiple phases. In the first phase, it analyzes individual test cases to partition test steps into candidate subroutines, based on how user-interface elements are accessed in the steps. This phase can analyze the test cases only or also leverage execution traces of the tests, which involves a cost-accuracy tradeoff. In the second phase, the technique compares candidate subroutines across test cases, and refines them to compute the final set of subroutines. In the last phase, it creates callable subroutines, with parameterized data and control flow, and refactors the original tests to call the subroutines with context-specific data and control parameters. Our empirical results, collected using open-source applications, illustrate the effectiveness of the approach.

This is a program that takes in a description of a cryptographic algorithm implementation's capabilities, and generates test vectors to ensure the implementation conforms to the standard. After generating the test vectors, the program also validates the correctness of the responses from the user.

Feature preparation Preprocessing was applied to the data, such as creating dummy variables and performing transformations (centering, scaling, YeoJohnson) using the preProcess() function from the “caret” package in R. The correlation among the variables was examined and no serious multicollinearity problems were found. A stepwise variable selection was performed using a logistic regression model. The final set of variables included: Demographic: age, body mass index, sex, ethnicity, smoking History of disease: heart disease, migraine, insomnia, gastrointestinal disease, COVID-19 history: covid vaccination, rashes, conjunctivitis, shortness of breath, chest pain, cough, runny nose, dysgeusia, muscle and joint pain, fatigue, fever ,COVID-19 reinfection, and ICU admission. These variables were used to train and test various machine-learning models Model selection and training The data was randomly split into 80% training and 20% testing subsets. The “h2o” package in R version 4.3.1 was employed to implement different algorithms. AutoML was first used, which automatically explored a range of models with different configurations. Gradient Boosting Machines (GBM), Random Forest (RF), and Regularized Generalized Linear Model (GLM) were identified as the best-performing models on our data and their parameters were fine-tuned. An ensemble method that stacked different models together was also used, as it could sometimes improve the accuracy. The models were evaluated using the area under the curve (AUC) and C-statistics as diagnostic measures. The model with the highest AUC was selected for further analysis using the confusion matrix, accuracy, sensitivity, specificity, and F1 and F2 scores. The optimal prediction threshold was determined by plotting the sensitivity, specificity, and accuracy and choosing the point of intersection as it balanced the trade-off between the three metrics. The model’s predictions were also plotted, and the quantile ranges were used to classify the model’s prediction as follows: > 1st quantile, > 2nd quantile, > 3rd quartile and < 3rd quartile (very low, low, moderate, high) respectively. Metric Formula C-statistics (TPR + TNR - 1) / 2 Sensitivity/Recall TP / (TP + FN) Specificity TN / (TN + FP) Accuracy (TP + TN) / (TP + TN + FP + FN) F1 score 2 * (precision * recall) / (precision + recall) Model interpretation We used the variable importance plot, which is a measure of how much each variable contributes to the prediction power of a machine learning model. In H2O package, variable importance for GBM and RF is calculated by measuring the decrease in the model's error when a variable is split on. The more a variable's split decreases the error, the more important that variable is considered to be. The error is calculated using the following formula: 𝑆𝐸=𝑀𝑆𝐸∗𝑁=𝑉𝐴𝑅∗𝑁 and then it is scaled between 0 and 1 and plotted. Also, we used The SHAP summary plot which is a graphical tool to visualize the impact of input features on the prediction of a machine learning model. SHAP stands for SHapley Additive exPlanations, a method to calculate the contribution of each feature to the prediction by averaging over all possible subsets of features [28]. SHAP summary plot shows the distribution of the SHAP values for each feature across the data instances. We use the h2o.shap_summary_plot() function in R to generate the SHAP summary plot for our GBM model. We pass the model object and the test data as arguments, and optionally specify the columns (features) we want to include in the plot. The plot shows the SHAP values for each feature on the x-axis, and the features on the y-axis. The color indicates whether the feature value is low (blue) or high (red). The plot also shows the distribution of the feature values as a density plot on the right.

Introduction

This datasets have SQL injection attacks (SLQIA) as malicious Netflow data. The attacks carried out are SQL injection for Union Query and Blind SQL injection. To perform the attacks, the SQLMAP tool has been used.

NetFlow traffic has generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic). NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device.

Datasets

The firts dataset was colleted to train the detection models (D1) and other collected using different attacks than those used in training to test the models and ensure their generalization (D2).

The datasets contain both benign and malicious traffic. All collected datasets are balanced.

The version of NetFlow used to build the datasets is 5.

    Dataset
    Aim
    Samples
    Benign-malicious
    traffic ratio




    D1
    Training
    400,003
    50%


    D2
    Test
    57,239
    50%

Infrastructure and implementation

Two sets of flow data were collected with DOROTHEA. DOROTHEA is a Docker-based framework for NetFlow data collection. It allows you to build interconnected virtual networks to generate and collect flow data using the NetFlow protocol. In DOROTHEA, network traffic packets are sent to a NetFlow generator that has a sensor ipt_netflow installed. The sensor consists of a module for the Linux kernel using Iptables, which processes the packets and converts them to NetFlow flows.

DOROTHEA is configured to use Netflow V5 and export the flow after it is inactive for 15 seconds or after the flow is active for 1800 seconds (30 minutes)

Benign traffic generation nodes simulate network traffic generated by real users, performing tasks such as searching in web browsers, sending emails, or establishing Secure Shell (SSH) connections. Such tasks run as Python scripts. Users may customize them or even incorporate their own. The network traffic is managed by a gateway that performs two main tasks. On the one hand, it routes packets to the Internet. On the other hand, it sends it to a NetFlow data generation node (this process is carried out similarly to packets received from the Internet).

The malicious traffic collected (SQLI attacks) was performed using SQLMAP. SQLMAP is a penetration tool used to automate the process of detecting and exploiting SQL injection vulnerabilities.

The attacks were executed on 16 nodes and launch SQLMAP with the parameters of the following table.

    Parameters
    Description




    '--banner','--current-user','--current-db','--hostname','--is-dba','--users','--passwords','--privileges','--roles','--dbs','--tables','--columns','--schema','--count','--dump','--comments', --schema'
    Enumerate users, password hashes, privileges, roles, databases, tables and columns


    --level=5
    Increase the probability of a false positive identification


    --risk=3
    Increase the probability of extracting data


    --random-agent
    Select the User-Agent randomly


    --batch
    Never ask for user input, use the default behavior


    --answers="follow=Y"
    Predefined answers to yes

Every node executed SQLIA on 200 victim nodes. The victim nodes had deployed a web form vulnerable to Union-type injection attacks, which was connected to the MYSQL or SQLServer database engines (50% of the victim nodes deployed MySQL and the other 50% deployed SQLServer).

The web service was accessible from ports 443 and 80, which are the ports typically used to deploy web services. The IP address space was 182.168.1.1/24 for the benign and malicious traffic-generating nodes. For victim nodes, the address space was 126.52.30.0/24. The malicious traffic in the test sets was collected under different conditions. For D1, SQLIA was performed using Union attacks on the MySQL and SQLServer databases.

However, for D2, BlindSQL SQLIAs were performed against the web form connected to a PostgreSQL database. The IP address spaces of the networks were also different from those of D1. In D2, the IP address space was 152.148.48.1/24 for benign and malicious traffic generating nodes and 140.30.20.1/24 for victim nodes.

To run the MySQL server we ran MariaDB version 10.4.12. Microsoft SQL Server 2017 Express and PostgreSQL version 13 were used.

Data Description

We release the synthetic data generated using the method described in the paper Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (ACL 2024 Findings). The external knowledge we use is based on LLM-generated topics and writing styles.

  Generated Datasets

The original train/validation/test data, and the generated synthetic training data are listed as follows. For each dataset, we generate 5000… See the full description on the dataset page: https://huggingface.co/datasets/ritaranx/clinical-synthetic-text-llm.

Data Package for "Learning How to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection"

This package contains data generated as part of our experiments on adaptive fitness function selection as part of unit test generation for Java systems.

This paper is currently under submission. A draft of the paper is included in the data package.

This package contains experimental data (in folder "experiment_data"), including goal attainment, fault detection, time per generation, and choices made by the reinforcement learning algorithm. In the folder "test_suites", the suites generated by each technique are included for each project.

If you have questions, please contact Gregory Gay at greg@greggay.com.

NOTE: A small number of items are currently missing from this data package and will be added shortly. Please make sure you have the latest version of this package.

The Test Data Generation Tools market is experiencing robust growth, driven by the increasing demand for high-quality software and the rising adoption of agile and DevOps methodologies. The market's expansion is fueled by several factors, including the need for realistic and representative test data to ensure thorough software testing, the growing complexity of applications, and the increasing pressure to accelerate software delivery cycles. The market is segmented by type (Random, Pathwise, Goal, Intelligent) and application (Large Enterprises, SMEs), each demonstrating unique growth trajectories. Intelligent test data generation, offering advanced capabilities like data masking and synthetic data creation, is gaining significant traction, while large enterprises are leading the adoption due to their higher testing volumes and budgets. Geographically, North America and Europe currently hold the largest market shares, but the Asia-Pacific region is expected to witness significant growth due to rapid digitalization and increasing software development activities. Competitive intensity is high, with a mix of established players like IBM and Informatica and emerging innovative companies continuously introducing advanced features and functionalities. The market's growth is, however, constrained by challenges such as the complexity of implementing and managing test data generation tools and the need for specialized expertise. Overall, the market is projected to maintain a healthy growth rate throughout the forecast period (2025-2033), driven by continuous technological advancements and evolving software testing requirements. While the precise CAGR isn't provided, assuming a conservative yet realistic CAGR of 15% based on industry trends and the factors mentioned above, the market is poised for significant expansion. This growth will be fueled by the increasing adoption of cloud-based solutions, improved data masking techniques for enhanced security and privacy, and the rise of AI-powered test data generation tools that automatically create comprehensive and realistic datasets. The competitive landscape will continue to evolve, with mergers and acquisitions likely shaping the market structure. Furthermore, the focus on data privacy regulations will influence the development and adoption of advanced data anonymization and synthetic data generation techniques. The market will see further segmentation as specialized tools catering to specific industry needs (e.g., financial services, healthcare) emerge. The long-term outlook for the Test Data Generation Tools market remains positive, driven by the relentless demand for higher software quality and faster development cycles.

Dataset, splits, models, and scripts from the manuscript "When Do Quantum Mechanical Descriptors Help Graph Neural Networks Predict Chemical Properties?" are provided. The curated dataset includes 37 QM descriptors for 64,921 unique molecules across six levels of theory: wB97XD, B3LYP, M06-2X, PBE0, TPSS, and BP86. This dataset is stored in the data.tar.gz file, which also contains a file for multitask constraints applied to various atomic and bond properties. The data splits (training, validation, and test splits) for both random and scaffold-based divisions are saved as separate index files in splits.tar.gz. The trained D-MPNN models for predicting QM descriptors are saved in the models.tar.gz file. The scripts.tar.gz file contains ready-to-use scripts for training machine learning models to predict QM descriptors, as well as scripts for predicting QM descriptors using our trained models on unseen molecules and for applying radial basis function (RBF) expansion to QM atom and bond features.

Below are descriptions of the available scripts:

1. atom_bond_descriptors.sh: Trains atom/bond targets.
2. atom_bond_descriptors_predict.sh: Predicts atom/bond targets from pre-trained model.
3. dipole_quadrupole_moments.sh: Trains dipole and quadrupole moments.
4. dipole_quadrupole_moments_predict.sh: Predicts dipole and quadrupole moments from pre-trained model.
5. energy_gaps_IP_EA.sh: Trains energy gaps, ionization potential (IP), and electron affinity (EA).
6. energy_gaps_IP_EA_predict.sh: Predicts energy gaps, IP, and EA from pre-trained model.
7. get_constraints.py: Generates constraints file for testing dataset. This generated file needs to be provided before using our trained models to predict the atom/bond QM descriptors of your testing data.
8. csv2pkl.py: Converts QM atom and bond features to .pkl files using RBF expansion for use with Chemprop software.

Below is the procedure for running the ml-QM-GNN on your own dataset:

1. Use get_constraints.py to generate a constraint file required for predicting atom/bond QM descriptors with the trained ML models.
2. Execute atom_bond_descriptors_predict.sh to predict atom and bond properties. Run dipole_quadrupole_moments_predict.sh and energy_gaps_IP_EA_predict.sh to calculate molecular QM descriptors.
3. Utilize csv2pkl.py to convert the data from predicted atom/bond descriptors .csv file into separate atom and bond feature files (which are saved as .pkl files here).
4. Run Chemprop to train your models using the additional predicted features supported here.

The code, strainenergy_v4_1.m, was used for generating and processing the dataset for load-displacement and stress-strain. Software Matlab version 6.1 was used for running the code. The specific variables of the parameters used to generate the current dataset are as follows:• ip1: input file containing the load-displacement data• diameter: fascicle diameter• laststrainpt: an estimate of the strain at rupture, r• orderpoly: an integral value from 2-7 which represents the order of the polynomial for fitting to the data from O to q• loadat1percent: y/n; to determine the value of the load (set at 1% of the maximum load) at which the specimen became taut. ‘y’ denotes yes; ‘n’ denotes no.The logfile.txt, contains the parameters used for deriving the values of the respective mechanical properties.

While automated test generation can decrease the human burden associated with testing, it does not eliminate this burden. Humans must still work with generated test cases to interpret testing results, debug the code, build and maintain a comprehensive test suite, and many other tasks. Therefore, a major challenge with automated test generation is understandability of generated test test cases.

Large language models (LLMs), machine learning models trained on massive corpora of textual data - including both natural language and programming languages - are an emerging technology with great potential for performing language-related predictive tasks such as translation, summarization, and decision support.

In this study, we are exploring the capabilities of LLMs with regard to improving test case understandability.

This package contains the data produced during this exploration:

The examples directory contains the three case studies we tested our transformation process on:

queue_example: Tests of a basic queue data structure

httpie_sessions: Tests of the sessions module from the httpie project.

string_utils_validation: Tests of the validation module from the python-string-utils project.

Each directory contains the modules-under-test, the original test cases generated by Pynguin, and the transformed test cases.

Two trials were performed per case example of the transformation technique to assess the impact of different results from the LLM.

The survey directory contains the survey that was sent to assess the impact of the transformation on test readability.

survey.pdf contains the survey questions.

responses.xlsx contains the survey results.

Published at the 46th International Conference on Software Engineering (ICSE 2024). Here you can find a preprint.About the artifactsdataset.csv.gzeach row represents one test casecolumn "test_type": was the generated or developer-writtencolumn "flaky": has the test shown flaky behavior, and what kind? (NOD = non-order-dependent, OD = order-dependent)used to answer RQ1 (Prevalence) and RQ2 (Flakiness Suppression).LoC.zipcontains lines-of-code data for the Java and Python projectsflaky_java_projects.zip and flaky_python_projects.ziparchives containing the 418 Java and 531 Python projects that contained at least one flaky testeach project contains the developer written and generated test suitesmanual_rootCausing.zipresults of the manual root cause classificationfull_sample.csvcolumn "rater": which of the four researchers conducting the classification rated this test (alignment = all four)used to answer RQ3 (Root Causes)Running the jupyter notebookDownload all artifactsCreate and activate virtual environmentvirtualenv -p venvsource venv/bin/activateInstall dependenciespip install -r requirements.txtStart jupyter labpython -m jupyter labScripts used for test generation and executionJava (EvoSuite)Python (Pynguin)

SDC-Scissor tool for Cost-effective Simulation-based Test Selection in Self-driving Cars Software

This dataset provides test cases for self-driving cars with the BeamNG simulator. Check out the repository and demo video to get started.

GitHub: github.com/ChristianBirchler/sdc-scissor

This project extends the tool competition platform from the Cyber-Phisical Systems Testing Competition which was part of the SBST Workshop in 2021.

Usage

Demo

YouTube Link

Installation

The tool can either be run with Docker or locally using Poetry.

When running the simulations a working installation of BeamNG.research is required. Additionally, this simulation cannot be run in a Docker container but must run locally.

To install the application use one of the following approaches:

• Docker: docker build --tag sdc-scissor .
• Poetry: poetry install

Using the Tool

The tool can be used with the following two commands:

• Docker: docker run --volume "$(pwd)/results:/out" --rm sdc-scissor [COMMAND] [OPTIONS] (this will write all files written to /out to the local folder results)
• Poetry: poetry run python sdc-scissor.py [COMMAND] [OPTIONS]

There are multiple commands to use. For simplifying the documentation only the command and their options are described.

• Generation of tests:
  • generate-tests --out-path /path/to/store/tests
• Automated labeling of Tests:
  • label-tests --road-scenarios /path/to/tests --result-folder /path/to/store/labeled/tests
  • Note: This only works locally with BeamNG.research installed
• Model evaluation:
  • evaluate-models --dataset /path/to/train/set --save
• Split train and test data:
  • split-train-test-data --scenarios /path/to/scenarios --train-dir /path/for/train/data --test-dir /path/for/test/data --train-ratio 0.8
• Test outcome prediction:
  • predict-tests --scenarios /path/to/scenarios --classifier /path/to/model.joblib
• Evaluation based on random strategy:
  • evaluate --scenarios /path/to/test/scenarios --classifier /path/to/model.joblib

The possible parameters are always documented with --help.

Linting

The tool is verified the linters flake8 and pylint. These are automatically enabled in Visual Studio Code and can be run manually with the following commands:

poetry run flake8 .
poetry run pylint **/*.py

License

The software we developed is distributed under GNU GPL license. See the LICENSE.md file.

Contacts

Christian Birchler - Zurich University of Applied Science (ZHAW), Switzerland - birc@zhaw.ch

Nicolas Ganz - Zurich University of Applied Science (ZHAW), Switzerland - gann@zhaw.ch

Sajad Khatiri - Zurich University of Applied Science (ZHAW), Switzerland - mazr@zhaw.ch

Dr. Alessio Gambi - Passau University, Germany - alessio.gambi@uni-passau.de

Dr. Sebastiano Panichella - Zurich University of Applied Science (ZHAW), Switzerland - panc@zhaw.ch

References

• Christian Birchler, Nicolas Ganz, Sajad Khatiri, Alessio Gambi, and Sebastiano Panichella. 2022. Cost-effective Simulation-based Test Selection in Self-driving Cars Software with SDC-Scissor. In 2022 IEEE 29th International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE.

If you use this tool in your research, please cite the following papers:

@INPROCEEDINGS{Birchler2022,
 author={Birchler, Christian and Ganz, Nicolas and Khatiri, Sajad and Gambi, Alessio, and Panichella, Sebastiano},
 booktitle={2022 IEEE 29th International Conference on Software Analysis, Evolution and Reengineering (SANER), 
 title={Cost-effective Simulationbased Test Selection in Self-driving Cars Software with SDC-Scissor}, 
 year={2022},
}

The NHTSA Vehicle Crash Test Database contains engineering data measured during various types of research, the New Car Assessment Program (NCAP), and compliance crash tests. Information in this database refers to the performance and response of vehicles and other structures in impacts. This database is not intended to support general consumer safety issues. For general consumer information please see the NHTSA's information on buying a safer car.