Netflow traffic generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic) NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. Netflow flows have been captured by sampling at the packet level. A sampling means that 1 out of every X packets is selected to be flow while the rest of the packets are not valued. In the construction of the datasets, different percentages of flows considered attacks and flows considered normal traffic have been used. These datasets have been used to test previously trained models.

NetFlow traffic generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic) NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. NetFlow flows have been captured with different sampling at the packet level. A sampling means that 1 out of every X packets is selected to be flow while the rest of the packets are not valued. The version of NetFlow used to build the datasets is 5.

NetFlow traffic generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic) NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. NetFlow flows have been captured with sampling 1000 at the packet level. A sampling means that 1 out of every X packets is selected to be flow while the rest of the packets are not valued. The version of NetFlow used to build the datasets is 5.

AIT Netflow Data Sets

This repository contains labeled synthetic netflows suitable for evaluation of intrusion detection systems, federated learning, and alert aggregation. The netflows are generated from the packet captures contained in the AIT-LDS-v2.0. A detailed description of that dataset is available in [1]. The packet captures were collected from eight testbeds that were built at the Austrian Institute of Technology (AIT) following the approach by [2]. Please cite these papers if the data is used for academic publications.

In brief, each of the datasets corresponds to a testbed representing a small enterprise network including mail server, file share, WordPress server, VPN, firewall, etc. Normal user behavior is simulated to generate background noise over a time span of 4-6 days. At some point, a sequence of attack steps is launched against the network. The following attacks are launched in the network:

• Scans (nmap, WPScan, dirb)
• Webshell upload (CVE-2020-24186)
• Password cracking (John the Ripper)
• Privilege escalation
• Remote command execution
• Data exfiltration (DNSteal)

This repository contains the following files:

• : CSV files of labeled TCP and UDP netflows for each testbed.
• label_info.txt: File describing which labels in TCP and UDP are benign and which ones are malicious.
• README.md: Instructions on how to reproduce the generation and labeling of the netflows from the AIT-LDS-v2.0. Note that it is only necessary to run the python scripts if you want to extend or change the labeling procedure.
• 1_format_dataset_info.ipynb: Generates the tables necessary for labeling (see README.md).
• 2_label_logs.ipynb: Labels the netflows (see README.md).

Acknowledgements: Partially funded by the FFG projects INDICAETING (868306) and DECEPT (873980), and the EU projects GUARD (833456) and PANDORA (SI2.835928).

If you use the dataset, please cite the following publications:

[1] M. Landauer, F. Skopik, M. Frank, W. Hotwagner, M. Wurzenberger, and A. Rauber. "Maintainable Log Datasets for Evaluation of Intrusion Detection Systems". IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 4, pp. 3466-3482. [PDF]

[2] M. Landauer, F. Skopik, M. Wurzenberger, W. Hotwagner and A. Rauber, "Have it Your Way: Generating Customized Log Datasets With a Model-Driven Simulation Testbed," in IEEE Transactions on Reliability, vol. 70, no. 1, pp. 402-415, March 2021, doi: 10.1109/TR.2020.3031317. [PDF]

NetFlow traffic generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic) NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. NetFlow flows have been captured with sampling 500 at the packet level. A sampling means that 1 out of every X packets is selected to be flow while the rest of the packets are not valued. The version of NetFlow used to build the datasets is 5.

Introduction This datasets have SQL injection attacks (SLQIA) as malicious Netflow data. The attacks carried out are SQL injection for Union Query and Blind SQL injection. To perform the attacks, the SQLMAP tool has been used. NetFlow traffic has generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic). NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device. Datasets The firts dataset was colleted to train the detection models (D1) and other collected using different attacks than those used in training to test the models and ensure their generalization (D2). The datasets contain both benign and malicious traffic. All collected datasets are balanced. The version of NetFlow used to build the datasets is 5. Dataset Aim Samples Benign-malicious
traffic ratio D1 Training 400,003 50% D2 Test 57,239 50% Infrastructure and implementation Two sets of flow data were collected with DOROTHEA. DOROTHEA is a Docker-based framework for NetFlow data collection. It allows you to build interconnected virtual networks to generate and collect flow data using the NetFlow protocol. In DOROTHEA, network traffic packets are sent to a NetFlow generator that has a sensor ipt_netflow installed. The sensor consists of a module for the Linux kernel using Iptables, which processes the packets and converts them to NetFlow flows. DOROTHEA is configured to use Netflow V5 and export the flow after it is inactive for 15 seconds or after the flow is active for 1800 seconds (30 minutes) Benign traffic generation nodes simulate network traffic generated by real users, performing tasks such as searching in web browsers, sending emails, or establishing Secure Shell (SSH) connections. Such tasks run as Python scripts. Users may customize them or even incorporate their own. The network traffic is managed by a gateway that performs two main tasks. On the one hand, it routes packets to the Internet. On the other hand, it sends it to a NetFlow data generation node (this process is carried out similarly to packets received from the Internet). The malicious traffic collected (SQLI attacks) was performed using SQLMAP. SQLMAP is a penetration tool used to automate the process of detecting and exploiting SQL injection vulnerabilities. The attacks were executed on 16 nodes and launch SQLMAP with the parameters of the following table. Parameters Description '--banner','--current-user','--current-db','--hostname','--is-dba','--users','--passwords','--privileges','--roles','--dbs','--tables','--columns','--schema','--count','--dump','--comments', --schema' Enumerate users, password hashes, privileges, roles, databases, tables and columns --level=5 Increase the probability of a false positive identification --risk=3 Increase the probability of extracting data --random-agent Select the User-Agent randomly --batch Never ask for user input, use the default behavior --answers="follow=Y" Predefined answers to yes Every node executed SQLIA on 200 victim nodes. The victim nodes had deployed a web form vulnerable to Union-type injection attacks, which was connected to the MYSQL or SQLServer database engines (50% of the victim nodes deployed MySQL and the other 50% deployed SQLServer). The web service was accessible from ports 443 and 80, which are the ports typically used to deploy web services. The IP address space was 182.168.1.1/24 for the benign and malicious traffic-generating nodes. For victim nodes, the address space was 126.52.30.0/24.
The malicious traffic in the test sets was collected under different conditions. For D1, SQLIA was performed using Union attacks on the MySQL and SQLServer databases. However, for D2, BlindSQL SQLIAs were performed against the web form connected to a PostgreSQL database. The IP address spaces of the networks were also different from those of D1. In D2, the IP address space was 152.148.48.1/24 for benign and malicious traffic generating nodes and 140.30.20.1/24 for victim nodes. To run the MySQL server we ran MariaDB version 10.4.12.
Microsoft SQL Server 2017 Express and PostgreSQL version 13 were used.

This dataset is comprised of NetFlow records, which capture the outbound network traffic of 8 commercial IoT devices and 5 non-IoT devices, collected during a period of 37 days in a lab at Ben-Gurion University of The Negev. The dataset was collected in order to develop a method for telecommunication providers to detect vulnerable IoT models behind home NATs. Each NetFlow record is labeled with the device model which produced it; for research reproducibilty, each NetFlow is also allocated to either the "training" or "test" set, in accordance with the partitioning described in:

Y. Meidan, V. Sachidananda, H. Peng, R. Sagron, Y. Elovici, and A. Shabtai, A novel approach for detecting vulnerable IoT devices connected behind a home NAT, Computers & Security, Volume 97, 2020, 101968, ISSN 0167-4048, https://doi.org/10.1016/j.cose.2020.101968. (http://www.sciencedirect.com/science/article/pii/S0167404820302418)

Please note:

• The dataset itself is free to use, however users are requested to cite the above-mentioned paper, which describes in detail the research objectives as well as the data collection, preparation and analysis.
• Following is a brief description of the features used in this dataset.

# NetFlow features, used in the related paper for analysis

'FIRST_SWITCHED': System uptime at which the first packet of this flow was switched
'IN_BYTES': Incoming counter for the number of bytes associated with an IP Flow
'IN_PKTS': Incoming counter for the number of packets associated with an IP Flow
'IPV4_DST_ADDR': IPv4 destination address
'L4_DST_PORT': TCP/UDP destination port number
'L4_SRC_PORT': TCP/UDP source port number
'LAST_SWITCHED': System uptime at which the last packet of this flow was switched
'PROTOCOL': IP protocol byte (6: TCP, 17: UDP)
'SRC_TOS': Type of Service byte setting when there is an incoming interface
'TCP_FLAGS': Cumulative of all the TCP flags seen for this flow

# Features added by the authors

'IP': Prefix of the destination IP address, representing the network (without the host)
'DURATION': Time (seconds) between first/last packet switching

# Label
'device_model':

# Partition
'partition': Training or test

# Additional NetFlow features (mostly zero-variance)
'SRC_AS': Source BGP autonomous system number
'DST_AS': Destination BGP autonomous system number
'INPUT_SNMP': Input interface index
'OUTPUT_SNMP': Output interface index
'IPV4_SRC_ADDR': IPv4 source address
'MAC': MAC address of the source

# Additional data
'category': IoT or non-IoT
'type': IoT, access_point, smartphone, laptop
'date': Datepart of FIRST_SWITCHED
'inter_arrival_time': Time (seconds) between successive flows of the same device (identified by its MAC address)

One day of Netflow version 5 collected in flow tools format at an academic department. Collection includes traffic between all switches within the department and the egress switch to the college, university, and Internet. Departmental IP addresses in the flows are anaonymized via constant substitution.

Train dataset only contains benign traffic, this dataset has been collected using Netflow and implementing a sampling rate of 1 packet out of each 1000 to generate the flows, simulating the conditions of the RedCayle’s routers.

The traffic has been generated using three Python scripts. The first one simulates email sending using SMTP protocol. The second script simulates SSH connections. The third script simulates a user browsing the internet using different search engines and different protocols like HTTP and HTTPS.

NF-BoT-IoT-V2 is the extended NetFlow version of NF-BoT-IoT. Compared to the original NF-NIDS datasets, the feature set of NetFlow features has expanded from 8 to 43.

This is one dataset in the NFV2-collection by the university of Queensland aimed at standardizing network-security datasets to achieve interoperability and larger analyses.

All credit goes to the original authors: Dr. Mohanad Sarhan, Dr. Siamak Layeghy and Dr. Marius Portmann. Please cite their original journal article when using this dataset.

V1: Base dataset in CSV format as downloaded from here V2: Cleaning -> parquet files

In the parquet files all data types are already set correctly, there are 0 records with missing information and 0 duplicate records.

AssureMOSS Kubernetes Run-time Monitoring Dataset This dataset contains NetFlow data that is collected from a Kubernetes cluster. The cluster is used to monitor the microservice applications that are running on the cluster. The goal is to use the (NetFlow) logs to learn a state machine model that models the normal network behaviour within the cluster. The state machine model is then used to monitor and detect potential anomalies that might occur during the runtime of the cluster. This dataset contains both benign data (produced by real-life users) and malicious data (produced by launching several attacks against the clusters). The label of each flow is included in the dataset.

Traffic collected from the Merit border router at Chicago. This real-world traffic is currently collected without packet sampling. Combined with the Netflow-1 and Netflow-3 datasets, it describes the majority of ingress and egress traffic of the Merit Network.

This dataset has SQL injection attacks as malicious Netflow data. The attacks carried out are SQL injection for Union Query and Blind SQL injection. To perform the attacks, the SQLmap tool has been used.

NetFlow traffic has generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic). NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device.

The version of NetFlow used to build the datasets is 5.

This dataset has SQL injection attacks as malicious Netflow data. The attacks carried out are SQL injection for Union Query and Blind SQL injection. To perform the attacks, the SQLmap tool has been used.

NetFlow traffic has generated using DOROTHEA (DOcker-based fRamework fOr gaTHering nEtflow trAffic). NetFlow is a network protocol developed by Cisco for the collection and monitoring of network traffic flow data generated. A flow is defined as a unidirectional sequence of packets with some common properties that pass through a network device.

The version of NetFlow used to build the datasets is 5.

Net flow of internationally mobile students (inbound - outbound), both sexes (number) in Ukraine was reported at --22431 Persons in 2018, according to the World Bank collection of development indicators, compiled from officially recognized sources. Ukraine - Net flow of internationally mobile students (inbound - outbound), both sexes - actual values, historical data, forecasts and projections were sourced from the World Bank on June of 2025.

Net flow ratio of internationally mobile students (inbound - outbound), both sexes (%) in India was reported at --0.95807 % in 2018, according to the World Bank collection of development indicators, compiled from officially recognized sources. India - Net flow ratio of internationally mobile students (inbound - outbound), both sexes - actual values, historical data, forecasts and projections were sourced from the World Bank on June of 2025.

Net flow of internationally mobile students (inbound - outbound), both sexes (number) in Gibraltar was reported at --1750 Persons in 2016, according to the World Bank collection of development indicators, compiled from officially recognized sources. Gibraltar - Net flow of internationally mobile students (inbound - outbound), both sexes - actual values, historical data, forecasts and projections were sourced from the World Bank on July of 2025.