Dataflow

## Overview

Dataflow is a dataset for object detection tasks - it contains Defect annotations for 613 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Dataflow(test)

## Overview

Dataflow(test) is a dataset for object detection tasks - it contains Dots Defect Test annotations for 405 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

VIVO aims to support an open and networked research ecosystem. This workshop will apply methods to understand VIVO’s interaction with various data source and the existing data ingest needs and challenges, highlighting how one can architect data ingest flows into your VIVO. We will cover the use of Karma, the VIVO harvester, and how Symplectic uses the Harvester, and how these tools are connected architecturally to the whole of the VIVO platform. The goal is to understand the diversity of tools and learn why and how different approaches to data ingest would meet specific use cases.

This deposit provides code and additional proofs associated to the paper "Data-flow analyses as effects and graded monads" appearing at FSCD 2020 (5th International Conference On Formal Structures for Computation and Deduction). extra-proofs.pdf provides additional proofs not included in the appendix of the published paper for space reasons. GradedMonad.agda provides further mechanised proofs, referred to from extra-proofs.pdf dataflow-effects-as-grades-fscd2020.zip provides the source code corresponding to Section 4.4 and Appendix B The code is hosted on GitHub as well: https://github.com/dorchard/dataflow-effects-as-grades This .zip corresponds to this release https://github.com/dorchard/dataflow-effects-as-grades/releases/tag/fscd2020 Unzip and see README.md for details on how to build and interact with this code

The AGENT project aims at integrating data from different sources (genebanks, research institutes, international archives) and types (passport, phenotypic, genomic data).

These guidelines have been developed to explain the data flow within the AGENT project and should be useful for other projects.

The phenotypic data templates are included.

The Healthcare Operational Data Flows (HODF): Acute Data Set provides an automated patient-based daily data collection to support NHS delivery plans for the recovery of elective care and emergency and urgent care.

The AGENT project aims at integrating data from different sources (genebanks, research institutes, international archives) and types (passport, phenotypic, genomic data). These guidelines have been developed to explain the data flow within the AGENT project and should be useful for other projects. The phenotypic data templates are included.

Amidst growing skepticism towards globalization and rising digital trade, this study investigates the impact of Restrictions on Cross-Border Data Flows (RCDF) on Domestic Value Chains (DVCs) stability. As global value chains participation declines, the stability of DVCs—integral to internal economic dynamics—becomes crucial. This study situates within a framework exploring the role of innovation and RCDF in the increasingly interconnected global trade. Using a panel data fixed effect model, our analysis provides insights into the varying effects of RCDF on DVCs stability across countries with diverse economic structures and technological advancement levels. This approach allows for a nuanced understanding of the interplay between digital trade policies, value chain stability, and innovation. RCDF tend to disrupt DVCs by negatively impacting innovation, which necessitates proactive policy measures to mitigate these effects. In contrast, low-income countries experience a less detrimental impact; RCDF may even aid in integrating their DVCs into Global Value Chains, enhancing economic stability. It underscores the need for dynamic, adaptable policies and global collaboration to harmonize digital trade standards, thus offering guidance for policy-making in the context of an interconnected global economy.

Company Demographic Profile: General data: Flows of employing legal units by autonomous community. Quarterly. Autonomous Communities and Cities.

Company Demographic Profile: General data: Flows of self-employed workers by autonomous community. Quarterly. Autonomous Communities and Cities.

The global market for stream data pipeline processing tools is experiencing robust growth, driven by the exponential increase in real-time data generated from various sources, including IoT devices, social media, and e-commerce platforms. The demand for immediate insights and actionable intelligence from this data is fueling the adoption of these tools across diverse industries, such as finance, healthcare, and manufacturing. The market, estimated at $15 billion in 2025, is projected to exhibit a Compound Annual Growth Rate (CAGR) of 20% from 2025 to 2033, reaching approximately $60 billion by 2033. This growth is propelled by several factors, including the increasing adoption of cloud-based solutions, the need for enhanced data security and governance, and the growing prevalence of advanced analytics techniques like machine learning and AI, all requiring efficient stream processing capabilities. Key players like Google, AWS, Microsoft, and IBM are leading the market, driving innovation through continuous product enhancements and strategic acquisitions. However, challenges such as data complexity, integration complexities across diverse systems, and the need for skilled professionals to manage these systems act as restraints. The market segmentation reveals a strong preference for cloud-based solutions due to their scalability and cost-effectiveness. The North American region currently holds the largest market share, followed by Europe and Asia-Pacific. However, the Asia-Pacific region is witnessing the fastest growth, fueled by rising digitalization and technological advancements. The competitive landscape is highly dynamic, with established players and emerging startups vying for market share. This necessitates continuous innovation in areas like enhanced real-time analytics capabilities, improved data security features, and integration with other business intelligence platforms. The future of the stream data pipeline processing tool market appears promising, with continued growth driven by the increasing volume and velocity of data generated in a rapidly digitalizing world.

[Deprecated version, used in the support article, please download the last version]

This record is a global open-source passenger air traffic dataset primarily dedicated to the research community.
It gives a seating capacity available on each origin-destination route for a given year, 2019, and the associated aircraft and airline when this information is available.

Context on the original work is given in the related article (https://journals.open.tudelft.nl/joas/article/download/7201/5683) and on the associated GitHub page (https://github.com/AeroMAPS/AeroSCOPE/).
A simple data exploration interface will be available at www.aeromaps.eu/aeroscope.
The dataset was created by aggregating various available open-source databases with limited geographical coverage. It was then completed using a route database created by parsing Wikipedia and Wikidata, on which the traffic volume was estimated using a machine learning algorithm (XGBoost) trained using traffic and socio-economical data.

1- DISCLAIMER

The dataset was gathered to allow highly aggregated analyses of the air traffic, at the continental or country levels. At the route level, the accuracy is limited as mentioned in the associated article and improper usage could lead to erroneous analyses.

2- DESCRIPTION

Each data entry represents an (Origin-Destination-Operator-Aircraft type) tuple.

Please refer to the support article for more details (see above).

The dataset contains the following columns:

• "First column" : index
• airline_iata : IATA code of the operator in nominal cases. An ICAO -> IATA code conversion was performed for some sources, and the ICAO code was kept if no match was found.
• acft_icao : ICAO code of the aircraft type
• acft_class : Aircraft class identifier, own classification.
  • WB: Wide Body
  • NB: Narrow Body
  • RJ: Regional Jet
  • PJ: Private Jet
  • TP: Turbo Propeller
  • PP: Piston Propeller
  • HE: Helicopter
  • OTHER
• seymour_proxy: Aircraft code for Seymour Surrogate (https://doi.org/10.1016/j.trd.2020.102528), own classification to derive proxy aircraft when nominal aircraft type unavailable in the aircraft performance model.
• source: Original data source for the record, before compilation and enrichment.
  • ANAC: Brasilian Civil Aviation Authorities
  • AUS Stats: Australian Civil Aviation Authorities
  • BTS: US Bureau of Transportation Statistics T100
  • Estimation: Own model, estimation on Wikipedia-parsed route database
  • Eurocontrol: Aggregation and enrichment of R&D database
  • OpenSky
  • World Bank
• seats: Number of seats available for the data entry, AFTER airport residual scaling
• n_flights: Number of flights of the data entry, when available
• iata_departure, iata_arrival : IATA code of the origin and destination airports. Some BTS inhouse identifiers could remain but it is marginal.
• departure_lon, departure_lat, arrival_lon, arrival_lat : Origin and destination coordinates, could be NaN if the IATA identifier is erroneous
• departure_country, arrival_country: Origin and destination country ISO2 code. WARNING: disable NA (Namibia) as default NaN at import
• departure_continent, arrival_continent: Origin and destination continent code. WARNING: disable NA (North America) as default NaN at import
• seats_no_est_scaling: Number of seats available for the data entry, BEFORE airport residual scaling
• distance_km: Flight distance (km)
• ask: Available Seat Kilometres
• rpk: Revenue Passenger Kilometres (simple calculation from ASK using IATA average load factor)
• fuel_burn_seymour: Fuel burn per flight (kg) when seymour proxy available
• fuel_burn: Total fuel burn of the data entry (kg)
• co2: Total CO2 emissions of the data entry (kg)
• domestic: Domestic/international boolean (Domestic=1, International=0)

3- Citation

Please cite the support paper instead of the dataset itself.

Salgas, A., Sun, J., Delbecq, S., Planès, T., & Lafforgue, G. (2023). Compilation of an open-source traffic and CO2 emissions dataset for commercial aviation. Journal of Open Aviation Science. https://doi.org/10.59490/joas.2023.7201

The global market for stream data pipeline processing tools is experiencing robust growth, driven by the increasing volume and velocity of data generated across diverse industries. The market, estimated at $15 billion in 2025, is projected to expand at a Compound Annual Growth Rate (CAGR) of 18% from 2025 to 2033. This significant growth is fueled by several key factors: the rising adoption of cloud-native architectures, the proliferation of real-time analytics applications (particularly in finance and security), and the increasing need for efficient and scalable data processing solutions to handle the ever-growing data streams from IoT devices, social media, and other sources. The demand for real-time insights is a major driver, pushing organizations to adopt tools capable of processing and analyzing data instantly, rather than relying on batch processing methods. Further, the continued expansion of cloud computing and the availability of sophisticated, managed services are simplifying implementation and reducing the total cost of ownership for these tools. The market is segmented by tool type (real-time, proprietary, and cloud-native) and application (finance and security, with other sectors like healthcare and logistics also showing increasing adoption). While North America currently holds a dominant market share, fueled by early adoption and a strong technology ecosystem, regions like Asia-Pacific are experiencing rapid growth due to increasing digitalization and investment in data infrastructure. However, factors such as the complexity of implementation, the need for skilled personnel, and data security concerns pose challenges to market expansion. The competitive landscape is highly fragmented, with a mix of established players like Google, IBM, and Microsoft, alongside emerging niche providers. The ongoing innovation in areas such as AI-powered data processing, serverless architectures, and enhanced security features will continue to shape the market landscape in the coming years.

data of the simulation.

Real-time neuron detection and neural activity extraction are critical components of real-time neural decoding. They are modeled effectively in dataflow graphs. However, these graphs and the components within them in general have many parameters, including hyper-parameters associated with machine learning sub-systems. The dataflow graph parameters induce a complex design space, where alternative configurations (design points) provide different trade-offs involving key operational metrics including accuracy and time-efficiency. In this paper, we propose a novel optimization framework that automatically configures the parameters in different neural decoders. The proposed optimization framework is evaluated in depth through two case studies. Significant performance improvement in terms of accuracy and efficiency is observed in both case studies compared to the manual parameter optimization that was associated with the published results of those case studies. Additionally, we investigate the application of efficient multi-threading strategies to speed-up the running time of our parameter optimization framework. Our proposed optimization framework enables efficient and effective estimation of parameters, which leads to more powerful neural decoding capabilities and allows researchers to experiment more easily with alternative decoding models.

The Data Pipeline Tools Market size was valued at USD 8.4 billion in 2023 and is projected to reach USD 38.95 billion by 2032, exhibiting a CAGR of 24.5 % during the forecasts period. Data pipeline tools are software solutions engineered to streamline and automate the efficient movement of data from diverse sources to destinations like databases, data warehouses, or analytical systems. These tools are pivotal in contemporary data architecture, facilitating the ingestion, processing, transformation, and storage of data. They typically offer functionalities such as extracting data from sources (e.g., databases, APIs, files), transforming data (cleaning, filtering, aggregating), and loading data into target systems. Key characteristics of data pipeline tools include scalability to manage large data volumes, fault tolerance to ensure data reliability and integrity, and support for both real-time and batch processing based on business requirements. They often provide graphical user interfaces or APIs for configuring data workflows, scheduling tasks, monitoring data flows, and managing dependencies between operations. Data pipeline tools cater to a wide range of applications across industries, encompassing data integration for business intelligence, system-to-system data migration, ETL processes for data warehousing, and real-time data processing for operational analytics. Notable examples of these tools include Apache Airflow, Apache Kafka, AWS Glue, Google Cloud Dataflow, and Informatica. By automating data workflows and maintaining consistency and reliability in data movement, these tools empower organizations to accelerate decision-making, enhance data quality, and optimize operational efficiency. They are indispensable for modern enterprises striving to harness data as a strategic asset for achieving competitive advantages and fostering business growth.

Data was collected using a boat-mounted Dataflow onboard flow-through collection system (Madden and Day 1992). While the boat is underway, the Dataflow receives a continuous stream of water from an onboard pump that is routed to a series of sensors operating in flow-through mode. These sensors measure the physical and optical properties of water passing through the system at 6 second intervals (approximately every 70 m of boat travel). Measurements are georeferenced by an onboard GPS unit with a horizontal accuracy of ± 250 cm. Each Dataflow survey is supplemented by a set of discrete grab samples. These samples were collected from the Dataflow outflow hose while underway and were analyzed for chlorophyll concentration, total suspended solids, as well as a suite of organic and inorganic nutrient species. Dataflow surveys took place on a quarterly to bimonthly interval from 2008 to 2015.

The global stream processing frameworks market is experiencing robust growth, driven by the exponential increase in data volume and velocity across diverse sectors. The market, estimated at $5 billion in 2025, is projected to exhibit a Compound Annual Growth Rate (CAGR) of 15% from 2025 to 2033, reaching approximately $15 billion by 2033. This growth is fueled by the rising adoption of real-time analytics, the proliferation of IoT devices generating massive data streams, and the increasing need for faster, more efficient data processing solutions in applications like fraud detection, risk management, and personalized marketing. Key drivers include the demand for low-latency processing, enhanced scalability, and improved data integration capabilities. The market is witnessing significant innovation in areas such as serverless stream processing, edge computing for real-time analytics, and the adoption of cloud-based stream processing platforms. Leading players like VMware, Amazon, Google, and IBM are actively investing in research and development, further propelling market expansion. The market segmentation reveals a diverse landscape with several prominent players competing across various niches. While established vendors like VMware and IBM offer comprehensive enterprise-grade solutions, cloud providers like Amazon and Google are capturing significant market share with their scalable and cost-effective offerings. Specialized vendors like WISI Germany and VITEC cater to specific industry needs. The competitive landscape is characterized by strategic partnerships, mergers, and acquisitions, as companies strive to expand their market reach and offer integrated solutions. Despite the robust growth, the market faces certain restraints, including the complexity of implementing stream processing frameworks, the need for specialized skills, and concerns surrounding data security and privacy. However, ongoing advancements in technology and the increasing availability of skilled professionals are expected to mitigate these challenges over the forecast period.

The global stream processing frameworks market is experiencing robust growth, driven by the exponential increase in data volume generated from various sources like IoT devices, social media, and e-commerce platforms. The need for real-time analytics and immediate insights from this data fuels the demand for efficient and scalable stream processing solutions. Major players like VMware, Amazon, Google, and IBM are heavily invested in this space, offering cloud-based and on-premise solutions catering to diverse business needs. The market is segmented by deployment (cloud, on-premise), application (fraud detection, real-time analytics, risk management), and organization size (SMEs, large enterprises). We estimate the market size in 2025 to be $5 billion, growing at a Compound Annual Growth Rate (CAGR) of 20% through 2033. This growth is fueled by the increasing adoption of cloud computing, the rise of big data analytics, and the increasing demand for real-time decision-making across industries. The market's expansion is, however, tempered by challenges like data security concerns, the need for skilled professionals, and the complexity of integrating stream processing frameworks with existing IT infrastructure. The competitive landscape is highly dynamic, with established tech giants competing with specialized vendors like WISI Germany, Harmonic, and VITEC. Open-source frameworks like Apache Kafka also play a significant role, offering cost-effective alternatives. Future growth will be shaped by advancements in technologies like AI and machine learning, which are being integrated into stream processing platforms to enhance their analytical capabilities. The focus will also shift towards edge computing, enabling real-time processing closer to the data source, thereby reducing latency and improving efficiency. The market is expected to see increased consolidation as larger players acquire smaller companies to expand their product portfolios and strengthen their market position. Furthermore, the development of more user-friendly interfaces and simplified deployment models will accelerate adoption across diverse industry verticals.

The global Streaming Data Processing System Software market is experiencing robust growth, projected to reach $7,578.2 million in 2025 and exhibiting a Compound Annual Growth Rate (CAGR) of 14.5% from 2025 to 2033. This significant expansion is fueled by the increasing volume and velocity of data generated across diverse sectors, demanding real-time insights and analytics. Key drivers include the rising adoption of cloud-based solutions offering scalability and cost-effectiveness, coupled with the expanding need for efficient data processing in applications like financial services (high-frequency trading, fraud detection), healthcare (real-time patient monitoring), and manufacturing (predictive maintenance). Furthermore, advancements in technologies such as AI and machine learning are enhancing the capabilities of these systems, leading to more sophisticated applications. While market restraints include the complexities associated with data integration and security concerns, the overall market trajectory remains exceptionally positive. The market segmentation reveals a strong preference for cloud-based solutions over on-premises deployments, reflecting the ongoing shift towards cloud computing. Among application segments, Financial Services and Healthcare and Life Sciences currently lead, driven by their critical need for immediate data analysis. However, other sectors like Manufacturing/Supply Chain, Communications, Media & Entertainment, and Public Sector are rapidly adopting streaming data processing, contributing to the overall market expansion. The competitive landscape is intensely dynamic, featuring major technology players like Google, Microsoft, AWS, and Oracle, alongside specialized providers like Confluent and TIBCO. The geographic distribution of the market shows North America and Europe holding a significant share currently; however, Asia-Pacific is poised for rapid growth, driven by increasing digitalization and infrastructure investments in emerging economies like India and China. The market's future growth will hinge on continued technological innovation, expanding adoption across diverse sectors, and the development of robust security frameworks to address data privacy and integrity concerns.