Streaming Database as a Service Market Outlook

According to our latest research, the global Streaming Database as a Service market size reached USD 2.74 billion in 2024, driven by the increasing demand for real-time data processing and analytics across industries. The market is anticipated to expand at a robust CAGR of 26.8% during the forecast period, resulting in a projected market value of USD 23.25 billion by 2033. This dynamic growth is primarily fueled by the proliferation of data-intensive applications, the shift towards cloud-native architectures, and the need for businesses to derive actionable insights from streaming data sources in real time.

One of the primary growth factors for the Streaming Database as a Service market is the exponential increase in data generation from connected devices, IoT sensors, and digital platforms. As organizations strive to gain a competitive edge, the ability to analyze and act upon data as it is generated has become a critical differentiator. Streaming databases, delivered as a service, enable enterprises to ingest, process, and analyze vast volumes of data streams with minimal latency, supporting use cases such as fraud detection, real-time analytics, and dynamic customer engagement. The scalability and flexibility of cloud-based streaming databases further lower the barriers for adoption, making advanced analytics accessible to organizations of all sizes.

Another significant driver is the growing adoption of cloud computing and hybrid IT environments. Enterprises are increasingly migrating workloads to the cloud to enhance agility, reduce operational complexity, and optimize costs. Streaming Database as a Service solutions, available via public, private, and hybrid cloud models, provide seamless integration with existing cloud ecosystems and DevOps workflows. This enables organizations to build and deploy data-driven applications with rapid time-to-market, while benefiting from managed services that handle infrastructure provisioning, maintenance, and security. The convergence of cloud-native development and real-time data streaming is accelerating the adoption of Streaming Database as a Service across sectors such as BFSI, IT & telecommunications, and retail.

Furthermore, advancements in artificial intelligence (AI) and machine learning (ML) are amplifying the value proposition of streaming databases. These platforms are increasingly being leveraged to support intelligent automation, predictive analytics, and anomaly detection in real time. The integration of AI/ML capabilities with streaming databases allows enterprises to identify patterns, trends, and threats as they emerge, enabling proactive decision-making and operational efficiency. As the ecosystem of AI-powered applications expands, the demand for Streaming Database as a Service is expected to witness sustained momentum, particularly in industries with high-frequency and high-volume data streams.

From a regional perspective, North America continues to dominate the Streaming Database as a Service market, accounting for the largest revenue share in 2024. This leadership position is attributed to the strong presence of technology giants, early adoption of cloud-based solutions, and significant investments in digital transformation initiatives. Meanwhile, Asia Pacific is poised for the fastest growth over the forecast period, driven by rapid industrialization, expanding digital infrastructure, and increasing adoption of real-time analytics in emerging economies such as China and India. Europe, Latin America, and the Middle East & Africa are also witnessing growing interest in streaming database solutions, supported by regulatory mandates, data privacy concerns, and the proliferation of smart city projects.

Component Analysis

The component segment of the Streaming Database as a Service market is bifurcated into software and services. Software solutions form the backbone of streaming database platforms, providing the core functionalities for data ingestion, processing, storage, and analytics. These solutions are designed to handle high-velocity data streams, deliver low-latency query performance, and support a variety of data models, including SQL, NoSQL, and NewSQL. The rapid evolution of open-source streaming technologies, such as Apache Kafka, Apache Flink, and Apache Pulsar, has further accelerated innovation in this segment, enabling vendors to deliver feature-rich, scalable, and interoperable database se

Streaming Database Market Outlook

According to our latest research, the global streaming database market size stood at USD 1.9 billion in 2024, demonstrating robust momentum driven by the rising adoption of real-time data processing across industries. The market is projected to grow at a compound annual growth rate (CAGR) of 22.8% from 2025 to 2033, reaching an estimated USD 14.4 billion by 2033. This remarkable expansion is primarily fueled by the increasing need for instantaneous analytics, rapid developments in IoT ecosystems, and the proliferation of digital transformation initiatives across both developed and emerging economies. As per our latest research, the streaming database market is experiencing accelerated growth due to these transformative factors, making it a focal point for technology investments worldwide.

A major growth factor for the streaming database market is the exponential increase in data generation from connected devices, social media platforms, and enterprise applications. Organizations are under mounting pressure to process and analyze vast volumes of data as it is generated, rather than relying on traditional batch processing methods. This demand for real-time data insights is particularly pronounced in sectors such as financial services, telecommunications, and e-commerce, where milliseconds can make a significant difference in decision-making and customer experience. The ability of streaming databases to ingest, process, and analyze data streams on the fly is enabling businesses to respond proactively to market changes, detect anomalies, and unlock new revenue opportunities.

Another critical driver is the surge in adoption of cloud-based solutions and the ongoing shift toward hybrid IT environments. Cloud deployment models offer unparalleled scalability, flexibility, and cost-efficiency, making advanced streaming analytics accessible to organizations of all sizes. Enterprises are increasingly leveraging cloud-native streaming databases to support distributed architectures, microservices, and edge computing scenarios. This trend is further reinforced by the growing popularity of hybrid and multi-cloud strategies, which allow businesses to optimize workloads, enhance data security, and ensure business continuity. As cloud infrastructure matures and becomes more secure, its role in accelerating the deployment and management of streaming databases will continue to grow.

The integration of artificial intelligence (AI) and machine learning (ML) with streaming databases is also propelling market growth. By embedding AI/ML capabilities into streaming data pipelines, organizations can automate complex analytics, detect patterns in real time, and derive actionable insights with minimal latency. This is particularly valuable for applications such as fraud detection, predictive maintenance, and personalized recommendations. The synergy between streaming databases and advanced analytics is enabling enterprises to move beyond traditional reporting, toward intelligent automation and data-driven innovation. As AI and ML technologies evolve, their integration with streaming databases will become a key competitive differentiator.

From a regional perspective, North America continues to dominate the streaming database market, accounting for the largest revenue share in 2024. The region's leadership is underpinned by the presence of major technology vendors, high digital adoption rates, and significant investments in cloud infrastructure. However, Asia Pacific is emerging as the fastest-growing region, driven by rapid digitization, expanding IoT networks, and the proliferation of smart devices. Europe is also witnessing steady growth, supported by regulatory initiatives and the increasing emphasis on data-driven decision-making. Meanwhile, Latin America and the Middle East & Africa are gradually catching up, fueled by investments in telecommunications and financial services. The global landscape is thus characterized by both mature and emerging markets, each contributing to the overall expansion of the streaming database ecosystem.

The evolution of streaming analytics has been a game-changer for businesses looking to harness the power of real-time data. By leveraging streaming analytics, companies can process data as it arrives, allowing for immediate insights and actio

Event Streaming Database Market Outlook

As per our latest research findings, the global Event Streaming Database market size reached USD 1.85 billion in 2024, with a robust CAGR of 22.7% expected from 2025 to 2033. By the end of 2033, the market is forecasted to attain a significant value of USD 12.16 billion. The remarkable growth of the Event Streaming Database market is primarily driven by the increasing demand for real-time data processing and analytics across industries, as organizations seek to harness the power of streaming data for faster and more informed decision-making.

One of the most influential growth factors for the Event Streaming Database market is the rapid adoption of real-time analytics solutions across sectors such as BFSI, retail, healthcare, and manufacturing. Enterprises today generate massive volumes of data from different sources, including IoT devices, customer interactions, and transaction systems. This data needs to be processed and analyzed in real time to enable agile responses to market changes, detect anomalies, and enhance operational efficiency. Event streaming databases, with their ability to ingest, store, and analyze continuous streams of data, have become indispensable for organizations aiming to gain a competitive edge. As digital transformation initiatives accelerate globally, the relevance of event streaming databases in supporting mission-critical applications is only set to rise.

Another key driver fueling the expansion of the Event Streaming Database market is the proliferation of IoT devices and the increasing complexity of IT infrastructures. With billions of interconnected devices generating streams of data every second, traditional data management systems are unable to cope with the velocity and variety of information. Event streaming databases offer the scalability, flexibility, and low-latency performance required to manage and analyze these high-velocity data streams effectively. This capability is particularly crucial for sectors like manufacturing and logistics, where real-time monitoring and predictive analytics are essential for optimizing supply chains, minimizing downtime, and ensuring safety compliance.

Furthermore, the rising emphasis on enhancing customer experiences and personalizing services is propelling enterprises to invest in event streaming technologies. Businesses, especially in retail and e-commerce, are leveraging event streaming databases to capture and analyze customer interactions as they happen, enabling the delivery of targeted promotions, proactive customer support, and seamless omnichannel experiences. The ability to process and act on data in real time is becoming a strategic differentiator, driving further adoption of event streaming databases across diverse industries.

In the realm of data management, the concept of a Real-Time Database has become increasingly pivotal. As organizations strive to process and analyze data instantaneously, real-time databases offer a solution that allows for the immediate updating and querying of data. This capability is essential for businesses that require up-to-the-minute information to make critical decisions. Real-time databases are particularly beneficial in environments where data is constantly changing, such as financial markets or online retail platforms. By enabling continuous data flow and immediate access, these databases support enhanced operational efficiency and responsiveness, which are key to maintaining a competitive edge in today's fast-paced digital landscape.

From a regional perspective, North America currently dominates the Event Streaming Database market, accounting for the largest share owing to the presence of leading technology providers, advanced IT infrastructure, and early adoption of innovative data solutions. However, Asia Pacific is anticipated to witness the fastest growth during the forecast period, fueled by rapid digitalization, expanding internet penetration, and increasing investments in smart cities and Industry 4.0 initiatives. Europe is also emerging as a significant market, driven by stringent data regulations and a growing focus on real-time business intelligence. Latin America and the Middle East & Africa are gradually catching up as organizations in these regions recognize the value of real-time data processing for driving business transformation.
<

This USGS data release is intended to provide a baselayer of information on likely stream crossings throughout the United States. The geopackage provides likely crossings of infrastructure and streams and provides observed information that helps validate modeled crossings and build knowledge about associated conditions through time (e.g. crossing type, crossing condition). Stream crossings were developed by intersecting the 2020 United States Census Bureau Topologically Integrated Geographic Encoding and Referencing (TIGER) U.S. road lines with the National Hydrography Dataset High Resolution flowlines. The current version of this data release specifically focuses on road stream crossings (i.e. TIGER2020 Roads) but is designed to support additions of other crossing types that may be included in future iterations (e.g. rail). In total 6,608,268 crossings are included in the dataset and 496,564 observations from the U.S. Department of Transportation, Federal Highway Administration's 2019 National Bridge Inventory (NBI)are included to help identify crossing types of bridges and culverts. This data release also contains Python code that documents methods of data development.

Event Streaming Database Market Outlook

As per our latest research, the global Event Streaming Database market size reached USD 1.46 billion in 2024, reflecting robust adoption across industries. The market is expected to expand at a compelling CAGR of 21.7% from 2025 to 2033, reaching a forecasted value of USD 10.6 billion by 2033. This remarkable growth is primarily driven by the rising demand for real-time data processing and analytics, which is critical for modern digital enterprises seeking to enhance operational efficiency and customer engagement.

One of the most significant growth factors for the Event Streaming Database market is the exponential increase in data generation from various sources such as IoT devices, social media, and enterprise applications. Organizations are increasingly recognizing the value of processing and analyzing this data in real-time to gain actionable insights and maintain a competitive edge. As industries such as BFSI, retail, and telecommunications transition to data-driven decision-making, the need for event streaming databases that can handle high-velocity data streams has become paramount. These databases enable organizations to monitor, process, and respond to events as they happen, facilitating innovations in fraud detection, personalized marketing, and operational optimization.

Another key driver propelling the growth of the Event Streaming Database market is the rapid digital transformation initiatives undertaken by enterprises worldwide. As businesses shift towards cloud-native architectures and microservices, the demand for scalable and resilient event streaming solutions is surging. The proliferation of cloud computing has further amplified this trend, as organizations seek flexible deployment models that can support both on-premises and cloud-based workloads. Additionally, the integration of artificial intelligence and machine learning with event streaming databases is unlocking new use cases, from predictive analytics to automated decision-making, thereby broadening the market's appeal across diverse industry verticals.

Furthermore, the growing emphasis on enhancing customer experience is fueling the adoption of event streaming databases. Real-time data processing capabilities empower enterprises to deliver personalized services, respond to customer queries instantly, and optimize supply chain operations. In sectors like healthcare and manufacturing, these databases enable real-time monitoring of equipment and patient data, leading to improved outcomes and reduced downtime. As regulatory requirements for data integrity and compliance become more stringent, event streaming databases offer robust solutions to ensure data traceability and security, which is especially crucial for industries handling sensitive information.

From a regional perspective, North America continues to dominate the Event Streaming Database market in 2024, accounting for the largest revenue share due to early technology adoption and the presence of major industry players. However, the Asia Pacific region is emerging as the fastest-growing market, driven by rapid industrialization, increasing digitalization, and substantial investments in IT infrastructure. Europe also holds a significant share, propelled by stringent data regulations and a strong focus on innovation. Meanwhile, Latin America and the Middle East & Africa are witnessing steady growth, supported by expanding digital ecosystems and rising awareness of the benefits of real-time data analytics.

Component Analysis

The Component segment of the Event Streaming Database market is categorized into Software, Hardware, and Services. Software solutions constitute the largest share within this segment, driven by the growing demand for advanced event streaming platforms that enable real-time data ingestion, processing, and analytics. These solutions are critical for organizations aiming to harness the power of continuous data streams, enabling them to detect anomalies, automate responses, and derive actionable insights. The evolution of open-source event streaming platforms and the integration of AI and ML capabilities have further propelled the adoption of software solutions, as enterprises seek to build scalable and intelligent data architectures.

Hardware components, while representing a smaller shar

The NorWeST webpage hosts stream temperature data and climate scenarios in a variety of user-friendly digital formats for streams and rivers across the western U.S. The temperature database was compiled from hundreds of biologists and hydrologists working for >100 resource agencies and contains >200,000,000 hourly temperature recordings at >20,000 unique stream sites. Those temperature data were used with spatial statistical network models to develop 36 historical and future climate scenarios at 1-kilometer resolution for >1,000,000 kilometers of stream. Temperature data and model outputs, registered to NHDPlus stream lines, are posted to the website after QA/QC procedures and development of the final temperature model within a river basin. Open access to the data and the availability of accurate stream temperature scenarios will foster new research and collaborative relationships that enhance management and conservation of aquatic resources. Funding for the project was provided by the GNLCC and NPLCC with additional funds and in-kind support from the USFS, USGS, USFWS, NFWF, California Fish Passage Forum, and NASA. Resources in this dataset:Resource Title: Website Pointer to NorWeST Stream Temperature Regional Database and Model. File Name: Web Page, url: https://www.fs.fed.us/rm/boise/AWAE/projects/NorWeST.html The NorWeST webpage hosts stream temperature data and climate scenarios in a variety of user-friendly digital formats for streams and rivers across the western U.S. The temperature database was compiled from hundreds of biologists and hydrologists working for >100 resource agencies and contains >200,000,000 hourly temperature recordings at >20,000 unique stream sites. Those temperature data were used with spatial statistical network models to develop 36 historical and future climate scenarios at 1-kilometer resolution for >1,000,000 kilometers of stream. Temperature data and model outputs, registered to NHDPlus stream lines, are posted to the website after QA/QC procedures and development of the final temperature model within a river basin.

Relational MS Access database that holds all data from monitoring streams in the Klamath I&M Network. Observations of stream characteristics (physical, chemical, and biological parameters) at Crater Lake NP, Lassen Volcanic NP, Oregon Caves NM, Redwood NSP, and Whiskeytown NRA.

Streaming Database Market Outlook

According to our latest research, the global streaming database market size in 2024 stands at USD 1.85 billion, reflecting robust demand for real-time data processing across industries. The market is projected to grow at a remarkable CAGR of 21.1% from 2025 to 2033, reaching an estimated USD 12.29 billion by the end of the forecast period. This impressive expansion is primarily driven by the increasing need for instant data-driven decision-making, rapid digital transformation, and the proliferation of IoT devices and real-time analytics applications.

One of the primary growth drivers for the streaming database market is the surging adoption of real-time analytics across diverse industry verticals. Organizations today are inundated with massive volumes of data generated from various sources such as online transactions, IoT sensors, social media, and mobile devices. The ability to process, analyze, and act on this data in real-time is becoming a critical differentiator, especially for sectors like BFSI, retail, and telecommunications, where customer experience and operational agility are paramount. Streaming databases enable enterprises to gain actionable insights within milliseconds, thereby supporting use cases such as fraud detection, personalized marketing, and dynamic pricing. As enterprises continue to embrace digital transformation initiatives, the demand for robust, scalable, and high-performance streaming database solutions is expected to accelerate further.

Another significant factor fueling the growth of the streaming database market is the exponential rise in IoT deployments and connected devices. With billions of sensors and devices generating continuous streams of data, traditional batch-processing databases are increasingly inadequate for handling the velocity and volume of information. Streaming databases are purpose-built to ingest, process, and analyze data as it arrives, making them indispensable for IoT applications such as predictive maintenance, real-time monitoring, and smart city solutions. Furthermore, advancements in edge computing and 5G networks are amplifying the need for low-latency data processing, further boosting the adoption of streaming database technologies in both industrial and consumer IoT landscapes.

The evolution of cloud computing is also playing a pivotal role in shaping the streaming database market. Cloud-based streaming databases offer unparalleled scalability, flexibility, and cost-efficiency, enabling organizations to process vast streams of data without the need for significant upfront infrastructure investments. This is particularly attractive for small and medium-sized enterprises (SMEs) and startups that require agile, pay-as-you-go solutions to support real-time analytics and business intelligence initiatives. The growing ecosystem of cloud-native streaming database platforms, coupled with seamless integration capabilities with other cloud services, is expected to drive sustained market growth throughout the forecast period.

From a regional perspective, North America continues to dominate the streaming database market, driven by the presence of leading technology vendors, early adoption of advanced analytics, and strong investments in digital infrastructure. However, the Asia Pacific region is witnessing the fastest growth, fueled by rapid industrialization, expanding internet penetration, and government-led smart city initiatives. Europe is also emerging as a significant market, particularly in sectors such as manufacturing, healthcare, and financial services, where real-time data processing is becoming increasingly critical. Latin America and the Middle East & Africa are gradually catching up, supported by growing digitalization efforts and investments in IT modernization.

Component Analysis

The component segment of the streaming database market is broadly categorized into software, hardware, and services, each playing a crucial role in enabling real-time data processing capabilities for enterprises. The software segment holds the largest market share, primarily due to the continuous innovation in streaming database engines, data integration tools, and advanced analytics platforms. Modern streaming database software is designed to deliver high throughput, low latency, and seamless scalability, catering to the evolving needs of data-driven organizations. Vendors are increasingly focusing on incorpor

This dataset contains the streams derived from the Digital Elevation Model (DEM) for the African continent from the Hydrological Derivatives for Modeling and Analysis (HDMA) database. The HDMA database provides comprehensive and consistent global coverage of raster and vector topographically derived layers, including raster layers of digital elevation model (DEM) data, flow direction, flow accumulation, slope, and compound topographic index (CTI); and vector layers of streams and catchment boundaries. The coverage of the data is global (-180º, 180º, -90º, 90º) with the underlying DEM being a hybrid of three datasets: HydroSHEDS (Hydrological data and maps based on SHuttle Elevation Derivatives at multiple Scales), Global Multi-resolution Terrain Elevation Data 2010 (GMTED2010) and the Shuttle Radar Topography Mission (SRTM). For most of the globe south of 60º North, the raster resolution of the data is 3-arc-seconds, corresponding to the resolution of the SRTM. For the areas North of 60º, the resolution is 7.5-arc-seconds (the smallest resolution of the GMTED2010 dataset) except for Greenland, where the resolution is 30-arc-seconds. The streams and catchments are attributed with Pfafstetter codes, based on a hierarchical numbering system, that carry important topological information.

Fish occurrence data to support high-resolution distribution models and test various community and macroecological hypotheses have not been available at the national scale. We present IchthyMaps, a database of high-quality historical fish occurrences covering fishes of the conterminous United States. Designed on the principles of metacommunity ecology, IchthyMaps is a compilation of presence records from atlases up to 1990, at the resolution of the 1:100,000 National Hydrography Database Plus (NHDPlus) inter-confluence stream segment, readily aggregated into hierarchically coarser units (e.g. hydrologic unit code 8-digit and 12-digit watersheds). IchthyMaps contains about 606,550 presence records for 1,038 species and subspecies. These presence records occurred on 224,305 NHDPlus interconfluence stream segments, representing > 10% average sampling intensity. IchthyMaps is publicly accessible through USGS' ScienceBase infrastructure. It offers unprecedented opportunities f ...

United States agricultural researchers have many options for making their data available online. This dataset aggregates the primary sources of ag-related data and determines where researchers are likely to deposit their agricultural data. These data serve as both a current landscape analysis and also as a baseline for future studies of ag research data. Purpose As sources of agricultural data become more numerous and disparate, and collaboration and open data become more expected if not required, this research provides a landscape inventory of online sources of open agricultural data. An inventory of current agricultural data sharing options will help assess how the Ag Data Commons, a platform for USDA-funded data cataloging and publication, can best support data-intensive and multi-disciplinary research. It will also help agricultural librarians assist their researchers in data management and publication. The goals of this study were to establish where agricultural researchers in the United States-- land grant and USDA researchers, primarily ARS, NRCS, USFS and other agencies -- currently publish their data, including general research data repositories, domain-specific databases, and the top journals compare how much data is in institutional vs. domain-specific vs. federal platforms determine which repositories are recommended by top journals that require or recommend the publication of supporting data ascertain where researchers not affiliated with funding or initiatives possessing a designated open data repository can publish data Approach The National Agricultural Library team focused on Agricultural Research Service (ARS), Natural Resources Conservation Service (NRCS), and United States Forest Service (USFS) style research data, rather than ag economics, statistics, and social sciences data. To find domain-specific, general, institutional, and federal agency repositories and databases that are open to US research submissions and have some amount of ag data, resources including re3data, libguides, and ARS lists were analysed. Primarily environmental or public health databases were not included, but places where ag grantees would publish data were considered. Search methods We first compiled a list of known domain specific USDA / ARS datasets / databases that are represented in the Ag Data Commons, including ARS Image Gallery, ARS Nutrition Databases (sub-components), SoyBase, PeanutBase, National Fungus Collection, i5K Workspace @ NAL, and GRIN. We then searched using search engines such as Bing and Google for non-USDA / federal ag databases, using Boolean variations of “agricultural data” /“ag data” / “scientific data” + NOT + USDA (to filter out the federal / USDA results). Most of these results were domain specific, though some contained a mix of data subjects. We then used search engines such as Bing and Google to find top agricultural university repositories using variations of “agriculture”, “ag data” and “university” to find schools with agriculture programs. Using that list of universities, we searched each university web site to see if their institution had a repository for their unique, independent research data if not apparent in the initial web browser search. We found both ag specific university repositories and general university repositories that housed a portion of agricultural data. Ag specific university repositories are included in the list of domain-specific repositories. Results included Columbia University – International Research Institute for Climate and Society, UC Davis – Cover Crops Database, etc. If a general university repository existed, we determined whether that repository could filter to include only data results after our chosen ag search terms were applied. General university databases that contain ag data included Colorado State University Digital Collections, University of Michigan ICPSR (Inter-university Consortium for Political and Social Research), and University of Minnesota DRUM (Digital Repository of the University of Minnesota). We then split out NCBI (National Center for Biotechnology Information) repositories. Next we searched the internet for open general data repositories using a variety of search engines, and repositories containing a mix of data, journals, books, and other types of records were tested to determine whether that repository could filter for data results after search terms were applied. General subject data repositories include Figshare, Open Science Framework, PANGEA, Protein Data Bank, and Zenodo. Finally, we compared scholarly journal suggestions for data repositories against our list to fill in any missing repositories that might contain agricultural data. Extensive lists of journals were compiled, in which USDA published in 2012 and 2016, combining search results in ARIS, Scopus, and the Forest Service's TreeSearch, plus the USDA web sites Economic Research Service (ERS), National Agricultural Statistics Service (NASS), Natural Resources and Conservation Service (NRCS), Food and Nutrition Service (FNS), Rural Development (RD), and Agricultural Marketing Service (AMS). The top 50 journals' author instructions were consulted to see if they (a) ask or require submitters to provide supplemental data, or (b) require submitters to submit data to open repositories. Data are provided for Journals based on a 2012 and 2016 study of where USDA employees publish their research studies, ranked by number of articles, including 2015/2016 Impact Factor, Author guidelines, Supplemental Data?, Supplemental Data reviewed?, Open Data (Supplemental or in Repository) Required? and Recommended data repositories, as provided in the online author guidelines for each the top 50 journals. Evaluation We ran a series of searches on all resulting general subject databases with the designated search terms. From the results, we noted the total number of datasets in the repository, type of resource searched (datasets, data, images, components, etc.), percentage of the total database that each term comprised, any dataset with a search term that comprised at least 1% and 5% of the total collection, and any search term that returned greater than 100 and greater than 500 results. We compared domain-specific databases and repositories based on parent organization, type of institution, and whether data submissions were dependent on conditions such as funding or affiliation of some kind. Results A summary of the major findings from our data review: Over half of the top 50 ag-related journals from our profile require or encourage open data for their published authors. There are few general repositories that are both large AND contain a significant portion of ag data in their collection. GBIF (Global Biodiversity Information Facility), ICPSR, and ORNL DAAC were among those that had over 500 datasets returned with at least one ag search term and had that result comprise at least 5% of the total collection. Not even one quarter of the domain-specific repositories and datasets reviewed allow open submission by any researcher regardless of funding or affiliation. See included README file for descriptions of each individual data file in this dataset. Resources in this dataset:Resource Title: Journals. File Name: Journals.csvResource Title: Journals - Recommended repositories. File Name: Repos_from_journals.csvResource Title: TDWG presentation. File Name: TDWG_Presentation.pptxResource Title: Domain Specific ag data sources. File Name: domain_specific_ag_databases.csvResource Title: Data Dictionary for Ag Data Repository Inventory. File Name: Ag_Data_Repo_DD.csvResource Title: General repositories containing ag data. File Name: general_repos_1.csvResource Title: README and file inventory. File Name: README_InventoryPublicDBandREepAgData.txt

These data are collected as part of the Fisheries and Oceans Canada (DFO) Pacific Science Enterprise Centre (PSEC) Community Stream Monitoring (CoSMo) project, which is a collaborative monitoring initiative that strives to produce quality, long-term datasets for use in resource management, research, and stewardship. We currently monitor streams in southwest BC in the region spanning from Howe Sound, south to the USA border, and east to Abbotsford, primarily using automated dataloggers. This project is made possible thanks to the many stewardship groups in the region whose dedicated volunteers are committed to protecting, conserving, and educating the public about their local streams.

We rely on volunteers to download the data from the dataloggers. If you download and/or use the CoSMo data, we would appreciate if you could send a quick email to Nikki Kroetsch (Nikki.Kroetsch@dfo-mpo.gc.ca) with a brief (1-2 sentence) description of what you will be using the data for. Though not a requirement for using the data, this information reassures our volunteers that their data collection efforts are appreciated and worthwhile, which motivates them to continue to help collect the data.

Partner Organizations: Alouette River Management Society, Cariboo Heights Forest Preservation Society, Cougar Creek Streamkeepers, City of Surrey, Bowen Island Fish and Wildlife Club, West Vancouver Streamkeepers, Stoney Creek Environment Committee, City of Port Moody, Capilano Golf/Country Club, Eagle Creek Streamkeepers, North Shore Streamkeepers, Nicomekl Enhancement Society, Hoy/Scott Watershed Society, Hyde Creek Watershed Society, WaterWealth Project, Univ. of BC, Burrard Inlet Marine Enhancement Society, Yorkson Watershed Enhancement Society, Johnston Heights Secondary School (Surrey), Seymour Salmonid Society, Still Creek Streamkeepers, PSEC staff

Dedicated volunteers (not associated with an organization) who steward Carlson Creek, Mosquito Creek, McDonald Creek, and McNally Creek.

The U.S. Geological Survey (USGS), in cooperation with the Illinois Center for Transportation and the Illinois Department of Transportation, prepared hydro-conditioned geographic information systems (GIS) layers for use in the Illinois StreamStats application. These data were used to delineate drainage basins and compute basin characteristics for updated peak flow and flow duration regression equations for Illinois. This dataset consists of raster grid files for elevation (dem), flow accumulation (fac), flow direction (fdr), and stream definition (str900) for each 8-digit Hydrologic Unit Code (HUC) area in Illinois merged into a single dataset. There are 51 full or partial HUC 8s represented by this data set: 04040002, 05120108, 05120109, 05120111, 05120112, 05120113, 05120114, 05120115, 05140202, 05140203, 05140204, 05140206, 07060005, 07080101, 07080104, 07090001, 07090002, 07090003, 07090004, 07090005, 07090006, 07090007, 07110001, 07110004, 07110009, 07120001, 07120002, 07120004 (0712003 was combined into this HUC), 07120005, 07120006, 07120007, 07130001, 07130002, 07130003, 07130004, 07130005, 07130006, 07130007, 07130008, 07130009, 07130010, 07130011, 07130012, 07140101, 07140105, 07140106, 07140108, 07140201, 07140202, 07140203, and 07140204.

This layer represents modeled stream temperatures derived from the NorWeST point feature class (NorWest_TemperaturePoints). NorWeST summer stream temperature scenarios were developed for all rivers and streams in the western U.S. from the > 20,000 stream sites in the NorWeST database where mean August stream temperatures were recorded. The resulting dataset includes stream lines (NorWeST_PredictedStreams) and associated mid-points NorWest_TemperaturePoints) representing 1 kilometer intervals along the stream network. Stream lines were derived from the 1:100,000 scale NHDPlus dataset (USEPA and USGS 2010; McKay et al. 2012). Shapefile extents correspond to NorWeST processing units, which generally relate to 6 digit (3rd code) hydrologic unit codes (HUCs) or in some instances closely correspond to state borders. The line and point shapefiles contain identical modeled stream temperature results. The two feature classes are meant to complement one another for use in different applications. In addition, spatial and temporal covariates used to generate the modeled temperatures are included in the attribute tables at https://www.fs.usda.gov/rm/boise/AWAE/projects/NorWeST/ModeledStreamTemperatureScenarioMaps.shtml. The NorWeST NHDPlusV1 processing units include: Salmon, Clearwater, Spokoot, Missouri Headwaters, Snake-Bear, MidSnake, MidColumbia, Oregon Coast, South-Central Oregon, Upper Columbia-Yakima, Washington Coast, Upper Yellowstone-Bighorn, Upper Missouri-Marias, and Upper Green-North Platte. The NorWeST NHDPlusV2 processing units include: Lahontan Basin, Northern California-Coastal Klamath, Utah, Coastal California, Central California, Colorado, New Mexico, Arizona, and Black Hills.

The rate at which new information about stream resources is being created has accelerated with the recent development of spatial stream-network models (SSNMs), the growing availability of stream databases, and ongoing advances in geospatial science and computational efficiency. To further enhance information development, the National Stream Internet (NSI) project was developed as a means of providing a consistent, flexible analytical infrastructure that can be applied with many types of stream data anywhere in the country. A key part of that infrastructure is the NSI network, a digital GIS layer which has a specific topological structure that was designed to work effectively with SSNMs. The NSI network was derived from the National Hydrography Dataset Plus, Version 2 (NHDPlusV2) following technical procedures that ensure compatibility with SSNMs. The SSN models outperform traditional statistical techniques applied to stream data, enable predictions at unsampled locations to create status maps for river networks, and work particularly well with databases aggregated from multiple sources that contain clustered sampling locations. The NSI project is funded by the U.S. Fish & Wildlife Service's Landscape Conservation Cooperative program and has two simple objectives: 1) refine key spatial and statistical stream software and digital databases for compatibility so that a nationally consistent analytical infrastructure exists and is easy to apply; and 2) engage a grassroots user-base in application of this infrastructure so they are empowered to create new and valuable information from stream databases anywhere in the country. This website is a hub designed to connect users with software, data, and tools for creating that information. As better information is developed, it should enable stronger science, management, and conservation as pertains to stream ecosystems. Resources in this dataset:Resource Title: Website Pointer to the National Stream Internet. File Name: Web Page, url: https://www.fs.fed.us/rm/boise/AWAE/projects/NationalStreamInternet.html The National Stream Internet (NSI) is a network of people, data, and analytical techniques that interact synergistically to create information about streams. Elements and tools composing the NSI, including STARS, NHDPlusV2, and SSNs, enable integration of existing databases (e.g., water quality parameters, biological surveys, habitat condition) and development of new information using sophisticated spatial-statistical network models (SSNMs). The NSI provides a nationally consistent framework for analysis of stream data that can greatly improve the accuracy of status and trend assessments. The NSI project is described, together with an analytical infrastructure for using the spatial statistical network models with many types of stream datasets.

Fish kills have become a focus of public attention as more interest is placed on the quality and condition of Iowa's streams and rivers. The Integrated Report, which combines federal requirements for state Section 305(b) water quality assessments and Section 303(d) impaired waters listings, required the Iowa Department of Natural Resources Watershed Monitoring and Assessment Section to begin tracking fish kills. A fish kill can affect the 305(b) water quality assessment of the waterbody and can potentially result in the addition of the water body to the 303(d) list of impaired waters. The online database stores the data for all fish kills in the state from 1995 to the present.

This resource references a large temperature database developed for the western U.S. that contains records for >23,000 unique stream and river sites and consists of data contributed by hundreds of professionals working for dozens of natural resource agencies. All the records have been QA/QC’d and linked to the National Hydrography Dataset and fully documented with metadata for easy use. The website describing the NorWeST project and serving the data is here (https://www.fs.fed.us/rm/boise/AWAE/projects/NorWeST.html). The publication, The NorWeST Summer Stream Temperature Model and Scenarios for the Western U.S.: A Crowd‐Sourced Database and New Geospatial Tools Foster a User Community and Predict Broad Climate Warming of Rivers and Streams can be found here: https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/2017WR020969

Swedish monitoring data obtained for the Swedish Environmental Protection Agency (Naturvårdsverket; national data) and Sweden's counties (Länsstyrelsen; regional data). The data are available on-line at http://www.slu.se/vatten-miljo(downloading of data only in Swedish, otherwise some information on the webstie is available in English) and are stored in separate databases including one on water chemistry (Databank för vattenkemi) and Phytoplankton (växtplankton), benthic algae (Påväxtalger (Bentiska kiselalger), zooplankton (djurplankton), macroinvertebrates (bottenfauna). The monitoring is divided into frequent monitoring of water chemistry, phytoplankton and zooplankton (once a month during the open water season, referred to as Trendstationer). Monthly data during the open water season are also avaialable for Sweden's largest lakes Vänern, Vättern and Mälaren. Macroinvertebrates are only sampled once a year. The monitoring of benthic algae is seperated from the other monitoring but is currently coordinated with the phytoplankton database. There is also a large database on water chemistry from several thousands of lakes, sampled once a year (omdrevssjöar). Some data are available also for sediments and vegetation. More information on this dataset can be found in the Freshwater Metadatabase - BFE_7 (http://www.freshwatermetadata.eu/metadb/bf_mdb_view.php?entryID=BFE_7).

These four NetCDF databases constitute the bulk of the spatial and spatiotemporal environmental covariates used in a latent health factor index (LHFI) model for assessment and prediction of ecosystem health across the MDB. The data formatting and hierarchical statistical modelling were conducted under a CSIRO appropriation project funded by the Water for a Healthy Country Flagship from July 2012 to June 2014. Each database was created by collating and aligning raw data downloaded from the respective state government websites (QLD, NSW, VIC, and SA). (ACT data were unavailable.) There are two primary components in each state-specific database: (1) a temporally static data matrix with axes "Site ID" and "Variable," and (2) a 3D data cube with axes "Site ID", "Variable," and "Date." Temporally static variables in (1) include geospatial metadata (all states), drainage area (VIC and SA only), and stream distance (SA only). Temporal variables in (2) include discharge, water temperature, etc. Missing data (empty cells) are highly abundant in the data cubes. The attached state-specific README.pdf files contain additional details on the contents of these databases, and any computer code that was used for semi-automation of raw data downloads. Lineage: (1) For NSW I created the NetCDF database by (a) downloading CSV raw data from the NSW Office of Water real-time data website (http://realtimedata.water.nsw.gov.au/water.stm) during February-April 2013, then (b) writing computer programs to preprocess such raw data into the current format. (2) The same was done for QLD, except through the Queensland Water Monitoring Data Portal (http://watermonitoring.derm.qld.gov.au/host.htm). (3) The same was also done for SA, except through the SA WaterConnect => Data Systems => Surface Water Data website (https://www.waterconnect.sa.gov.au/Systems/SWD/SitePages/Home.aspx) during April 2013 as well as May 2014. (4) For Victoria I created the NetCDF database by (a) manually downloading XLS raw data during November and December in 2013 from the Victoria DEPI Water Measurement Information System => Download Rivers and Streams sites website (http://data.water.vic.gov.au/monitoring.htm), then (b) writing computer programs to preprocess such raw data into CSV format (intermediate), then into the current final format.

Additional details on lineage are available from the attached README.pdf files.

This dataset compiles heat flow and temperature gradient data from over 44,000 wells across the United States, along with more than 6,000 related geothermal exploration resources. Originally assembled prior to 2014 for the now-retired National Geothermal Data System (NGDS), the collection includes curated well data, scanned field notes, temperature-depth curves, publications, maps, and other supporting documents. SMU Geothermal Laboratory contributed two different nationwide heat flow databases to the project. One is based on equilibrium temperature measurements (over 14,000 sites) and the other is based on corrected bottom hole temperature (BHT) data from oil and gas industry wells (over 30,000 sites). In addition, scanned field notes and temperature-depth curves were associated with approximately 6,000 specific sites in the heat flow database. Records were corrected and overlapping sites in the equilibrium heat flow database were linked between the original SMU National database and the UND Global Heat Flow database. New or related sites, which were not previously published because they lacked full heat flow content, are now included as gradient only information along with their detailed temperature data to fill in data gaps. Finally, SMU submitted over 920 scanned publications, reports, and maps suitable for full text searching. The dataset is provided in two flat-structured zip archives: one containing the curated well data and another containing related resources. An Excel index file is provided for each archive, allowing filtering by well name, location, and description. Data files are labeled with state or institutional origin where available.