The statistic shows the problems caused by poor quality data for enterprises in North America, according to a survey of North American IT executives conducted by 451 Research in 2015. As of 2015, 44 percent of respondents indicated that having poor quality data can result in extra costs for the business.

Recent developments include: January 2022: IBM and Francisco Partners disclosed the execution of a definitive contract under which Francisco Partners will purchase medical care information and analytics resources from IBM, which are currently part of the IBM Watson Health business., October 2021: Informatica LLC announced an important cloud storage agreement with Google Cloud in October 2021. This collaboration allows Informatica clients to transition to Google Cloud as much as twelve times quicker. Informatica's Google Cloud Marketplace transactable solutions now incorporate Master Data Administration and Data Governance capabilities., Completing a unit of labor with incorrect data costs ten times more estimates than the Harvard Business Review, and finding the correct data for effective tools has never been difficult. A reliable system may be implemented by selecting and deploying intelligent workflow-driven, self-service options tools for data quality with inbuilt quality controls.. Key drivers for this market are: Increasing demand for data quality: Businesses are increasingly recognizing the importance of data quality for decision-making and operational efficiency. This is driving demand for data quality tools that can automate and streamline the data cleansing and validation process.

Growing adoption of cloud-based data quality tools: Cloud-based data quality tools offer several advantages over on-premises solutions, including scalability, flexibility, and cost-effectiveness. This is driving the adoption of cloud-based data quality tools across all industries.

Emergence of AI-powered data quality tools: AI-powered data quality tools can automate many of the tasks involved in data cleansing and validation, making it easier and faster to achieve high-quality data. This is driving the adoption of AI-powered data quality tools across all industries.. Potential restraints include: Data privacy and security concerns: Data privacy and security regulations are becoming increasingly stringent, which can make it difficult for businesses to implement data quality initiatives.

Lack of skilled professionals: There is a shortage of skilled data quality professionals who can implement and manage data quality tools. This can make it difficult for businesses to achieve high-quality data.

Cost of data quality tools: Data quality tools can be expensive, especially for large businesses with complex data environments. This can make it difficult for businesses to justify the investment in data quality tools.. Notable trends are: Adoption of AI-powered data quality tools: AI-powered data quality tools are becoming increasingly popular, as they can automate many of the tasks involved in data cleansing and validation. This makes it easier and faster to achieve high-quality data.

Growth of cloud-based data quality tools: Cloud-based data quality tools are becoming increasingly popular, as they offer several advantages over on-premises solutions, including scalability, flexibility, and cost-effectiveness.

Focus on data privacy and security: Data quality tools are increasingly being used to help businesses comply with data privacy and security regulations. This is driving the development of new data quality tools that can help businesses protect their data..

This dataset includes bibliographic information for 501 papers that were published from 2010-April 2017 (time of search) and use online biodiversity databases for research purposes. Our overarching goal in this study is to determine how research uses of biodiversity data developed during a time of unprecedented growth of online data resources. We also determine uses with the highest number of citations, how online occurrence data are linked to other data types, and if/how data quality is addressed. Specifically, we address the following questions:

1.) What primary biodiversity databases have been cited in published research, and which

databases have been cited most often?

2.) Is the biodiversity research community citing databases appropriately, and are

the cited databases currently accessible online?

3.) What are the most common uses, general taxa addressed, and data linkages, and how

have they changed over time?

4.) What uses have the highest impact, as measured through the mean number of citations

per year?

5.) Are certain uses applied more often for plants/invertebrates/vertebrates?

6.) Are links to specific data types associated more often with particular uses?

7.) How often are major data quality issues addressed?

8.) What data quality issues tend to be addressed for the top uses?

Relevant papers for this analysis include those that use online and openly accessible primary occurrence records, or those that add data to an online database. Google Scholar (GS) provides full-text indexing, which was important to identify data sources that often appear buried in the methods section of a paper. Our search was therefore restricted to GS. All authors discussed and agreed upon representative search terms, which were relatively broad to capture a variety of databases hosting primary occurrence records. The terms included: “species occurrence” database (8,800 results), “natural history collection” database (634 results), herbarium database (16,500 results), “biodiversity database” (3,350 results), “primary biodiversity data” database (483 results), “museum collection” database (4,480 results), “digital accessible information” database (10 results), and “digital accessible knowledge” database (52 results)--note that quotations are used as part of the search terms where specific phrases are needed in whole. We downloaded all records returned by each search (or the first 500 if there were more) into a Zotero reference management database. About one third of the 2500 papers in the final dataset were relevant. Three of the authors with specialized knowledge of the field characterized relevant papers using a standardized tagging protocol based on a series of key topics of interest. We developed a list of potential tags and descriptions for each topic, including: database(s) used, database accessibility, scale of study, region of study, taxa addressed, research use of data, other data types linked to species occurrence data, data quality issues addressed, authors, institutions, and funding sources. Each tagged paper was thoroughly checked by a second tagger.

The final dataset of tagged papers allow us to quantify general areas of research made possible by the expansion of online species occurrence databases, and trends over time. Analyses of this data will be published in a separate quantitative review.

The Florida State University Center for Ocean-Atmospheric Predictions Studies (COAPS) has been operating a data assembly center (DAC) to collect, quality evaluate, and distribute Shipboard Automated Meteorological and Oceanographic System (SAMOS) observations since 2005. A SAMOS is typically a computerized data logging system that records underway meteorological and near-surface oceanographic observations collected on research vessels. The SAMOS initiative does not provide specific instrumentation for vessels, but instead takes advantage of science quality instrumentation already deployed on research vessels and select merchant ships. The SAMOS initiative provides vessel operators with desired sampling protocols and metadata requirements that will ensure the DAC receives a consistent series of observations from each vessel. The DAC and its partners in U. S. National Oceanic and Atmospheric Administration (NOAA), the University National Oceanographic Laboratory System, the U. S. Coast Guard, and the U. S. Antarctic Program have implemented a series of daily data transmissions from ship-to-shore using an email protocol. A set of observations recorded at one-minute intervals for the previous day arrive at the DAC soon after 0000 UTC and undergo automated quality evaluation. A trained data analyst reviews data and responds directly to vessels at sea when problems are identified. A secondary level of visual quality control is completed after all data from a single ship and day are merged into a common daily file (allowing for delayed data receipts). All quality-evaluated data are freely available to the user community and are distributed to national archive centers. This dataset contains all of these data.

Online imbalanced learning is an emerging topic that combines the challenges of class imbalance and concept drift. However, current works account for issues of class imbalance and concept drift. And only few works have considered these issues simultaneously. To this end, this paper proposes an entropy-based dynamic ensemble classification algorithm (EDAC) to consider data streams with class imbalance and concept drift simultaneously. First, to address the problem of imbalanced learning in training data chunks arriving at different times, EDAC adopts an entropy-based balanced strategy. It divides the data chunks into multiple balanced sample pairs based on the differences in the information entropy between classes in the sample data chunk. Additionally, we propose a density-based sampling method to improve the accuracy of classifying minority class samples into high quality samples and common samples via the density of similar samples. In this manner high quality and common samples are randomly selected for training the classifier. Finally, to solve the issue of concept drift, EDAC designs and implements an ensemble classifier that uses a self-feedback strategy to determine the initial weight of the classifier by adjusting the weight of the sub-classifier according to the performance on the arrived data chunks. The experimental results demonstrate that EDAC outperforms five state-of-the-art algorithms considering four synthetic and one real-world data streams.

The Water Quality Portal (WQP) is a cooperative service sponsored by the United States Geological Survey (USGS), the Environmental Protection Agency (EPA), and the National Water Quality Monitoring Council (NWQMC). It serves data collected by over 400 state, federal, tribal, and local agencies. Water quality data can be downloaded in Excel, CSV, TSV, and KML formats. Fourteen site types are found in the WQP: aggregate groundwater use, aggregate surface water use, atmosphere, estuary, facility, glacier, lake, land, ocean, spring, stream, subsurface, well, and wetland. Water quality characteristic groups include physical conditions, chemical and bacteriological water analyses, chemical analyses of fish tissue, taxon abundance data, toxicity data, habitat assessment scores, and biological index scores, among others. Within these groups, thousands of water quality variables registered in the EPA Substance Registry Service (https://iaspub.epa.gov/sor_internet/registry/substreg/home/overview/home.do) and the Integrated Taxonomic Information System (https://www.itis.gov/) are represented. Across all site types, physical characteristics (e.g., temperature and water level) are the most common water quality result type in the system. The Water Quality Exchange data model (WQX; http://www.exchangenetwork.net/data-exchange/wqx/), initially developed by the Environmental Information Exchange Network, was adapted by EPA to support submission of water quality records to the EPA STORET Data Warehouse [USEPA, 2016], and has subsequently become the standard data model for the WQP. Contributing organizations: ACWI The Advisory Committee on Water Information (ACWI) represents the interests of water information users and professionals in advising the federal government on federal water information programs and their effectiveness in meeting the nation's water information needs. ARS The Agricultural Research Service (ARS) is the U.S. Department of Agriculture's chief in-house scientific research agency, whose job is finding solutions to agricultural problems that affect Americans every day, from field to table. ARS conducts research to develop and transfer solutions to agricultural problems of high national priority and provide information access and dissemination to, among other topics, enhance the natural resource base and the environment. Water quality data from STEWARDS, the primary database for the USDA/ARS Conservation Effects Assessment Project (CEAP) are ingested into WQP via a web service. EPA The Environmental Protection Agency (EPA) gathers and distributes water quality monitoring data collected by states, tribes, watershed groups, other federal agencies, volunteer groups, and universities through the Water Quality Exchange framework in the STORET Warehouse. NWQMC The National Water Quality Monitoring Council (NWQMC) provides a national forum for coordination of comparable and scientifically defensible methods and strategies to improve water quality monitoring, assessment, and reporting. It also promotes partnerships to foster collaboration, advance the science, and improve management within all elements of the water quality monitoring community. USGS The United States Geological Survey (USGS) investigates the occurrence, quantity, quality, distribution, and movement of surface waters and ground waters and disseminates the data to the public, state, and local governments, public and private utilities, and other federal agencies involved with managing the United States' water resources. Resources in this dataset:Resource Title: Website Pointer for Water Quality Portal. File Name: Web Page, url: https://www.waterqualitydata.us/ The Water Quality Portal (WQP) is a cooperative service sponsored by the United States Geological Survey (USGS), the Environmental Protection Agency (EPA), and the National Water Quality Monitoring Council (NWQMC). It serves data collected by over 400 state, federal, tribal, and local agencies. Links to Download Data, User Guide, Contributing Organizations, National coverage by state.

Automated Weather Station and AWS-like networks are the primary source of surface-level meteorological data in remote polar regions. These networks have developed organically and independently, and deliver data to researchers in idiosyncratic ASCII formats that hinder automated processing and intercomparison among networks. Moreover, station tilt causes significant biases in polar AWS measurements of radiation and wind direction. Researchers, network operators, and data centers would benefit from AWS-like data in a common format, amenable to automated analysis, and adjusted for known biases. This project addresses these needs by developing a scientific software workflow called "Justified AWS" (JAWS) to ingest Level 2 (L2) data in the multiple formats now distributed, harmonize it into a common format, and deliver value-added Level 3 (L3) output suitable for distribution by the network operator, analysis by the researcher, and curation by the data center. Polar climate researchers currently face daunting problems including how to easily: 1. Automate analysis (subsetting, statistics, unit conversion) of AWS-like L2 ASCII data. 2. Combine or intercompare data and data quality from among unharmonized L2 datasets. 3. Adjust L2 data for biases such as AWS tilt angle and direction. JAWS addresses these common issues by harmonizing AWS L2 data into a common format, and applying accepted methods to quantify quality and estimate biases. Specifically, JAWS enables users and network operators to 1. Convert L2 data (usually ASCII tables) into a netCDF-based L3 format compliant with metadata conventions (Climate-Forecast and ACDD) that promote automated discovery and analysis. 2. Include value-added L3 features like the Retrospective, Iterative, Geometry-Based (RIGB) tilt angle and direction corrections, solar angles, and standardized quality flags. 3. Provide a scriptable API to extend the initial L2-to-L3 conversion to newer AWS-like networks and instruments. Polar AWS network experts and NSIDC DAAC personnel, each with decades of experience, will help guide and deliberate the L3 conventions implemented in Stages 2-3. The project will start on July 1, 2017 at entry Technology Readiness Level 3 and will exit on June 30, 2019 at TRL 6. JAWS is now a heterogeneous collection of scripts and methods developed and validated at UCI over the past 15 years. At exit, JAWS will comprise three modular stages written in or wrapped by Python, installable by Conda: Stage 1 ingests and translates L2 data into netCDF. Stage 2 annotates the netCDF with CF and ACDD metadata. Stage 3 derives value-added scientific and quality information. The labor-intensive tasks include turning our heterogeneous workflow into a robust, standards-compliant, extensible workflow with an API based on best practices of modern scientific information systems and services. Implementation of Stages 1-2 may be straightforward though tedious due to the menagerie of L2 formats, instruments, and assumptions. The RIGB component of Stage 3 requires ongoing assimilation of ancillary NASA data (CERES, AIRS) and use of automated data transfer protocols (DAP, THREDDS). The immediate target recipient elements are polar AWS network managers, users, and data distributors. L2 borehole data suffers from similar interoperability issues, as does non-polar AWS data. Hence our L3 format will be extensible to global AWS and permafrost networks. JAWS will increase in situ data accessibility and utility, and enable new derived products (both are AIST goals). The PI is a long-standing researcher, open source software developer, and educator who understands obstacles to harmonizing disparate datasets with NASA interoperability recommendations. Our team participates in relevant geoscience communities, including ESDS working groups, ESIP, AGU, and EarthCube.

ARM's scanning cloud radars are fully coherent dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (WSACR; 1.7 kW peak power) or X-band (XSACR; 20 kW peak power). At ARM's tropical sites, X-band radars are paired with the Ka-band because they are better suited for the atmospheric attenuation in this region. Beamwidth for the X-band is approximately 1 degree, and the Ka-band beamwidth is roughly 0.3 degrees. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the XSACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. For the second ARM Mobile Facility (AMF2), the XSACR can be dismounted from the pedestal it shares with the KASACR and mounted on a separate pedestal. This allows the XSACR to operate more like a weather (precipitation) radar in deployments where such local coverage is lacking. Measurements collected with the XSACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, differentialmore » Reflectivity (Zdr), correlation coefficient (rho-hv), and specific differential phase (phi-dp). XSACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.« less

The scientific community has entered an era of big data. However, with big data comes big responsibilities, and best practices for how data are contributed to databases have not kept pace with the collection, aggregation, and analysis of big data. Here, we rigorously assess the quantity of data for specific leaf area (SLA) available within the largest and most frequently used global plant trait database, the TRY Plant Trait Database, exploring how much of the data were applicable (i.e., original, representative, logical, and comparable) and traceable (i.e., published, cited, and consistent). Over three-quarters of the SLA data in TRY either lacked applicability or traceability, leaving only 22.9% of the original data usable compared to the 64.9% typically deemed usable by standard data cleaning protocols. The remaining usable data differed markedly from the original for many species, which led to altered interpretation of ecological analyses. Though the data we consider here make up only 4.5% of SLA data within TRY, similar issues of applicability and traceability likely apply to SLA data for other species as well as other commonly measured, uploaded, and downloaded plant traits. We end with suggested steps forward for global ecological databases, including suggestions for both uploaders to and curators of databases with the hope that, through addressing the issues raised here, we can increase data quality and integrity within the ecological community.

The World Council on City Data (WCCD) awarded the City of Melbourne a platinum designation for its compliance with ISO 37120 (http://www.iso.org/iso/catalogue_detail?csnumber=62436), the world’s first international standard for city indicators. Reporting to the standard allows cities to compare their service delivery and quality of life to other cities globally. The City of Melbourne was one on 20 cities to, globally to help pilot this program and is one of sixteen cities to receive the highest level of accreditation (platinum). \r
Having an international standard methodology to measure city performance allows the City of Melbourne to share data about practices in service delivery, learn from other global cities, rank its results relative to those cities, and address common challenges through more informed decision making. \r
Indicators include: Fire and emergency response; Governance; Health; Recreation; Safety; Shelter; Solid Waste; Telecommunications and Innovation; Transportation; Urban Planning; Wastewater; Water and Sanitation; Economy; Education; Energy; Environment; and Finance.\r
City of Melbourne also submitted an application for accreditation, on behalf of ‘Greater Melbourne’, to the World Council on City Data and this resulted in an ‘Aspirational’ accreditation awarded to wider Melbourne. \r
A summary of Melbourne's results is available here (http://open.dataforcities.org/). Visit the World Council on City Data’s Open Data Portal to compare our results to other cities from around the world.

ARM's scanning cloud radars are fully coherent dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (WSACR; 1.7 kW peak power) or X-band (XSACR; 20 kW peak power). At ARM's tropical sites, X-band radars are paired with the Ka-band because they are better suited for the atmospheric attenuation in this region. Beamwidth for the X-band is approximately 1 degree, and the Ka-band beamwidth is roughly 0.3 degrees. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the XSACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. For the second ARM Mobile Facility (AMF2), the XSACR can be dismounted from the pedestal it shares with the KASACR and mounted on a separate pedestal. This allows the XSACR to operate more like a weather (precipitation) radar in deployments where such local coverage is lacking. Measurements collected with the XSACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, differentialmore » Reflectivity (Zdr), correlation coefficient (rho-hv), and specific differential phase (phi-dp). XSACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.« less

Description While at sea, Commonwealth fishers must record all catch and effort in their AFMA daily fishing logbooks. This dataset shows the annual fishing effort by fishery and gear, that fishers …Show full description##Description While at sea, Commonwealth fishers must record all catch and effort in their AFMA daily fishing logbooks. This dataset shows the annual fishing effort by fishery and gear, that fishers have recorded in their logbooks for Commonwealth fisheries managed by the Australian Fisheries Management Authority (AFMA). Location of effort within a fishery is not included. These data are sourced from AFMA daily fishing logbooks, which fishers complete while at sea. Location of effort within a fishery is not included. Fishing effort is measured differently depending on the fishing method used. In this dataset we provide the most common measures that are used by scientists. ##Appropriate use and limitations of the data The effort data in logbooks are estimates only, which have been recorded by fishers while at sea. Effort data are used in combination with catch data, spatial data, and other relevant information (such as independent surveys) to inform stock assessments. These data may differ from other summations over the same time-period (such as stock assessment reports) due to differences in data handling procedures. For example, scientists may standardise effort for assessments to account for issues such as changes in gear efficiency, catchability of fish and skipper expertise. They may also filter out effort from their assessments because of data quality issues. Care should be taken when interpreting trends in this data because there are many factors that can affect effort each year. Catch quantities can be impacted by management measures such as closures of fishing areas or reducing boat numbers. Similarly, changes in market demand can change fishing decisions and affect effort. ##Disclaimer The dataset provided by AFMA is comprised of raw (unprocessed) data as received from fishers, and may contain errors or be incomplete. AFMA makes no warranty or representation that the data is accurate or complete. Those who choose to use this data should make their own enquiries as to its accuracy and completeness and AFMA assumes no liability for any errors or omissions in the data provided, or for any decision by a person who chooses to rely on the data.

There are over 400 service requests types that are reported in the 311 system that affect the quality of life of our citizens, neighborhoods, and communities. The most popular service requests include but are not limited to animal services requests, high weeds, junk motor vehicles, and a number of other code compliance-related issues. Requests that deal with streets and mobility such as street and pot hole repair are also very common. 311 also receives requests to address environmental issues such as water conservation and air quality complaints. This dataset represents all Service Request from October 1, 2018 to present.

The Ka-Band Scanning ARM Cloud Radar (KASACR) records cloud properties.

ARM's scanning cloud radars are dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (1.7 kW peak power) or an X-band (20 kW peak power).

Beamwidths for Ka-bands paired with W-bands are roughly matched at 0.3 degrees. The X-band beamwidth is approximately 1 degree. Due to the narrow antenna beamwidth, ARM’s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the KASACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. Measurements collected with the KASACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, and linear depolarization ratio.

KASACR data from the 2018–2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina, the 2019–2020 Cold-Air Outbreaks in the Marine Boundary Layer Experiment (COMBLE) in Norway, and the 2021–2022 TRacking Aerosol Convection interactions ExpeRiment (TRACER) in the Houston, Texas, area are available as b1-level products. Building on the original operational data from these campaigns, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaigns, and data quality/ground clutter masks and basic derived products are incorporated. For more information, read the b1-level processing reports for CACTI, COMBLE, and TRACER.

BackgroundThe use of routinely collected health data for secondary research purposes is increasingly recognised as a methodology that advances medical research, improves patient outcomes, and guides policy. This secondary data, as found in electronic medical records (EMRs), can be optimised through conversion into a uniform data structure to enable analysis alongside other comparable health metric datasets. This can be achieved with the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM), which employs a standardised vocabulary to facilitate systematic analysis across various observational databases. The concept behind the OMOP-CDM is the conversion of data into a common format through the harmonisation of terminologies, vocabularies, and coding schemes within a unique repository. The OMOP model enhances research capacity through the development of shared analytic and prediction techniques; pharmacovigilance for the active surveillance of drug safety; and ‘validation’ analyses across multiple institutions across Australia, the United States, Europe, and the Asia Pacific. In this research, we aim to investigate the use of the open-source OMOP-CDM in the PATRON primary care data repository.MethodsWe used standard structured query language (SQL) to construct, extract, transform, and load scripts to convert the data to the OMOP-CDM. The process of mapping distinct free-text terms extracted from various EMRs presented a substantial challenge, as many terms could not be automatically matched to standard vocabularies through direct text comparison. This resulted in a number of terms that required manual assignment. To address this issue, we implemented a strategy where our clinical mappers were instructed to focus only on terms that appeared with sufficient frequency. We established a specific threshold value for each domain, ensuring that more than 95% of all records were linked to an approved vocabulary like SNOMED once appropriate mapping was completed. To assess the data quality of the resultant OMOP dataset we utilised the OHDSI Data Quality Dashboard (DQD) to evaluate the plausibility, conformity, and comprehensiveness of the data in the PATRON repository according to the Kahn framework.ResultsAcross three primary care EMR systems we converted data on 2.03 million active patients to version 5.4 of the OMOP common data model. The DQD assessment involved a total of 3,570 individual evaluations. Each evaluation compared the outcome against a predefined threshold. A ’FAIL’ occurred when the percentage of non-compliant rows exceeded the specified threshold value. In this assessment of the primary care OMOP database described here, we achieved an overall pass rate of 97%.ConclusionThe OMOP CDM’s widespread international use, support, and training provides a well-established pathway for data standardisation in collaborative research. Its compatibility allows the sharing of analysis packages across local and international research groups, which facilitates rapid and reproducible data comparisons. A suite of open-source tools, including the OHDSI Data Quality Dashboard (Version 1.4.1), supports the model. Its simplicity and standards-based approach facilitates adoption and integration into existing data processes.

The Ka-band ARM Zenith Radar (KAZR) remotely probes the extent and composition of clouds at millimeter wavelengths. The KAZR is a zenith-pointing Doppler radar that operates at a frequency of approximately 35 GHz. The main purpose of this radar is to determine the first three Doppler moments (reflectivity, vertical velocity, and spectral width) at a range resolution of approximately 30 meters from near-ground to nearly 20 kilometers in altitude.

The KAZR replaces the millimeter-wavelength cloud radar (MMCR) and uses a new digital receiver that provides higher spatial and temporal resolution than the MMCR. In addition, spectral artifacts in the data are significantly reduced in the KAZR, allowing researchers to study cloud dynamics much more closely than with the MMCR.

KAZR data from the 2018 2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

This dataset contains summaries from interviews with data sharing experts from internationa funding agencies on their organisation's data sharing policy. Data Sharing is widely recognised as crucial for accelerating scientific research and improving its quality. However, data sharing is still not a common practice. Funding agencies tend to facilitate the sharing of research data by both providing incentives and requiring data sharing as part of their policies and conditions for awarding grants. The goal of our article is to answer the following question: What challenges do international funding agencies see when it comes to their own efforts to foster and implement data sharing through their policies? We conducted a series of sixteen guideline-based expert interviews with representatives of leading international funding agencies. As contact persons for open science at their respective agencies, they offered their perspectives and experiences concerning their organisations’ data sharing policies. We performed a qualitative content analysis of the interviews and categorised the challenges perceived by funding agencies.

The Ka-Band Scanning ARM Cloud Radar (KASACR) records cloud properties.

ARM's scanning cloud radars are dual-frequency, dual-polarization Doppler radars mounted on a common scanning pedestal. Each pedestal includes a Ka-band radar (2kW peak power) and the deployment location determines whether the second radar is a W-band (1.7 kW peak power) or an X-band (20 kW peak power).

Beamwidths for Ka-bands paired with W-bands are roughly matched at 0.3 degrees. The X-band beamwidth is approximately 1 degree. Due to the narrow antenna beamwidth, ARM s scanning cloud radars use scanning strategies unlike typical weather radars. Rather than focusing on plan position indicator, or PPI, scans, the KASACR uses range height indicator, or RHI, scans at numerous azimuths to obtain cloud volume data. Measurements collected with the KASACR are copolar and cross-polar radar reflectivity, Doppler velocity, spectra width and spectra when not scanning, and linear depolarization ratio.

KASACR data from the 2018 2019 Cloud, Aerosol, and Complex Terrain Interactions (CACTI) field campaign in Argentina are now available as b1-level products. Building on the original CACTI operational data, the b1-level products feature improved data quality resulting from extensive analyses and corrections. The data are cross-calibrated to a common point, datastreams are corrected for operational issues that occurred during the campaign, and several data quality masks and basic derived products are incorporated. For more information, read the CACTI radar b1-level processing report.

The ‘UpStream’ project was co-created, co-developed and co-delivered with two active community groups in the UK (Friends of Bradford's Becks) and Taiwan (Taiwan Clean Water Alliance) who both were concerned about water pollution in their local rivers. The project has provided a testbed to achieve the aim of the project of aiding citizen scientists better understand local water quality. This has led to the development and deployment of a cost-effective Continuous Water Quality Monitoring device, the WaterBox, along with methods for transmitting, storing, visualising, and analysing the data collected. Parameters collected include: pH, temperature, conductivity and turbidity. A total of 104 practicalities of continuous water quality monitoring were observed and categorised as either technical, social, economic or wider responsibilities. These have been summarised in a publication that is current;y under review.

The UpStream project aims to improve water quality in the UK and Taiwan by working with citizens to gather data, share knowledge and experiences, and develop new technologies. Motivated by environmental issues already identified by the public, this participatory project will increase connectivity and action across a range of organisations and community groups.

Both the UK and Taiwan have problems with pollution of rivers. Across Europe, laws state that river water quality should not be impacted by human activity, but latest assessments suggest that just 38% of waters meet this standard. In Taiwan, rapid industrialisation and economic growth have had an impact on water pollution. In 2016, 65% of Taiwanese rivers were classed as moderately polluted. As economic growth stabilises and society evolves, attention is shifting to water quality issues; tighter water quality standards have been set and are incorporated into the government's Forward-looking Infrastructure plan.

In both the UK and Taiwan citizens feel strongly about water quality, and have founded local community action groups to instigate improvements. The UpStream project aims to improve water quality in the UK and Taiwan by creating an innovative partnership between these community groups and a range of academic and non-academic organisations to gather data, share knowledge and experiences, and develop new technologies.

The project partners from Taiwan (Academia Sinica, NTU and Location Aware Sensing System (LASS)) are experts in creating innovative technology with citizens that leads to real environmental improvements. They have developed low-cost air quality sensors that are now installed in 4,000 locations across Taiwan, and that feed into apps to help citizens avoid air pollution. Our UK partners at Newcastle University, Rain++ and RPS are experts in mobilising citizen science to address water problems. They have worked with the public on water issues in the UK and internationally for over 15 years. Our new and unique partnership will combine Taiwanese expertise in co-creating technology with citizens with UK expertise in water to empower citizens in both countries to improve water quality. Lack of water quality data to identify sources of pollution is a common problem in both countries, and our project aims to fix that.

Benefits and direct outputs of the activities planned through the project will include:

-Community groups will benefit from technical advice and a new, international support network. -A natural legacy for citizen-led environmental management through the involvement of students and community groups. -The involvement of tech start-ups that can provide insights into water quality, through their inclusion in the project team (LASS, Rain++, FondUS). -Any data or tools created through the project will follow open data protocols, making them accessible to local communities, interested researchers and businesses. The co-production of data (evidence) and tools will empower community groups to manage their local environment alongside relevant organisations after the project has ended. -Policy makers and regulators will have access to the open data collected through our project and will participate in talks through the project, helping to initiate change. -A prototype data visualisation and analysis tool, to aid understanding of water quality issues. -A scope for follow-on work, to continue the work of our unique partnership.

Whilst the legislative and societal contexts differ, Taiwan and the UK (and beyond) share common challenges with river pollution. Both have citizens that want to get involved and see change. The UpStream project aims to help by bringing together citizens with academic and industry partners for knowledge-exchange and long-term support.

The ETL (Extract, Transform, Load) Automation Testing market is experiencing robust growth, driven by the increasing complexity of data integration processes and the rising demand for faster, more reliable data pipelines. Businesses across various sectors are adopting ETL automation to improve data quality, reduce testing time, and minimize manual errors. The market is segmented by software, service, and application type (large enterprises and SMEs), reflecting diverse needs and budgetary constraints. While precise market size figures are unavailable in the provided data, a reasonable estimate, considering typical growth rates in the software testing sector and the strong adoption drivers, would place the market size around $2 billion in 2025. Assuming a conservative Compound Annual Growth Rate (CAGR) of 15%—reflective of the expanding demand for automation in data management—the market is projected to reach approximately $6 billion by 2033. This significant growth is fueled by trends such as cloud adoption, big data analytics, and the increasing regulatory scrutiny around data accuracy. Several restraining factors currently affect market expansion, including the high initial investment costs associated with implementing ETL automation tools and a potential shortage of skilled professionals capable of designing and managing these sophisticated systems. Despite these challenges, the long-term benefits of improved data quality, reduced operational costs, and increased efficiency are driving substantial market adoption across North America, Europe, and the Asia-Pacific region. The market's competitive landscape is characterized by a mix of established players offering comprehensive solutions and emerging niche players focusing on specific aspects of ETL testing. The continuous innovation in ETL automation technologies, along with the growing awareness of the importance of data quality, will fuel continued market expansion in the coming years.