Embark on a transformative journey with our Data Cleaning Project, where we meticulously refine and polish raw data into valuable insights. Our project focuses on streamlining data sets, removing inconsistencies, and ensuring accuracy to unlock its full potential.

Through advanced techniques and rigorous processes, we standardize formats, address missing values, and eliminate duplicates, creating a clean and reliable foundation for analysis. By enhancing data quality, we empower organizations to make informed decisions, drive innovation, and achieve strategic objectives with confidence.

Join us as we embark on this essential phase of data preparation, paving the way for more accurate and actionable insights that fuel success."

Materials from workshop conducted for Monroe Library faculty as part of TLT/Faculty Development/Digital Scholarship on 2018-04-05. Objectives:Clean dataAnalyze data using pivot tablesVisualize dataDesign accessible instruction for working with dataAssociated Research Guide at http://researchguides.loyno.edu/data_workshopData sets are from the following:

BaroqueArt Dataset by CulturePlex Lab is licensed under CC0 What's on the Menu? Menus by New York Public Library is licensed under CC0 Dog movie stars and dog breed popularity by Ghirlanda S, Acerbi A, Herzog H is licensed under CC BY 4.0 NOPD Misconduct Complaints, 2016-2018 by City of New Orleans Open Data is licensed under CC0 U.S. Consumer Product Safety Commission Recall Violations by CU.S. Consumer Product Safety Commission, Violations is licensed under CC0 NCHS - Leading Causes of Death: United States by Data.gov is licensed under CC0 Bob Ross Elements by Episode by Walt Hickey, FiveThirtyEight, is licensed under CC BY 4.0 Pacific Walrus Coastal Haulout 1852-2016 by U.S. Geological Survey, Alaska Science Center is licensed under CC0 Australia Registered Animals by Sunshine Coast Council is licensed under CC0

Company Datasets for valuable business insights!

Discover new business prospects, identify investment opportunities, track competitor performance, and streamline your sales efforts with comprehensive Company Datasets.

These datasets are sourced from top industry providers, ensuring you have access to high-quality information:

• Owler: Gain valuable business insights and competitive intelligence. -AngelList: Receive fresh startup data transformed into actionable insights. -CrunchBase: Access clean, parsed, and ready-to-use business data from private and public companies. -Craft.co: Make data-informed business decisions with Craft.co's company datasets. -Product Hunt: Harness the Product Hunt dataset, a leader in curating the best new products.

We provide fresh and ready-to-use company data, eliminating the need for complex scraping and parsing. Our data includes crucial details such as:

• Company name;
• Size;
• Founding date;
• Location;
• Industry;
• Revenue;
• Employee count;
• Competitors.

You can choose your preferred data delivery method, including various storage options, delivery frequency, and input/output formats.

Receive datasets in CSV, JSON, and other formats, with storage options like AWS S3 and Google Cloud Storage. Opt for one-time, monthly, quarterly, or bi-annual data delivery.

With Oxylabs Datasets, you can count on:

• Fresh and accurate data collected and parsed by our expert web scraping team.
• Time and resource savings, allowing you to focus on data analysis and achieving your business goals.
• A customized approach tailored to your specific business needs.
• Legal compliance in line with GDPR and CCPA standards, thanks to our membership in the Ethical Web Data Collection Initiative.

Pricing Options:

Standard Datasets: choose from various ready-to-use datasets with standardized data schemas, priced from $1,000/month.

Custom Datasets: Tailor datasets from any public web domain to your unique business needs. Contact our sales team for custom pricing.

Experience a seamless journey with Oxylabs:

• Understanding your data needs: We work closely to understand your business nature and daily operations, defining your unique data requirements.
• Developing a customized solution: Our experts create a custom framework to extract public data using our in-house web scraping infrastructure.
• Delivering data sample: We provide a sample for your feedback on data quality and the entire delivery process.
• Continuous data delivery: We continuously collect public data and deliver custom datasets per the agreed frequency.

Unlock the power of data with Oxylabs' Company Datasets and supercharge your business insights today!

During a 2023 survey carried out among marketing leaders predominantly in consumer packaged goods and retail from North America, the most common driver for clean room strategies were in-depth analytics (named by 56 percent of respondents), ability to measure campaign results (54 percent), and ease of data integration (52 percent). In a different survey, 29 percent of responding U.S. marketers said they would focus more on data clean rooms in 2023 than they had in 2022.

Data Wrangling Market size was valued at USD 1.63 Billion in 2024 and is projected to reach USD 3.2 Billion by 2031, growing at a CAGR of 8.80 % during the forecast period 2024-2031.

Global Data Wrangling Market Drivers

Growing Volume and Variety of Data: As digitalization has progressed, organizations have produced an exponential increase in both volume and variety of data. Data from a variety of sources, including social media, IoT devices, sensors, and workplace apps, is included in this, both structured and unstructured. Data wrangling tools are an essential part of contemporary data management methods because they allow firms to manage this heterogeneous data landscape effectively.

Growing Adoption of Advanced Analytics: To extract useful insights from data, companies in a variety of sectors are utilizing advanced analytics tools like artificial intelligence and machine learning. Nevertheless, access to clean, well-researched data is essential to the accomplishment of many analytics projects. The need for data wrangling solutions is fueled by the necessity of ensuring that data is accurate, consistent, and clean for usage in advanced analytics models.

Self-service data preparation solutions are becoming more and more necessary as data volumes rise. These technologies enable business users to prepare and analyze data on their own without requiring significant IT assistance. Platforms for data wrangling provide non-technical users with easy-to-use interfaces and functionalities that make it simple for them to clean, manipulate, and combine data. Data wrangling solutions are being used more quickly because of this self-service approach’s ability to increase agility and facilitate quicker decision-making within enterprises.

Emphasis on Data Governance and Compliance: With the rise of regulated sectors including healthcare, finance, and government, data governance and compliance have emerged as critical organizational concerns. Data wrangling technologies offer features for auditability, metadata management, and data quality control, which help with adhering to data governance regulations. The adoption of data wrangling solutions is fueled by these features, which assist enterprises in ensuring data integrity, privacy, and regulatory compliance.

Big Data Technologies’ Emergence: Companies can now store and handle enormous amounts of data more affordably because to the emergence of big data technologies like Hadoop, Spark, and NoSQL databases. However, efficient data preparation methods are needed to extract value from massive data. Organizations may accelerate their big data analytics initiatives by preprocessing and cleansing large amounts of data at scale with the help of data wrangling solutions that seamlessly interact with big data platforms.

Put an emphasis on cost-cutting and operational efficiency: Organizations are under pressure to maximize operational efficiency and cut expenses in the cutthroat business environment of today. Organizations can increase productivity and reduce resource requirements by implementing data wrangling solutions, which automate manual data preparation processes and streamline workflows. Furthermore, the danger of errors and expensive aftereffects is reduced when data quality problems are found and fixed early in the data pipeline.

Dataset

This dataset was created by Sanjeev Sahu

Contents

The market for data center cleaning services is expected to grow from USD XXX million in 2025 to USD XXX million by 2033, at a CAGR of XX% during the forecast period 2025-2033. The growth of the market is attributed to the increasing number of data centers and the need to maintain these facilities in a clean environment. Data centers are critical to the functioning of the modern economy, as they house the servers that store and process vast amounts of data. Maintaining these facilities in a clean environment is essential to prevent the accumulation of dust and other contaminants, which can lead to equipment failures and downtime. The market for data center cleaning services is segmented by type, application, and region. By type, the market is segmented into equipment cleaning, ceiling cleaning, floor cleaning, and others. Equipment cleaning is the largest segment of the market, accounting for over XX% of the total market revenue in 2025. By application, the market is segmented into the internet industry, finance and insurance, manufacturing industry, government departments, and others. The internet industry is the largest segment of the market, accounting for over XX% of the total market revenue in 2025. By region, the market is segmented into North America, South America, Europe, the Middle East & Africa, and Asia Pacific. North America is the largest segment of the market, accounting for over XX% of the total market revenue in 2025.

Data Prep Market size was valued at USD 4.02 Billion in 2024 and is projected to reach USD 16.12 Billion by 2031, growing at a CAGR of 19% from 2024 to 2031.

Global Data Prep Market Drivers

Increasing Demand for Data Analytics: Businesses across all industries are increasingly relying on data-driven decision-making, necessitating the need for clean, reliable, and useful information. This rising reliance on data increases the demand for better data preparation technologies, which are required to transform raw data into meaningful insights.
Growing Volume and Complexity of Data: The increase in data generation continues unabated, with information streaming in from a variety of sources. This data frequently lacks consistency or organization, therefore effective data preparation is critical for accurate analysis. To assure quality and coherence while dealing with such a large and complicated data landscape, powerful technologies are required.
Increased Use of Self-Service Data Preparation Tools: User-friendly, self-service data preparation solutions are gaining popularity because they enable non-technical users to access, clean, and prepare data. independently. This democratizes data access, decreases reliance on IT departments, and speeds up the data analysis process, making data-driven insights more available to all business units.
Integration of AI and ML: Advanced data preparation technologies are progressively using AI and machine learning capabilities to improve their effectiveness. These technologies automate repetitive activities, detect data quality issues, and recommend data transformations, increasing productivity and accuracy. The use of AI and ML streamlines the data preparation process, making it faster and more reliable.
Regulatory Compliance Requirements: Many businesses are subject to tight regulations governing data security and privacy. Data preparation technologies play an important role in ensuring that data meets these compliance requirements. By giving functions that help manage and protect sensitive information these technologies help firms negotiate complex regulatory climates.
Cloud-based Data Management: The transition to cloud-based data storage and analytics platforms needs data preparation solutions that can work smoothly with cloud-based data sources. These solutions must be able to integrate with a variety of cloud settings to assist effective data administration and preparation while also supporting modern data infrastructure.

Overview

This repository contains ready-to-use frequency time series as well as the corresponding pre-processing scripts in python. The data covers three synchronous areas of the European power grid:

• Continental Europe
• Great Britain
• Nordic

This work is part of the paper "Predictability of Power Grid Frequency"[1]. Please cite this paper, when using the data and the code. For a detailed documentation of the pre-processing procedure we refer to the supplementary material of the paper.

Data sources

We downloaded the frequency recordings from publically available repositories of three different Transmission System Operators (TSOs).

• Continental Europe [2]: We downloaded the data from the German TSO TransnetBW GmbH, which retains the Copyright on the data, but allows to re-publish it upon request [3].
• Great Britain [4]: The download was supported by National Grid ESO Open Data, which belongs to the British TSO National Grid. They publish the frequency recordings under the NGESO Open License [5].
• Nordic [6]: We obtained the data from the Finish TSO Fingrid, which provides the data under the open license CC-BY 4.0 [7].

Content of the repository

A) Scripts

1. In the "Download_scripts" folder you will find three scripts to automatically download frequency data from the TSO's websites.
2. In "convert_data_format.py" we save the data with corrected timestamp formats. Missing data is marked as NaN (processing step (1) in the supplementary material of [1]).
3. In "clean_corrupted_data.py" we load the converted data and identify corrupted recordings. We mark them as NaN and clean some of the resulting data holes (processing step (2) in the supplementary material of [1]).

The python scripts run with Python 3.7 and with the packages found in "requirements.txt".

B) Yearly converted and cleansed data

The folders "

• File type: The files are zipped csv-files, where each file comprises one year.
• Data format: The files contain two columns. The second column contains the frequency values in Hz. The first one represents the time stamps in the format Year-Month-Day Hour-Minute-Second, which is given as naive local time. The local time refers to the following time zones and includes Daylight Saving Times (python time zone in brackets):
  • TransnetBW: Continental European Time (CE)
  • Nationalgrid: Great Britain (GB)
  • Fingrid: Finland (Europe/Helsinki)
• NaN representation: We mark corrupted and missing data as "NaN" in the csv-files.

Use cases

We point out that this repository can be used in two different was:

• Use pre-processed data: You can directly use the converted or the cleansed data. Note however, that both data sets include segments of NaN-values due to missing and corrupted recordings. Only a very small part of the NaN-values were eliminated in the cleansed data to not manipulate the data too much.

• Produce your own cleansed data: Depending on your application, you might want to cleanse the data in a custom way. You can easily add your custom cleansing procedure in "clean_corrupted_data.py" and then produce cleansed data from the raw data in "

License

This work is licensed under multiple licenses, which are located in the "LICENSES" folder.

• We release the code in the folder "Scripts" under the MIT license .
• The pre-processed data in the subfolders "**/Fingrid" and "**/Nationalgrid" are licensed under CC-BY 4.0.
• TransnetBW originally did not publish their data under an open license. We have explicitly received the permission to publish the pre-processed version from TransnetBW. However, we cannot publish our pre-processed version under an open license due to the missing license of the original TransnetBW data.

Changelog

Version 2:

• Add time zone information to description
• Include new frequency data
• Update references
• Change folder structure to yearly folders

Version 3:

• Correct TransnetBW files for missing data in May 2016

Introducing Job Posting Datasets: Uncover labor market insights!

Elevate your recruitment strategies, forecast future labor industry trends, and unearth investment opportunities with Job Posting Datasets.

Job Posting Datasets Source:

1. Indeed: Access datasets from Indeed, a leading employment website known for its comprehensive job listings.

2. Glassdoor: Receive ready-to-use employee reviews, salary ranges, and job openings from Glassdoor.

3. StackShare: Access StackShare datasets to make data-driven technology decisions.

Job Posting Datasets provide meticulously acquired and parsed data, freeing you to focus on analysis. You'll receive clean, structured, ready-to-use job posting data, including job titles, company names, seniority levels, industries, locations, salaries, and employment types.

Choose your preferred dataset delivery options for convenience:

Receive datasets in various formats, including CSV, JSON, and more. Opt for storage solutions such as AWS S3, Google Cloud Storage, and more. Customize data delivery frequencies, whether one-time or per your agreed schedule.

Why Choose Oxylabs Job Posting Datasets:

1. Fresh and accurate data: Access clean and structured job posting datasets collected by our seasoned web scraping professionals, enabling you to dive into analysis.

2. Time and resource savings: Focus on data analysis and your core business objectives while we efficiently handle the data extraction process cost-effectively.

3. Customized solutions: Tailor our approach to your business needs, ensuring your goals are met.

4. Legal compliance: Partner with a trusted leader in ethical data collection. Oxylabs is a founding member of the Ethical Web Data Collection Initiative, aligning with GDPR and CCPA best practices.

Pricing Options:

Standard Datasets: choose from various ready-to-use datasets with standardized data schemas, priced from $1,000/month.

Custom Datasets: Tailor datasets from any public web domain to your unique business needs. Contact our sales team for custom pricing.

Experience a seamless journey with Oxylabs:

• Understanding your data needs: We work closely to understand your business nature and daily operations, defining your unique data requirements.
• Developing a customized solution: Our experts create a custom framework to extract public data using our in-house web scraping infrastructure.
• Delivering data sample: We provide a sample for your feedback on data quality and the entire delivery process.
• Continuous data delivery: We continuously collect public data and deliver custom datasets per the agreed frequency.

Effortlessly access fresh job posting data with Oxylabs Job Posting Datasets.

Data Quality Management Software Market size was valued at USD 4.32 Billion in 2023 and is projected to reach USD 10.73 Billion by 2030, growing at a CAGR of 17.75% during the forecast period 2024-2030.

Global Data Quality Management Software Market Drivers

The growth and development of the Data Quality Management Software Market can be credited with a few key market drivers. Several of the major market drivers are listed below:

Growing Data Volumes: Organizations are facing difficulties in managing and guaranteeing the quality of massive volumes of data due to the exponential growth of data generated by consumers and businesses. Organizations can identify, clean up, and preserve high-quality data from a variety of data sources and formats with the use of data quality management software.
Increasing Complexity of Data Ecosystems: Organizations function within ever-more-complex data ecosystems, which are made up of a variety of systems, formats, and data sources. Software for data quality management enables the integration, standardization, and validation of data from various sources, guaranteeing accuracy and consistency throughout the data landscape.
Regulatory Compliance Requirements: Organizations must maintain accurate, complete, and secure data in order to comply with regulations like the GDPR, CCPA, HIPAA, and others. Data quality management software ensures data accuracy, integrity, and privacy, which assists organizations in meeting regulatory requirements.
Growing Adoption of Business Intelligence and Analytics: As BI and analytics tools are used more frequently for data-driven decision-making, there is a greater need for high-quality data. With the help of data quality management software, businesses can extract actionable insights and generate significant business value by cleaning, enriching, and preparing data for analytics.
Focus on Customer Experience: Put the Customer Experience First: Businesses understand that providing excellent customer experiences requires high-quality data. By ensuring data accuracy, consistency, and completeness across customer touchpoints, data quality management software assists businesses in fostering more individualized interactions and higher customer satisfaction.
Initiatives for Data Migration and Integration: Organizations must clean up, transform, and move data across heterogeneous environments as part of data migration and integration projects like cloud migration, system upgrades, and mergers and acquisitions. Software for managing data quality offers procedures and instruments to guarantee the accuracy and consistency of transferred data.
Need for Data Governance and Stewardship: The implementation of efficient data governance and stewardship practises is imperative to guarantee data quality, consistency, and compliance. Data governance initiatives are supported by data quality management software, which offers features like rule-based validation, data profiling, and lineage tracking.
Operational Efficiency and Cost Reduction: Inadequate data quality can lead to errors, higher operating costs, and inefficiencies for organizations. By guaranteeing high-quality data across business processes, data quality management software helps organizations increase operational efficiency, decrease errors, and minimize rework.

Product Review Datasets: Uncover user sentiment

Harness the power of Product Review Datasets to understand user sentiment and insights deeply. These datasets are designed to elevate your brand and product feature analysis, help you evaluate your competitive stance, and assess investment risks.

Data sources:

• Trustpilot: datasets encompassing general consumer reviews and ratings across various businesses, products, and services.

Leave the data collection challenges to us and dive straight into market insights with clean, structured, and actionable data, including:

• Product name;
• Product category;
• Number of ratings;
• Ratings average;
• Review title;
• Review body;

Choose from multiple data delivery options to suit your needs:

1. Receive data in easy-to-read formats like spreadsheets or structured JSON files.
2. Select your preferred data storage solutions, including SFTP, Webhooks, Google Cloud Storage, AWS S3, and Microsoft Azure Storage.
3. Tailor data delivery frequencies, whether on-demand or per your agreed schedule.

Why choose Oxylabs?

1. Fresh and accurate data: Access organized, structured, and comprehensive data collected by our leading web scraping professionals.

2. Time and resource savings: Concentrate on your core business goals while we efficiently handle the data extraction process at an affordable cost.

3. Adaptable solutions: Share your specific data requirements, and we'll craft a customized data collection approach to meet your objectives.

4. Legal compliance: Partner with a trusted leader in ethical data collection. Oxylabs is a founding member of the Ethical Web Data Collection Initiative, aligning with GDPR and CCPA standards.

Pricing Options:

Standard Datasets: choose from various ready-to-use datasets with standardized data schemas, priced from $1,000/month.

Custom Datasets: Tailor datasets from any public web domain to your unique business needs. Contact our sales team for custom pricing.

Experience a seamless journey with Oxylabs:

• Understanding your data needs: We work closely to understand your business nature and daily operations, defining your unique data requirements.
• Developing a customized solution: Our experts create a custom framework to extract public data using our in-house web scraping infrastructure.
• Delivering data sample: We provide a sample for your feedback on data quality and the entire delivery process.
• Continuous data delivery: We continuously collect public data and deliver custom datasets per the agreed frequency.

Join the ranks of satisfied customers who appreciate our meticulous attention to detail and personalized support. Experience the power of Product Review Datasets today to uncover valuable insights and enhance decision-making.

Dataset

This dataset was created by Pascal Pfeiffer

Contents

These data sets contain raw and processed data used in for analyses, figures, and tables in the Region 8 Memo: Characterization of chloride and conductivity levels in the Bitter Creek Watershed, WY. However, these data may be used for other analyses alone or in combination with other or new data. These data were used to assess whether chloride levels are naturally high in streams in the Bitter Creek, WY watershed and how chloride concentrations expected to protect 95 percent of aquatic genera in these streams compare to Wyoming’s chloride criteria applicable to the Bitter Creek watershed. Owing to the arid conditions, background conductivity and chloride levels were characterized for surface flow and ground water flow conditions. Natural chloride levels were found to be less than current water quality criteria for Wyoming. Although the report was prepared for USEPA Region 8 and OST, Office of Water, the report will be of interest to the WDEQ, Sweetwater County Conservation District, and the regulated community. No formal metadata standard was used. Pedigree.xlsx contains: 1. NOTES: Description of work and other worksheets. 2. Pedigree_Summary: Source files used to create figures and tables. 3. DataFiles: Data files used in the R code for creating the figures and tables 4. R_Script: Summary of the R scripts. 5. DataDictionary: Data file titles in all data files Folders: _Datasets Data file uploaded to Environmental Dataset Gateway "A list of subfolders: _R: Clean R scripts used to generate document figures and tables _Tables_Figures: Files generated from R script and used in the Region 6 memo R Code and Data: All additional files used for this project, including original files, intermediate files, extra output files, and extra functions the ""_R"" folder stores R scripts for input and output files and an R project file.. Users can open the R project and run R scripts directly from the ""_R"" folder or the XC95 folder by installing R, RStudio, and associated R packages."

This dataset contains the content of the subset of all files with a correct publication date from the 2017 release of files related to the JFK case (retrieved from https://www.archives.gov/research/jfk/2017-release). This content was extracted from the source PDF files using the R OCR libraries tesseract and pdftools.

The code to derive the dataset is given as follows:

### BEGIN R DATA PROCESSING SCRIPT

library(tesseract)
library(pdftools)

pdfs <- list.files("/home/STAFF/luczakma/RProjects/JFK/data/files/")

meta <- read.csv2("/home/STAFF/luczakma/RProjects/JFK/data/jfkrelease-2017-dce65d0ec70a54d5744de17d280f3ad2.csv",header = T,sep = ',')

meta$Doc.Date <- as.character(meta$Doc.Date)

meta.clean <- meta[-which(meta$Doc.Date=="" | grepl("/0000",meta$Doc.Date)),]
for(i in 1:nrow(meta.clean)){
meta.clean$Doc.Date[i] <- gsub("00","01",meta.clean$Doc.Date[i])

if(nchar(meta.clean$Doc.Date[i])<10){
meta.clean$Doc.Date[i]<-format(strptime(meta.clean$Doc.Date[i],format = "%d/%m/%y"),"%m/%d/%Y")
}

}

meta.clean$Doc.Date <- strptime(meta.clean$Doc.Date,format = "%m/%d/%Y")

meta.clean <- meta.clean[order(meta.clean$Doc.Date),]

docs <- data.frame(content=character(0),dpub=character(0),stringsAsFactors = F)
for(i in 1:nrow(meta.clean)){
#for(i in 1:3){
pdf_prop <- pdftools::pdf_info(paste0("/home/STAFF/luczakma/RProjects/JFK/data/files/",tolower(gsub("\\s+"," ",gsub(" ","",meta.clean$File.Name[i])))))
tmp_files <- c()
for(k in 1:pdf_prop$pages){
tmp_files <- c(tmp_files,paste0("/home/STAFF/luczakma/RProjects/JFK/data/tmp/",k))
}

img_file <- pdftools::pdf_convert(paste0("/home/STAFF/luczakma/RProjects/JFK/data/files/",tolower(gsub("\\s+"," ",gsub(" ","",meta.clean$File.Name[i])))), format = 'tiff', pages = NULL, dpi = 700,filenames = tmp_files)

txt <- ""

for(j in 1:length(img_file)){
extract <- ocr(img_file[j], engine = tesseract("eng"))
#unlink(img_file)
txt <- paste(txt,extract,collapse = " ")
}

docs <- rbind(docs,data.frame(content=iconv(tolower(gsub("\\s+"," ",gsub("[[:punct:]]|[ ]"," ",txt))),to="UTF-8"),dpub=format(meta.clean$Doc.Date[i],"%Y/%m/%d"),stringsAsFactors = F),stringsAsFactors = F)
}

### END R DATA PROCESSING SCRIPT

The Alaska Geochemical Database Version 3.0 (AGDB3) contains new geochemical data compilations in which each geologic material sample has one best value determination for each analyzed species, greatly improving speed and efficiency of use. Like the Alaska Geochemical Database Version 2.0 before it, the AGDB3 was created and designed to compile and integrate geochemical data from Alaska to facilitate geologic mapping, petrologic studies, mineral resource assessments, definition of geochemical baseline values and statistics, element concentrations and associations, environmental impact assessments, and studies in public health associated with geology. This relational database, created from databases and published datasets of the U.S. Geological Survey (USGS), Atomic Energy Commission National Uranium Resource Evaluation (NURE), Alaska Division of Geological & Geophysical Surveys (DGGS), U.S. Bureau of Mines, and U.S. Bureau of Land Management serves as a data archive in support of Alaskan geologic and geochemical projects and contains data tables in several different formats describing historical and new quantitative and qualitative geochemical analyses. The analytical results were determined by 112 laboratory and field analytical methods on 396,343 rock, sediment, soil, mineral, heavy-mineral concentrate, and oxalic acid leachate samples. Most samples were collected by personnel of these agencies and analyzed in agency laboratories or, under contracts, in commercial analytical laboratories. These data represent analyses of samples collected as part of various agency programs and projects from 1938 through 2017. In addition, mineralogical data from 18,138 nonmagnetic heavy-mineral concentrate samples are included in this database. The AGDB3 includes historical geochemical data archived in the USGS National Geochemical Database (NGDB) and NURE National Uranium Resource Evaluation-Hydrogeochemical and Stream Sediment Reconnaissance databases, and in the DGGS Geochemistry database. Retrievals from these databases were used to generate most of the AGDB data set. These data were checked for accuracy regarding sample location, sample media type, and analytical methods used. In other words, the data of the AGDB3 supersedes data in the AGDB and the AGDB2, but the background about the data in these two earlier versions are needed by users of the current AGDB3 to understand what has been done to amend, clean up, correct and format this data. Corrections were entered, resulting in a significantly improved Alaska geochemical dataset, the AGDB3. Data that were not previously in these databases because the data predate the earliest agency geochemical databases, or were once excluded for programmatic reasons, are included here in the AGDB3 and will be added to the NGDB and Alaska Geochemistry. The AGDB3 data provided here are the most accurate and complete to date and should be useful for a wide variety of geochemical studies. The AGDB3 data provided in the online version of the database may be updated or changed periodically.

The Data Science Platform market is experiencing robust growth, projected to reach $10.15 billion in 2025 and exhibiting a Compound Annual Growth Rate (CAGR) of 23.50% from 2025 to 2033. This expansion is driven by several key factors. The increasing availability and affordability of cloud computing resources are lowering the barrier to entry for organizations of all sizes seeking to leverage data science capabilities. Furthermore, the growing volume and complexity of data generated across various industries necessitates sophisticated platforms for efficient data processing, analysis, and model deployment. The rise of AI and machine learning further fuels demand, as organizations strive to gain competitive advantages through data-driven insights and automation. Strong demand from sectors like IT and Telecom, BFSI (Banking, Financial Services, and Insurance), and Retail & E-commerce are major contributors to market growth. The preference for cloud-based deployment models over on-premise solutions is also accelerating market expansion, driven by scalability, cost-effectiveness, and accessibility. Market segmentation reveals a diverse landscape. While large enterprises are currently major consumers, the increasing adoption of data science by small and medium-sized enterprises (SMEs) represents a significant growth opportunity. The platform offering segment is anticipated to maintain a substantial market share, driven by the need for comprehensive tools that integrate data ingestion, processing, modeling, and deployment capabilities. Geographically, North America and Europe are currently leading the market, but the Asia-Pacific region, particularly China and India, is poised for significant growth due to expanding digital economies and increasing investments in data science initiatives. Competitive intensity is high, with established players like IBM, SAS, and Microsoft competing alongside innovative startups like DataRobot and Databricks. This competitive landscape fosters innovation and further accelerates market expansion. Recent developments include: November 2023 - Stagwell announced a partnership with Google Cloud and SADA, a Google Cloud premier partner, to develop generative AI (gen AI) marketing solutions that support Stagwell agencies, client partners, and product development within the Stagwell Marketing Cloud (SMC). The partnership will help in harnessing data analytics and insights by developing and training a proprietary Stagwell large language model (LLM) purpose-built for Stagwell clients, productizing data assets via APIs to create new digital experiences for brands, and multiplying the value of their first-party data ecosystems to drive new revenue streams using Vertex AI and open source-based models., May 2023 - IBM launched a new AI and data platform, watsonx, it is aimed at allowing businesses to accelerate advanced AI usage with trusted data, speed and governance. IBM also introduced GPU-as-a-service, which is designed to support AI intensive workloads, with an AI dashboard to measure, track and help report on cloud carbon emissions. With watsonx, IBM offers an AI development studio with access to IBMcurated and trained foundation models and open-source models, access to a data store to gather and clean up training and tune data,. Key drivers for this market are: Rapid Increase in Big Data, Emerging Promising Use Cases of Data Science and Machine Learning; Shift of Organizations Toward Data-intensive Approach and Decisions. Potential restraints include: Lack of Skillset in Workforce, Data Security and Reliability Concerns. Notable trends are: Small and Medium Enterprises to Witness Major Growth.

This is the code and data used in the paper "Substituting Clean for Dirty Energy: A Bottom-Up Analysis": (i) GAMS code of the numerical bottom-up optimization model (ii) Input and output data files of the numerical model (iii) Mathematica code used for fitting CES/VES production functions and to generate plots

➡️ You can choose from multiple data formats, delivery frequency options, and delivery methods;

➡️ You can select raw or clean and AI-enriched datasets;

➡️ Multiple APIs designed for effortless search and enrichment (accessible using a user-friendly self-service tool);

➡️ Fresh data: daily updates, easy change tracking with dedicated data fields, and a constant flow of new data;

➡️ You get all necessary resources for evaluating our data: a free consultation, a data sample, or free credits for testing our APIs.

Coresignal's employee data enables you to create and improve innovative data-driven solutions and extract actionable business insights. These datasets are popular among companies from different industries, including HR and sales technology and investment.

Employee Data use cases:

✅ Source best-fit talent for your recruitment needs

Coresignal's Employee Data can help source the best-fit talent for your recruitment needs by providing the most up-to-date information on qualified candidates globally.

✅ Fuel your lead generation pipeline

Enhance lead generation with 712M+ up-to-date employee records from the largest professional network. Our Employee Data can help you develop a qualified list of potential clients and enrich your own database.

✅ Analyze talent for investment opportunities

Employee Data can help you generate actionable signals and identify new investment opportunities earlier than competitors or perform deeper analysis of companies you're interested in.

➡️ Why 400+ data-powered businesses choose Coresignal:

1. Experienced data provider (in the market since 2016);
2. Exceptional client service;
3. Responsible and secure data collection.

Abstract

The basic goal of this survey is to provide the necessary database for formulating national policies at various levels. It represents the contribution of the household sector to the Gross National Product (GNP). Household Surveys help as well in determining the incidence of poverty, and providing weighted data which reflects the relative importance of the consumption items to be employed in determining the benchmark for rates and prices of items and services. Generally, the Household Expenditure and Consumption Survey is a fundamental cornerstone in the process of studying the nutritional status in the Palestinian territory.

The raw survey data provided by the Statistical Office was cleaned and harmonized by the Economic Research Forum, in the context of a major research project to develop and expand knowledge on equity and inequality in the Arab region. The main focus of the project is to measure the magnitude and direction of change in inequality and to understand the complex contributing social, political and economic forces influencing its levels. However, the measurement and analysis of the magnitude and direction of change in this inequality cannot be consistently carried out without harmonized and comparable micro-level data on income and expenditures. Therefore, one important component of this research project is securing and harmonizing household surveys from as many countries in the region as possible, adhering to international statistics on household living standards distribution. Once the dataset has been compiled, the Economic Research Forum makes it available, subject to confidentiality agreements, to all researchers and institutions concerned with data collection and issues of inequality. Data is a public good, in the interest of the region, and it is consistent with the Economic Research Forum's mandate to make micro data available, aiding regional research on this important topic.

Geographic coverage

The survey data covers urban, rural and camp areas in West Bank and Gaza Strip.

Analysis unit

1- Household/families. 2- Individuals.

Universe

The survey covered all Palestinian households who are usually resident in the Palestinian Territory during 2010.

Kind of data

Sample survey data [ssd]

Sampling procedure

Sample and Frame:

The sampling frame consists of all enumeration areas which were enumerated in 2007, each numeration area consists of buildings and housing units with average of about 120 households in it. These enumeration areas are used as primary sampling units PSUs in the first stage of the sampling selection.

Sample Design:

The sample is a stratified cluster systematic random sample with two stages: First stage: selection of a systematic random sample of 192 enumeration areas. Second stage: selection of a systematic random sample of 24 households from each enumeration area selected in the first stage.

Note: in Jerusalem Governorate (J1), 13 enumeration areas were selected; then in the second phase, a group of households from each enumeration area were chosen using census-2007 method of delineation and enumeration. This method was adopted to ensure household response is to the maximum to comply with the percentage of non-response as set in the sample design.Enumeration areas were distributed to twelve months and the sample for each quarter covers sample strata (Governorate, locality type) Sample strata:

The population was divided by:

1- Governorate 2- Type of Locality (urban, rural, refugee camps)

Sample Size:

The calculated sample size for the Expenditure and Consumption Survey in 2010 is about 3,757 households, 2,574 households in West Bank and 1,183 households in Gaza Strip.

Mode of data collection

Face-to-face [f2f]

Research instrument

The questionnaire consists of two main parts:

First: Survey's questionnaire

Part of the questionnaire is to be filled in during the visit at the beginning of the month, while the other part is to be filled in at the end of the month. The questionnaire includes:

Control sheet: Includes household’s identification data, date of visit, data on the fieldwork and data processing team, and summary of household’s members by gender.

Household roster: Includes demographic, social, and economic characteristics of household’s members.

Housing characteristics: Includes data like type of housing unit, number of rooms, value of rent, and connection of housing unit to basic services like water, electricity and sewage. In addition, data in this section includes source of energy used for cooking and heating, distance of housing unit from transportation, education, and health centers, and sources of income generation like ownership of farm land or animals.

Food and Non-Food Items: includes food and non-food items, and household record her expenditure for one month.

Durable Goods Schedule: Includes list of main goods like washing machine, refrigerator,TV.

Assistances and Poverty: Includes data about cash and in kind assistances (assistance value,assistance source), also collecting data about household situation, and the procedures to cover expenses.

Monthly and annual income: Data pertinent to household’s income from different sources is collected at the end of the registration period.

Second: List of goods

The classification of the list of goods is based on the recommendation of the United Nations for the SNA under the name Classification of Personal Consumption by purpose. The list includes 55 groups of expenditure and consumption where each is given a sequence number based on its importance to the household starting with food goods, clothing groups, housing, medical treatment, transportation and communication, and lastly durable goods. Each group consists of important goods. The total number of goods in all groups amounted to 667 items for goods and services. Groups from 1-21 includes goods pertinent to food, drinks and cigarettes. Group 22 includes goods that are home produced and consumed by the household. The groups 23-45 include all items except food, drinks and cigarettes. The groups 50-55 include durable goods. The data is collected based on different reference periods to represent expenditure during the whole year except for cars where data is collected for the last three years.

Registration form

The registration form includes instructions and examples on how to record consumption and expenditure items. The form includes columns: 1.Monetary: If the good is purchased, or in kind: if the item is self produced. 2.Title of the service of the good 3.Unit of measurement (kilogram, liter, number) 4. Quantity 5. Value

The pages of the registration form are colored differently for the weeks of the month. The footer for each page includes remarks that encourage households to participate in the survey. The following are instructions that illustrate the nature of the items that should be recorded: 1. Monetary expenditures during purchases 2. Purchases based on debts 3.Monetary gifts once presented 4. Interest at pay 5. Self produced food and goods once consumed 6. Food and merchandise from commercial project once consumed 7. Merchandises once received as a wage or part of a wage from the employer.

Cleaning operations

Raw Data

Data editing took place through a number of stages, including: 1. Office editing and coding 2. Data entry 3. Structure checking and completeness 4. Structural checking of SPSS data files

Harmonized Data

• The Statistical Package for Social Science (SPSS) is used to clean and harmonize the datasets.
• The harmonization process starts with cleaning all raw data files received from the Statistical Office.
• Cleaned data files are then all merged to produce one data file on the individual level containing all variables subject to harmonization.
• A country-specific program is generated for each dataset to generate/compute/recode/rename/format/label harmonized variables.
• A post-harmonization cleaning process is run on the data.
• Harmonized data is saved on the household as well as the individual level, in SPSS and converted to STATA format.

Response rate

The survey sample consisted of 4,767 households, which includes 4,608 households of the original sample plus 159 households as an additional sample. A total of 3,757 households completed the interview: 2,574 households from the West Bank and 1,183 households in the Gaza Strip. Weights were modified to account for the non-response rate. The response rate in the Palestinian Territory 28.1% (82.4% in the West Bank was and 81.6% in Gaza Strip).

Sampling error estimates

The impact of errors on data quality was reduced to a minimum due to the high efficiency and outstanding selection, training, and performance of the fieldworkers. Procedures adopted during the fieldwork of the survey were considered a necessity to ensure the collection of accurate data, notably: 1) Develop schedules to conduct field visits to households during survey fieldwork. The objectives of the visits and the data collected on each visit were predetermined. 2) Fieldwork editing rules were applied during the data collection to ensure corrections were implemented before the end of fieldwork activities. 3) Fieldworkers were instructed to provide details in cases of extreme expenditure or consumption by the household. 4) Questions on income were postponed until the final visit at the end of the month. 5) Validation rules were embedded in the data processing systems, along with procedures to verify data entry and data edit.