The Data Cleaning Tools Market is projected to grow at 16.9% CAGR, reaching $6.78 Billion by 2029. Where is the industry heading next? Get the sample report now!

The data cleansing software market is expanding rapidly, with a market size of XXX million in 2023 and a projected CAGR of XX% from 2023 to 2033. This growth is driven by the increasing need for accurate and reliable data in various industries, including healthcare, finance, and retail. Key market trends include the growing adoption of cloud-based solutions, the increasing use of artificial intelligence (AI) and machine learning (ML) to automate the data cleansing process, and the increasing demand for data governance and compliance. The market is segmented by deployment type (cloud-based vs. on-premise) and application (large enterprises vs. SMEs vs. government agencies). Major players in the market include IBM, SAS Institute Inc, SAP SE, Trifacta, OpenRefine, Data Ladder, Analytics Canvas (nModal Solutions Inc.), Mo-Data, Prospecta, WinPure Ltd, Symphonic Source Inc, MuleSoft, MapR Technologies, V12 Data, and Informatica. This report provides a comprehensive overview of the global data cleansing software market, with a focus on market concentration, product insights, regional insights, trends, driving forces, challenges and restraints, growth catalysts, leading players, and significant developments.

Snapshot img

Data Science Platform Market Size 2025-2029

The data science platform market size is forecast to increase by USD 763.9 million at a CAGR of 40.2% between 2024 and 2029.

The market is experiencing significant growth, driven by the integration of artificial intelligence (AI) and machine learning (ML). This enhancement enables more advanced data analysis and prediction capabilities, making data science platforms an essential tool for businesses seeking to gain insights from their data. Another trend shaping the market is the emergence of containerization and microservices in platforms. This development offers increased flexibility and scalability, allowing organizations to efficiently manage their projects. 
However, the use of platforms also presents challenges, particularly In the area of data privacy and security. Ensuring the protection of sensitive data is crucial for businesses, and platforms must provide strong security measures to mitigate risks. In summary, the market is witnessing substantial growth due to the integration of AI and ML technologies, containerization, and microservices, while data privacy and security remain key challenges.

What will be the Size of the Data Science Platform Market During the Forecast Period?

Request Free Sample

The market is experiencing significant growth due to the increasing demand for advanced data analysis capabilities in various industries. Cloud-based solutions are gaining popularity as they offer scalability, flexibility, and cost savings. The market encompasses the entire project life cycle, from data acquisition and preparation to model development, training, and distribution. Big data, IoT, multimedia, machine data, consumer data, and business data are prime sources fueling this market's expansion. Unstructured data, previously challenging to process, is now being effectively managed through tools and software. Relational databases and machine learning models are integral components of platforms, enabling data exploration, preprocessing, and visualization.
Moreover, Artificial intelligence (AI) and machine learning (ML) technologies are essential for handling complex workflows, including data cleaning, model development, and model distribution. Data scientists benefit from these platforms by streamlining their tasks, improving productivity, and ensuring accurate and efficient model training. The market is expected to continue its growth trajectory as businesses increasingly recognize the value of data-driven insights.

How is this Data Science Platform Industry segmented and which is the largest segment?

The industry research report provides comprehensive data (region-wise segment analysis), with forecasts and estimates in 'USD million' for the period 2025-2029, as well as historical data from 2019-2023 for the following segments.

Deployment

  On-premises
  Cloud


Component

  Platform
  Services


End-user

  BFSI
  Retail and e-commerce
  Manufacturing
  Media and entertainment
  Others


Sector

  Large enterprises
  SMEs


Geography

  North America

    Canada
    US


  Europe

    Germany
    UK
    France


  APAC

    China
    India
    Japan


  South America

    Brazil


  Middle East and Africa

By Deployment Insights

The on-premises segment is estimated to witness significant growth during the forecast period.

On-premises deployment is a traditional method for implementing technology solutions within an organization. This approach involves purchasing software with a one-time license fee and a service contract. On-premises solutions offer enhanced security, as they keep user credentials and data within the company's premises. They can be customized to meet specific business requirements, allowing for quick adaptation. On-premises deployment eliminates the need for third-party providers to manage and secure data, ensuring data privacy and confidentiality. Additionally, it enables rapid and easy data access, and keeps IP addresses and data confidential. This deployment model is particularly beneficial for businesses dealing with sensitive data, such as those in manufacturing and large enterprises. While cloud-based solutions offer flexibility and cost savings, on-premises deployment remains a popular choice for organizations prioritizing data security and control.

Get a glance at the Data Science Platform Industry report of share of various segments. Request Free Sample

The on-premises segment was valued at USD 38.70 million in 2019 and showed a gradual increase during the forecast period.

Regional Analysis

North America is estimated to contribute 48% to the growth of the global market during the forecast period.

Technavio's analysts have elaborately explained the regional trends and drivers that shape the market during the forecast period.

For more insights on the market share of various regions, Request F

The Data Preparation Tools market is experiencing robust growth, projected to reach a market size of $3 billion in 2025 and exhibiting a Compound Annual Growth Rate (CAGR) of 17.7% from 2025 to 2033. This significant expansion is driven by several key factors. The increasing volume and velocity of data generated across industries necessitates efficient and effective data preparation processes to ensure data quality and usability for analytics and machine learning initiatives. The rising adoption of cloud-based solutions, coupled with the growing demand for self-service data preparation tools, is further fueling market growth. Businesses across various sectors, including IT and Telecom, Retail and E-commerce, BFSI (Banking, Financial Services, and Insurance), and Manufacturing, are actively seeking solutions to streamline their data pipelines and improve data governance. The diverse range of applications, from simple data cleansing to complex data transformation tasks, underscores the versatility and broad appeal of these tools. Leading vendors like Microsoft, Tableau, and Alteryx are continuously innovating and expanding their product offerings to meet the evolving needs of the market, fostering competition and driving further advancements in data preparation technology. This rapid growth is expected to continue, driven by ongoing digital transformation initiatives and the increasing reliance on data-driven decision-making. The segmentation of the market into self-service and data integration tools, alongside the varied applications across different industries, indicates a multifaceted and dynamic landscape. While challenges such as data security concerns and the need for skilled professionals exist, the overall market outlook remains positive, projecting substantial expansion throughout the forecast period. The adoption of advanced technologies like artificial intelligence (AI) and machine learning (ML) within data preparation tools promises to further automate and enhance the process, contributing to increased efficiency and reduced costs for businesses. The competitive landscape is dynamic, with established players alongside emerging innovators vying for market share, leading to continuous improvement and innovation within the industry.

Data Quality Management Software Market size was valued at USD 4.32 Billion in 2023 and is projected to reach USD 10.73 Billion by 2030, growing at a CAGR of 17.75% during the forecast period 2024-2030.

Global Data Quality Management Software Market Drivers

The growth and development of the Data Quality Management Software Market can be credited with a few key market drivers. Several of the major market drivers are listed below:

Growing Data Volumes: Organizations are facing difficulties in managing and guaranteeing the quality of massive volumes of data due to the exponential growth of data generated by consumers and businesses. Organizations can identify, clean up, and preserve high-quality data from a variety of data sources and formats with the use of data quality management software.
Increasing Complexity of Data Ecosystems: Organizations function within ever-more-complex data ecosystems, which are made up of a variety of systems, formats, and data sources. Software for data quality management enables the integration, standardization, and validation of data from various sources, guaranteeing accuracy and consistency throughout the data landscape.
Regulatory Compliance Requirements: Organizations must maintain accurate, complete, and secure data in order to comply with regulations like the GDPR, CCPA, HIPAA, and others. Data quality management software ensures data accuracy, integrity, and privacy, which assists organizations in meeting regulatory requirements.
Growing Adoption of Business Intelligence and Analytics: As BI and analytics tools are used more frequently for data-driven decision-making, there is a greater need for high-quality data. With the help of data quality management software, businesses can extract actionable insights and generate significant business value by cleaning, enriching, and preparing data for analytics.
Focus on Customer Experience: Put the Customer Experience First: Businesses understand that providing excellent customer experiences requires high-quality data. By ensuring data accuracy, consistency, and completeness across customer touchpoints, data quality management software assists businesses in fostering more individualized interactions and higher customer satisfaction.
Initiatives for Data Migration and Integration: Organizations must clean up, transform, and move data across heterogeneous environments as part of data migration and integration projects like cloud migration, system upgrades, and mergers and acquisitions. Software for managing data quality offers procedures and instruments to guarantee the accuracy and consistency of transferred data.
Need for Data Governance and Stewardship: The implementation of efficient data governance and stewardship practises is imperative to guarantee data quality, consistency, and compliance. Data governance initiatives are supported by data quality management software, which offers features like rule-based validation, data profiling, and lineage tracking.
Operational Efficiency and Cost Reduction: Inadequate data quality can lead to errors, higher operating costs, and inefficiencies for organizations. By guaranteeing high-quality data across business processes, data quality management software helps organizations increase operational efficiency, decrease errors, and minimize rework.

The size and share of the market is categorized based on Application (Data cleansing tools, Data integration software, Data transformation tools, Data enrichment solutions, Data validation tools) and Product (Data preparation, Data integration, Data cleansing, Data transformation, Data enrichment) and geographical regions (North America, Europe, Asia-Pacific, South America, and Middle-East and Africa).

The PC cleaner software market is experiencing steady growth, projected to reach a market size of $511.4 million in 2025, exhibiting a Compound Annual Growth Rate (CAGR) of 5.3%. This growth is fueled by several factors. The increasing prevalence of malware and unwanted software, coupled with the growing user base of personal computers, creates a consistent demand for effective PC cleaning solutions. Furthermore, the rise in sophisticated cyber threats necessitates robust security and optimization tools, driving adoption of both on-premises and cloud-based PC cleaner software across individual users, enterprises, and government sectors. The market's segmentation reflects this diverse user base; while on-premises solutions maintain a significant share, cloud-based options are rapidly gaining traction due to their accessibility, ease of use, and scalability. The enterprise and government segments are key growth drivers, as they require comprehensive solutions for managing large numbers of devices and ensuring data security. Competition in the market is intense, with established players like Norton and Avast alongside numerous smaller, specialized providers. This competitive landscape fosters innovation and drives the development of advanced features, such as real-time protection, performance optimization, and privacy enhancement tools. The market is expected to continue its growth trajectory throughout the forecast period (2025-2033), driven by ongoing technological advancements and the evolving digital landscape. The geographical distribution of the PC cleaner software market is spread across various regions, with North America and Europe currently holding the largest market shares. However, growth potential is significant in emerging markets within Asia-Pacific and the Middle East & Africa, driven by rising internet penetration and increasing PC usage. While factors such as evolving operating system capabilities (inbuilt cleaning utilities) and user awareness of best practices in digital hygiene pose some restraints, the overall market outlook remains positive, with continued growth driven by the persistent need for robust security and system optimization. The market will likely see further consolidation, with larger companies acquiring smaller players to expand their product portfolios and market reach. Focus on developing AI-powered features and proactive threat detection is expected to be a key differentiator in the competitive landscape.

During a 2023 survey carried out among marketing leaders predominantly in consumer packaged goods and retail from North America, the most common driver for clean room strategies were in-depth analytics (named by 56 percent of respondents), ability to measure campaign results (54 percent), and ease of data integration (52 percent). In a different survey, 29 percent of responding U.S. marketers said they would focus more on data clean rooms in 2023 than they had in 2022.

ApacheJIT: A Large Dataset for Just-In-Time Defect Prediction

This archive contains the ApacheJIT dataset presented in the paper "ApacheJIT: A Large Dataset for Just-In-Time Defect Prediction" as well as the replication package. The paper is submitted to MSR 2022 Data Showcase Track.

The datasets are available under directory dataset. There are 4 datasets in this directory.

1. apachejit_total.csv: This file contains the entire dataset. Commits are specified by their identifier and a set of commit metrics that are explained in the paper are provided as features. Column buggy specifies whether or not the commit introduced any bug into the system.
2. apachejit_train.csv: This file is a subset of the entire dataset. It provides a balanced set that we recommend for models that are sensitive to class imbalance. This set is obtained from the first 14 years of data (2003 to 2016).
3. apachejit_test_large.csv: This file is a subset of the entire dataset. The commits in this file are the commits from the last 3 years of data. This set is not balanced to represent a real-life scenario in a JIT model evaluation where the model is trained on historical data to be applied on future data without any modification.
4. apachejit_test_small.csv: This file is a subset of the test file explained above. Since the test file has more than 30,000 commits, we also provide a smaller test set which is still unbalanced and from the last 3 years of data.

In addition to the dataset, we also provide the scripts using which we built the dataset. These scripts are written in Python 3.8. Therefore, Python 3.8 or above is required. To set up the environment, we have provided a list of required packages in file requirements.txt. Additionally, one filtering step requires GumTree [1]. For Java, GumTree requires Java 11. For other languages, external tools are needed. Installation guide and more details can be found here.

The scripts are comprised of Python scripts under directory src and Python notebooks under directory notebooks. The Python scripts are mainly responsible for conducting GitHub search via GitHub search API and collecting commits through PyDriller Package [2]. The notebooks link the fixed issue reports with their corresponding fixing commits and apply some filtering steps. The bug-inducing candidates then are filtered again using gumtree.py script that utilizes the GumTree package. Finally, the remaining bug-inducing candidates are combined with the clean commits in the dataset_construction notebook to form the entire dataset.

More specifically, git_token.py handles GitHub API token that is necessary for requests to GitHub API. Script collector.py performs GitHub search. Tracing changed lines and git annotate is done in gitminer.py using PyDriller. Finally, gumtree.py applies 4 filtering steps (number of lines, number of files, language, and change significance).

References:

1. GumTree

• https://github.com/GumTreeDiff/gumtree

Jean-Rémy Falleri, Floréal Morandat, Xavier Blanc, Matias Martinez, and Martin Monperrus. 2014. Fine-grained and accurate source code differencing. In ACM/IEEE International Conference on Automated Software Engineering, ASE ’14,Vasteras, Sweden - September 15 - 19, 2014. 313–324

1. PyDriller

• https://pydriller.readthedocs.io/en/latest/

• Davide Spadini, Maurício Aniche, and Alberto Bacchelli. 2018. PyDriller: Python Framework for Mining Software Repositories. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering(Lake Buena Vista, FL, USA)(ESEC/FSE2018). Association for Computing Machinery, New York, NY, USA, 908–911

Janitorial Software Market size was valued at USD 2.43 Billion in 2024 and is projected to reach USD 3.45 Billion by 2031, growing at a CAGR of 7.97% during the forecast period 2024-2031.

Global Janitorial Software Market Drivers

Growing Need for Operational Efficiency: Organisations in a variety of sectors are putting more and more emphasis on streamlining their processes in order to increase output and efficiency. With the use of janitorial software, cleaning companies may increase overall productivity, optimise resource allocation, and streamline operations with features like task management, scheduling, and real-time monitoring.

Growing Adoption of Automation and Internet of Things: The janitorial sector is undergoing a transformation thanks to the combination of automation technologies and Internet of Things (IoT) devices. IoT-enabled sensors and devices can record cleaning activities, keep an eye on the operation of equipment, and gather information on how the facility is used. Utilising this data, janitorial software can automate repetitive processes, plan cleanings according to demand, and offer predictive maintenance features, all of which increase productivity and lower costs.

Growing Attention to Maintenance and Facility Management: Building managers are realising more and more how crucial cleanliness and proactive maintenance are to improving tenant happiness, safety, and health. With the help of janitorial software solutions, businesses can keep their surroundings safe, clean, and well-maintained. These solutions include work order administration, asset tracking, and compliance monitoring.

Strict Regulatory Requirements and Compliance Standards: Businesses, especially those in the healthcare, hotel, and food services sectors, are subject to stringent cleaning and hygiene regulations enforced by regulatory agencies and industry standards groups. By streamlining paperwork, audit trails, and reporting, janitorial software assists businesses in adhering to regulations and lowers their risk of fines, penalties, and reputational harm.

Transition to Green Cleaning Methods: As people become more conscious of how conventional cleaning methods and chemicals affect the environment, they are choosing more environmentally friendly and sustainable cleaning products. With the use of janitorial software, businesses may monitor and oversee green cleaning programmes, which include using eco-friendly materials, energy-saving equipment, and waste reduction techniques, in accordance with legal requirements and corporate sustainability objectives.

A Growing Emphasis on Health and Hygiene: The COVID-19 pandemic has increased consciousness regarding the significance of sanitation, hygiene, and disinfection in halting the transmission of infectious illnesses. By adding capabilities like contactless scheduling, touchless workflows, and hygiene compliance monitoring, janitorial software systems have evolved to meet the changing needs of businesses and assist them in keeping a safe and healthy workplace for workers, clients, and guests.

Emergence of Mobile and Cloud Technologies: Real-time access to cleaning data, remote monitoring, and mobile workforce management have all been made possible by the widespread use of mobile devices and cloud computing, which has completely changed the janitorial software market. Cleaning personnel may get assignments, turn in reports, and connect with supervisors from any location with the use of mobile-enabled janitorial apps, which enhances responsiveness, cooperation, and communication.

The JIRA Open-Source Software Effort (JOSSE) dataset consists of software development and maintenance tasks collected from the JIRA issue tracking system for Apache, JBoss, And Spring open-source projects. All the issues were annotated with actual effort and 19% of them were annotated with expert estimates. JOSSE is a task-based dataset with a textual attribute represented as a task description for each data point. This paper explains how the data were collected and details six data quality refinement procedures of the data points.

The Data Preparation Tools market is experiencing robust growth, projected to reach a value of $4.5 billion in 2025 and maintain a Compound Annual Growth Rate (CAGR) of 32.14% from 2025 to 2033. This expansion is fueled by several key drivers. The increasing volume and velocity of data generated by organizations necessitate efficient and automated data preparation processes. Businesses are increasingly adopting cloud-based solutions for data preparation, driven by scalability, cost-effectiveness, and enhanced collaboration capabilities. Furthermore, the rise of self-service data preparation tools empowers business users to directly access and prepare data, reducing reliance on IT departments and accelerating data analysis. The growing adoption of advanced analytics and machine learning initiatives also contributes to market growth, as these technologies require high-quality, prepared data. While the on-premise deployment model still holds a significant share, the cloud segment is expected to witness faster growth due to its inherent advantages. Within the platform segment, both data integration and self-service tools are experiencing strong demand, reflecting the diverse needs of various users and business functions. The competitive landscape is characterized by a mix of established players like Informatica, IBM, and Microsoft, and emerging innovative companies specializing in specific niches. These companies employ various competitive strategies, including product innovation, strategic partnerships, and mergers and acquisitions, to gain market share. Industry risks include the complexity of integrating data preparation tools with existing IT infrastructure, the need for skilled professionals to effectively utilize these tools, and the potential for data security breaches. Geographic growth is expected to be significant across all regions, with North America and Europe maintaining a strong presence due to high adoption rates of advanced technologies. However, the Asia-Pacific region is poised for substantial growth due to rapid technological advancements and increasing data volumes. The historical period (2019-2024) shows a steady increase in market size, providing a strong foundation for the projected future growth. The market is segmented by deployment (on-premise, cloud) and platform (data integration, self-service), reflecting the various approaches to data preparation.

Clean outs are a type of asset that allow access for maintenance purposes to smaller sewer lines which includes both main lines and laterals. Operations staff can use this layer to easily determine where cleaning of some sections of gravity based collections systems will not be possible with their primary equipment and to adjust accordingly. Locations are derived from as-builts and coordination with field staff.Attribute Information:Field Name DescriptionOBJECTIDESRI software specific field that serves as an index for the database.FacilityIDA unique identifier for the asset class. Infor required field.LocationDescriptionInformation related to the construction location or project name. Infor required fieldCommentsA catch all for asset information that is irregular and doesn't warrant the creation of a new field.LastUpdateDate when asset was most recently updated.LastEditorName of user whom most recently edited asset information.EnabledESRI software specific field related to the inclusion in a network.AncillaryRoleESRI software specific field related to the role played within a network.GlobalIDESRI software specific field that is automatically assigned by the geodatabase at row creation.ShapeESRI software specific field denoting the geometry type of the asset.created_userName of user whom created the asset.created_dateDate when the asset was created.last_edited_userName of user whom most recently edited asset information.last_edited_dateDate when asset was most recently updated.IsLocatedHas the location of the asset been field verified with a survey grade GPS unit?InstallDateThe date when the asset was installed. Typically pulled from the as-built cover sheet for consistency. Infor required field.LifecycleStatusThe current status of the asset with respect to its location in the asset management lifecycle. Infor required field.

The size and share of the market is categorized based on Type (Data quality assessment tools, Data cleansing solutions, Data governance platforms, Data monitoring software, Data stewardship tools) and Application (Data cleansing, Data profiling, Data validation, Data enrichment, Data governance) and geographical regions (North America, Europe, Asia-Pacific, South America, and Middle-East and Africa).

Subscribers can find out export and import data of 23 countries by HS code or product’s name. This demo is helpful for market analysis.

This dataset contains program, portfolio, and participant data from the New York State Clean Energy Dashboard (https://www.nyserda.ny.gov/Researchers-and-Policymakers/Clean-Energy-Dashboard/View-the-Dashboard). The Clean Energy Dashboard aggregates budgets and benefits progress data across dozens of programs administered by NYSERDA and utilities. The Clean Energy Dashboard features most of the programs and initiatives that contribute significantly to New York State’s aggressive clean energy goals while tracking progress against both utilities’ and New York State’s targets. The New York State Energy Research and Development Authority (NYSERDA) offers objective information and analysis, innovative programs, technical expertise, and support to help New Yorkers increase energy efficiency, save money, use renewable energy, and reduce reliance on fossil fuels. To learn more about NYSERDA’s programs, visit https://nyserda.ny.gov or follow us on X, Facebook, YouTube, or Instagram.

Lehrmaterial f��r einen Workshop zur Einf��hrung in das Datenbearbeitungstool OpenRefine im Umfang von vier Stunden, in deutscher Sprache. Das Lehrmaterial ist in zwei Teile gegliedert: Teil 1 (eine Stunde) stellt den Funktionsumfang des Datenbearbeitungswerkzeugs OpenRefine vor. Teil 2 (drei Stunden) f��hrt hands-on mittels verschiedener Aufgaben durch diesen Funktionsumfang und erkl��rt die grundlegende Funktionweise der OpenRefine-eigenen Sprache GREL. OpenRefine ist eine Open-Source-Software zur einfachen Manipulation von tabellarischen Daten aus unterschiedlichen Quellen. OpenRefine verf��gt ��ber eine intuitive Benutzeroberfl��che und stellt umfangreiche Funktionen f��r Datenbereinigungen und -transformationen zur Verf��gung. Eine Besonderheit von OpenRefine ist die ��Reconciliation��-Funktion, mit der eigene Daten gegen externe Datenanbieter (z.B. GND, Wikidata, Crossref) gepr��ft und angereichert werden k��nnen. Auch aus diesem Grund wird OpenRefine im bibliothekarischen Umfeld vermehrt eingesetzt. Lernziele: Teilnehmende des ersten Kursteils kennen den Funktionsumfang von OpenRefine und k��nnen ��ber einen m��glichen eigenen Einsatz entscheiden wissen, wo sie weitere Informationen zur Nutzung von OpenRefine erhalten. Teilnehmende des zweiten Kursteils k��nnen Daten laden, sortieren, filtern, bereinigen/transfomieren und exportieren Daten mit externen Reconciliation Services anreichern in der Bearbeitungshistorie vor und zur��ck navigieren und kennen die grundlegende Funktionsweise der General Refine Expression Language (GREL).

86 Global import shipment records of Software Maintenance with prices, volume & current Buyer's suppliers relationships based on actual Global export trade database.

The LSC (Leicester Scientific Corpus)

April 2020 by Neslihan Suzen, PhD student at the University of Leicester (ns433@leicester.ac.uk) Supervised by Prof Alexander Gorban and Dr Evgeny MirkesThe data are extracted from the Web of Science [1]. You may not copy or distribute these data in whole or in part without the written consent of Clarivate Analytics.[Version 2] A further cleaning is applied in Data Processing for LSC Abstracts in Version 1*. Details of cleaning procedure are explained in Step 6.* Suzen, Neslihan (2019): LSC (Leicester Scientific Corpus). figshare. Dataset. https://doi.org/10.25392/leicester.data.9449639.v1.Getting StartedThis text provides the information on the LSC (Leicester Scientific Corpus) and pre-processing steps on abstracts, and describes the structure of files to organise the corpus. This corpus is created to be used in future work on the quantification of the meaning of research texts and make it available for use in Natural Language Processing projects.LSC is a collection of abstracts of articles and proceeding papers published in 2014, and indexed by the Web of Science (WoS) database [1]. The corpus contains only documents in English. Each document in the corpus contains the following parts:1. Authors: The list of authors of the paper2. Title: The title of the paper 3. Abstract: The abstract of the paper 4. Categories: One or more category from the list of categories [2]. Full list of categories is presented in file ‘List_of _Categories.txt’. 5. Research Areas: One or more research area from the list of research areas [3]. Full list of research areas is presented in file ‘List_of_Research_Areas.txt’. 6. Total Times cited: The number of times the paper was cited by other items from all databases within Web of Science platform [4] 7. Times cited in Core Collection: The total number of times the paper was cited by other papers within the WoS Core Collection [4]The corpus was collected in July 2018 online and contains the number of citations from publication date to July 2018. We describe a document as the collection of information (about a paper) listed above. The total number of documents in LSC is 1,673,350.Data ProcessingStep 1: Downloading of the Data Online

The dataset is collected manually by exporting documents as Tab-delimitated files online. All documents are available online.Step 2: Importing the Dataset to R

The LSC was collected as TXT files. All documents are extracted to R.Step 3: Cleaning the Data from Documents with Empty Abstract or without CategoryAs our research is based on the analysis of abstracts and categories, all documents with empty abstracts and documents without categories are removed.Step 4: Identification and Correction of Concatenate Words in AbstractsEspecially medicine-related publications use ‘structured abstracts’. Such type of abstracts are divided into sections with distinct headings such as introduction, aim, objective, method, result, conclusion etc. Used tool for extracting abstracts leads concatenate words of section headings with the first word of the section. For instance, we observe words such as ConclusionHigher and ConclusionsRT etc. The detection and identification of such words is done by sampling of medicine-related publications with human intervention. Detected concatenate words are split into two words. For instance, the word ‘ConclusionHigher’ is split into ‘Conclusion’ and ‘Higher’.The section headings in such abstracts are listed below:

Background Method(s) Design Theoretical Measurement(s) Location Aim(s) Methodology Process Abstract Population Approach Objective(s) Purpose(s) Subject(s) Introduction Implication(s) Patient(s) Procedure(s) Hypothesis Measure(s) Setting(s) Limitation(s) Discussion Conclusion(s) Result(s) Finding(s) Material (s) Rationale(s) Implications for health and nursing policyStep 5: Extracting (Sub-setting) the Data Based on Lengths of AbstractsAfter correction, the lengths of abstracts are calculated. ‘Length’ indicates the total number of words in the text, calculated by the same rule as for Microsoft Word ‘word count’ [5].According to APA style manual [6], an abstract should contain between 150 to 250 words. In LSC, we decided to limit length of abstracts from 30 to 500 words in order to study documents with abstracts of typical length ranges and to avoid the effect of the length to the analysis.

Step 6: [Version 2] Cleaning Copyright Notices, Permission polices, Journal Names and Conference Names from LSC Abstracts in Version 1Publications can include a footer of copyright notice, permission policy, journal name, licence, author’s right or conference name below the text of abstract by conferences and journals. Used tool for extracting and processing abstracts in WoS database leads to attached such footers to the text. For example, our casual observation yields that copyright notices such as ‘Published by Elsevier ltd.’ is placed in many texts. To avoid abnormal appearances of words in further analysis of words such as bias in frequency calculation, we performed a cleaning procedure on such sentences and phrases in abstracts of LSC version 1. We removed copyright notices, names of conferences, names of journals, authors’ rights, licenses and permission policies identified by sampling of abstracts.Step 7: [Version 2] Re-extracting (Sub-setting) the Data Based on Lengths of AbstractsThe cleaning procedure described in previous step leaded to some abstracts having less than our minimum length criteria (30 words). 474 texts were removed.Step 8: Saving the Dataset into CSV FormatDocuments are saved into 34 CSV files. In CSV files, the information is organised with one record on each line and parts of abstract, title, list of authors, list of categories, list of research areas, and times cited is recorded in fields.To access the LSC for research purposes, please email to ns433@le.ac.uk.References[1]Web of Science. (15 July). Available: https://apps.webofknowledge.com/ [2]WoS Subject Categories. Available: https://images.webofknowledge.com/WOKRS56B5/help/WOS/hp_subject_category_terms_tasca.html [3]Research Areas in WoS. Available: https://images.webofknowledge.com/images/help/WOS/hp_research_areas_easca.html [4]Times Cited in WoS Core Collection. (15 July). Available: https://support.clarivate.com/ScientificandAcademicResearch/s/article/Web-of-Science-Times-Cited-accessibility-and-variation?language=en_US [5]Word Count. Available: https://support.office.com/en-us/article/show-word-count-3c9e6a11-a04d-43b4-977c-563a0e0d5da3 [6]A. P. Association, Publication manual. American Psychological Association Washington, DC, 1983.