Facebook
TwitterUnited States agricultural researchers have many options for making their data available online. This dataset aggregates the primary sources of ag-related data and determines where researchers are likely to deposit their agricultural data. These data serve as both a current landscape analysis and also as a baseline for future studies of ag research data. Purpose As sources of agricultural data become more numerous and disparate, and collaboration and open data become more expected if not required, this research provides a landscape inventory of online sources of open agricultural data. An inventory of current agricultural data sharing options will help assess how the Ag Data Commons, a platform for USDA-funded data cataloging and publication, can best support data-intensive and multi-disciplinary research. It will also help agricultural librarians assist their researchers in data management and publication. The goals of this study were to establish where agricultural researchers in the United States-- land grant and USDA researchers, primarily ARS, NRCS, USFS and other agencies -- currently publish their data, including general research data repositories, domain-specific databases, and the top journals compare how much data is in institutional vs. domain-specific vs. federal platforms determine which repositories are recommended by top journals that require or recommend the publication of supporting data ascertain where researchers not affiliated with funding or initiatives possessing a designated open data repository can publish data Approach The National Agricultural Library team focused on Agricultural Research Service (ARS), Natural Resources Conservation Service (NRCS), and United States Forest Service (USFS) style research data, rather than ag economics, statistics, and social sciences data. To find domain-specific, general, institutional, and federal agency repositories and databases that are open to US research submissions and have some amount of ag data, resources including re3data, libguides, and ARS lists were analysed. Primarily environmental or public health databases were not included, but places where ag grantees would publish data were considered. Search methods We first compiled a list of known domain specific USDA / ARS datasets / databases that are represented in the Ag Data Commons, including ARS Image Gallery, ARS Nutrition Databases (sub-components), SoyBase, PeanutBase, National Fungus Collection, i5K Workspace @ NAL, and GRIN. We then searched using search engines such as Bing and Google for non-USDA / federal ag databases, using Boolean variations of “agricultural data” /“ag data” / “scientific data” + NOT + USDA (to filter out the federal / USDA results). Most of these results were domain specific, though some contained a mix of data subjects. We then used search engines such as Bing and Google to find top agricultural university repositories using variations of “agriculture”, “ag data” and “university” to find schools with agriculture programs. Using that list of universities, we searched each university web site to see if their institution had a repository for their unique, independent research data if not apparent in the initial web browser search. We found both ag specific university repositories and general university repositories that housed a portion of agricultural data. Ag specific university repositories are included in the list of domain-specific repositories. Results included Columbia University – International Research Institute for Climate and Society, UC Davis – Cover Crops Database, etc. If a general university repository existed, we determined whether that repository could filter to include only data results after our chosen ag search terms were applied. General university databases that contain ag data included Colorado State University Digital Collections, University of Michigan ICPSR (Inter-university Consortium for Political and Social Research), and University of Minnesota DRUM (Digital Repository of the University of Minnesota). We then split out NCBI (National Center for Biotechnology Information) repositories. Next we searched the internet for open general data repositories using a variety of search engines, and repositories containing a mix of data, journals, books, and other types of records were tested to determine whether that repository could filter for data results after search terms were applied. General subject data repositories include Figshare, Open Science Framework, PANGEA, Protein Data Bank, and Zenodo. Finally, we compared scholarly journal suggestions for data repositories against our list to fill in any missing repositories that might contain agricultural data. Extensive lists of journals were compiled, in which USDA published in 2012 and 2016, combining search results in ARIS, Scopus, and the Forest Service's TreeSearch, plus the USDA web sites Economic Research Service (ERS), National Agricultural Statistics Service (NASS), Natural Resources and Conservation Service (NRCS), Food and Nutrition Service (FNS), Rural Development (RD), and Agricultural Marketing Service (AMS). The top 50 journals' author instructions were consulted to see if they (a) ask or require submitters to provide supplemental data, or (b) require submitters to submit data to open repositories. Data are provided for Journals based on a 2012 and 2016 study of where USDA employees publish their research studies, ranked by number of articles, including 2015/2016 Impact Factor, Author guidelines, Supplemental Data?, Supplemental Data reviewed?, Open Data (Supplemental or in Repository) Required? and Recommended data repositories, as provided in the online author guidelines for each the top 50 journals. Evaluation We ran a series of searches on all resulting general subject databases with the designated search terms. From the results, we noted the total number of datasets in the repository, type of resource searched (datasets, data, images, components, etc.), percentage of the total database that each term comprised, any dataset with a search term that comprised at least 1% and 5% of the total collection, and any search term that returned greater than 100 and greater than 500 results. We compared domain-specific databases and repositories based on parent organization, type of institution, and whether data submissions were dependent on conditions such as funding or affiliation of some kind. Results A summary of the major findings from our data review: Over half of the top 50 ag-related journals from our profile require or encourage open data for their published authors. There are few general repositories that are both large AND contain a significant portion of ag data in their collection. GBIF (Global Biodiversity Information Facility), ICPSR, and ORNL DAAC were among those that had over 500 datasets returned with at least one ag search term and had that result comprise at least 5% of the total collection. Not even one quarter of the domain-specific repositories and datasets reviewed allow open submission by any researcher regardless of funding or affiliation. See included README file for descriptions of each individual data file in this dataset. Resources in this dataset:Resource Title: Journals. File Name: Journals.csvResource Title: Journals - Recommended repositories. File Name: Repos_from_journals.csvResource Title: TDWG presentation. File Name: TDWG_Presentation.pptxResource Title: Domain Specific ag data sources. File Name: domain_specific_ag_databases.csvResource Title: Data Dictionary for Ag Data Repository Inventory. File Name: Ag_Data_Repo_DD.csvResource Title: General repositories containing ag data. File Name: general_repos_1.csvResource Title: README and file inventory. File Name: README_InventoryPublicDBandREepAgData.txt
Facebook
TwitterCC0 1.0 Universal Public Domain Dedicationhttps://creativecommons.org/publicdomain/zero/1.0/
License information was derived automatically
Facebook
Twitterhttps://www.technavio.com/content/privacy-noticehttps://www.technavio.com/content/privacy-notice
Facebook
TwitterAttribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
The Ultimate Arabic News Dataset is a collection of single-label modern Arabic texts that are used in news websites and press articles.
Arabic news data was collected by web scraping techniques from many famous news sites such as Al-Arabiya, Al-Youm Al-Sabea (Youm7), the news published on the Google search engine and other various sources.
UltimateArabic: A file containing more than 193,000 original Arabic news texts, without pre-processing. The texts contain words, numbers, and symbols that can be removed using pre-processing to increase accuracy when using the dataset in various Arabic natural language processing tasks such as text classification.
UltimateArabicPrePros: It is a file that contains the data mentioned in the first file, but after pre-processing, where the number of data became about 188,000 text documents, where stop words, non-Arabic words, symbols and numbers have been removed so that this file is ready for use directly in the various Arabic natural language processing tasks. Like text classification.
Sample_Youm7_Politic: An example of news in the "Politic" category collected from the Youm7 website.
Sample_alarabiya_Sport: An example of news in the "Sport" category collected from the Al-Arabiya website.
Facebook
Twitterhttps://www.wiseguyreports.com/pages/privacy-policyhttps://www.wiseguyreports.com/pages/privacy-policy
| BASE YEAR | 2024 |
| HISTORICAL DATA | 2019 - 2023 |
| REGIONS COVERED | North America, Europe, APAC, South America, MEA |
| REPORT COVERAGE | Revenue Forecast, Competitive Landscape, Growth Factors, and Trends |
| MARKET SIZE 2024 | 2.18(USD Billion) |
| MARKET SIZE 2025 | 2.35(USD Billion) |
| MARKET SIZE 2035 | 5.0(USD Billion) |
| SEGMENTS COVERED | Application, Deployment Type, End User, Component, Regional |
| COUNTRIES COVERED | US, Canada, Germany, UK, France, Russia, Italy, Spain, Rest of Europe, China, India, Japan, South Korea, Malaysia, Thailand, Indonesia, Rest of APAC, Brazil, Mexico, Argentina, Rest of South America, GCC, South Africa, Rest of MEA |
| KEY MARKET DYNAMICS | Increasing semantic data usage, Growth of linked data technologies, Demand for interoperability solutions, Rise in AI and ML applications, Need for efficient data integration |
| MARKET FORECAST UNITS | USD Billion |
| KEY COMPANIES PROFILED | IBM, Linked Data Company, Oracle, TopQuadrant, Neo4j, RDFLib, GraphDB, Apache Software Foundation, SAP, Cambridge Semantics, Microsoft, Ontotext, MarkLogic, Amazon, Google, Stardog |
| MARKET FORECAST PERIOD | 2025 - 2035 |
| KEY MARKET OPPORTUNITIES | Increased demand for data integration, Growth in semantic web applications, Rise in AI and machine learning, Expansion of connected data ecosystems, Adoption in healthcare and life sciences |
| COMPOUND ANNUAL GROWTH RATE (CAGR) | 7.8% (2025 - 2035) |
Facebook
Twitter
As per our latest research, the global personal data removal services market size reached USD 1.65 billion in 2024, reflecting a robust momentum driven by increasing privacy concerns and stringent data protection regulations worldwide. The market is exhibiting a strong CAGR of 18.2% and is forecasted to reach USD 8.17 billion by 2033. This remarkable growth is underpinned by rising consumer awareness about digital footprints, a surge in cyber threats, and evolving regulatory frameworks mandating stricter data privacy compliance for organizations and individuals alike.
The primary growth driver for the personal data removal services market is the exponential rise in data breaches and cyber-attacks, which have significantly heightened the need for robust data privacy solutions. Organizations and individuals are increasingly realizing the risks associated with personal data exposure, including identity theft, financial loss, and reputational damage. This awareness, coupled with the proliferation of digital platforms and social networks, has led to a higher demand for services that can effectively remove or anonymize personal information from online databases, directories, and search engines. Businesses are also leveraging these services to comply with regulations such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA), which enforce strict guidelines for data handling and deletion, further accelerating market adoption.
Technological advancements and the integration of artificial intelligence (AI) and machine learning (ML) have also catalyzed the growth of the personal data removal services market. AI-powered solutions enable automated identification and removal of personal information across a multitude of online sources, enhancing efficiency and accuracy. These innovations have made it easier for service providers to offer scalable, customizable, and cost-effective solutions to a diverse client base. As the digital ecosystem continues to expand with the Internet of Things (IoT), wearable devices, and smart applications, the volume of personal data generated and stored online is expected to surge, further fueling the demand for comprehensive data removal services.
Another significant growth factor is the rising emphasis on consumer rights and transparency in data usage. Governments and regulatory bodies across the globe are enacting new laws and updating existing frameworks to empower individuals with greater control over their personal data. This shift is compelling organizations to adopt personal data removal services as a proactive measure to build trust with customers, avoid legal penalties, and maintain a positive brand image. The increasing adoption of remote work and digital transactions in the post-pandemic era has also contributed to the marketÂ’s expansion, as both enterprises and individuals seek to safeguard sensitive information from unauthorized access.
Takedown Services have become increasingly vital in the realm of personal data removal, providing a crucial layer of protection against unauthorized online content. These services specialize in identifying and removing unwanted or harmful information from the internet, which can include anything from defamatory content to unauthorized personal data. As the digital landscape becomes more complex, individuals and businesses are finding it challenging to manage their online presence effectively. Takedown Services offer a proactive approach to maintaining privacy and reputation by swiftly addressing potential threats. By collaborating with legal experts and leveraging advanced technology, these services ensure that sensitive information is not only removed but also prevented from resurfacing, thereby offering peace of mind to clients concerned about their digital footprint.
Regionally, North America continues to dominate the personal data removal services market, accounting for the largest share in 2024, followed by Europe and the Asia Pacific. The presence of leading technology firms, early adoption of privacy-centric solutions, and stringent regulatory requirements have cemented North America's leadership position. EuropeÂ’s growth is propelled by the enforcement of GDPR and a strong culture of data privacy, while the Asia Pacific is witnessing rapid expansion due to
Facebook
TwitterA set of online services created in support of MIRIAM, a set of guidelines for the annotation and curation of computational models. The core of MIRIAM Resources is a catalogue of data types (namespaces corresponding to controlled vocabularies or databases), their URIs and the corresponding physical URLs or resources. Access to this data is made available via exports (XML) and Web Services (SOAP). MIRIAM Resources are developed and maintained under the BioModels.net initiative, and are free for use by all. MIRIAM Resources are composed of four components: a database, some Web Services, a Java library and this web application. * Database: The core of the system is a MySQL database. It allows us to store the data types (which can be controlled vocabularies or databases), their URIs and the corresponding physical URLs, and other details such as documentation and resource identifier patterns. Each entry contains a diverse set of details about the data type: official name and synonyms, root URI, pattern of identifiers, documentation, etc. Moreover, each data type can be associated with several resources (or physical locations). * Web Services: Programmatic access to the data is available via Web Services (based on Apache Axis and SOAP messages). In addition, REST-based services are currently being developed. This API allows one to not only resolve model annotations, but also to generate appropriate URIs, based upon the provision of a resource name and accession number. A list of available web services, and a WSDL are provided. A browser-based online demonstration of the Web Services is also available to try. * Java Library: A Java library is provided to access the Web Services. The documentation explains where to download it, its dependencies, and how to use it. * Web Application: A Web application, using an Apache Tomcat server, offers access to the whole data set via a Web browser. It is possible to browse by data type names as well as browse by tags. A search engine is also provided.
Facebook
Twitterhttps://www.technavio.com/content/privacy-noticehttps://www.technavio.com/content/privacy-notice
Facebook
TwitterAttribution-NonCommercial-NoDerivs 4.0 (CC BY-NC-ND 4.0)https://creativecommons.org/licenses/by-nc-nd/4.0/
License information was derived automatically
This dataset supplements publication "Multilingual Scraper of Privacy Policies and Terms of Service" at ACM CSLAW’25, March 25–27, 2025, München, Germany. It includes the first 12 months of scraped policies and terms from about 800k websites, see concrete numbers below.
The following table lists the amount of websites visited per month:
| Month | Number of websites |
|---|---|
| 2024-01 | 551'148 |
| 2024-02 | 792'921 |
| 2024-03 | 844'537 |
| 2024-04 | 802'169 |
| 2024-05 | 805'878 |
| 2024-06 | 809'518 |
| 2024-07 | 811'418 |
| 2024-08 | 813'534 |
| 2024-09 | 814'321 |
| 2024-10 | 817'586 |
| 2024-11 | 828'662 |
| 2024-12 | 827'101 |
The amount of websites visited should always be higher than the number of jobs (Table 1 of the paper) as a website may redirect, resulting in two websites scraped or it has to be retried.
To simplify the access, we release the data in large CSVs. Namely, there is one file for policies and another for terms per month. All of these files contain all metadata that are usable for the analysis. If your favourite CSV parser reports the same numbers as above then our dataset is correctly parsed. We use ‘,’ as a separator, the first row is the heading and strings are in quotes.
Since our scraper sometimes collects other documents than policies and terms (for how often this happens, see the evaluation in Sec. 4 of the publication) that might contain personal data such as addresses of authors of websites that they maintain only for a selected audience. We therefore decided to reduce the risks for websites by anonymizing the data using Presidio. Presidio substitutes personal data with tokens. If your personal data has not been effectively anonymized from the database and you wish for it to be deleted, please contact us.
The uncompressed dataset is about 125 GB in size, so you will need sufficient storage. This also means that you likely cannot process all the data at once in your memory, so we split the data in months and in files for policies and terms.
The files have the following names:
Both files contain the following metadata columns:
website_month_id - identification of crawled websitejob_id - one website can have multiple jobs in case of redirects (but most commonly has only one)website_index_status - network state of loading the index page. This is resolved by the Chromed DevTools Protocol.
DNS_ERROR - domain cannot be resolvedOK - all fineREDIRECT - domain redirect to somewhere elseTIMEOUT - the request timed outBAD_CONTENT_TYPE - 415 Unsupported Media TypeHTTP_ERROR - 404 errorTCP_ERROR - error in the network connectionUNKNOWN_ERROR - unknown errorwebsite_lang - language of index page detected based on langdetect librarywebsite_url - the URL of the website sampled from the CrUX list (may contain subdomains, etc). Use this as a unique identifier for connecting data between months.job_domain_status - indicates the status of loading the index page. Can be:
OK - all works well (at the moment, should be all entries)BLACKLISTED - URL is on our list of blocked URLsUNSAFE - website is not safe according to save browsing API by GoogleLOCATION_BLOCKED - country is in the list of blocked countriesjob_started_at - when the visit of the website was startedjob_ended_at - when the visit of the website was endedjob_crux_popularity - JSON with all popularity ranks of the website this monthjob_index_redirect - when we detect that the domain redirects us, we stop the crawl and create a new job with the target URL. This saves time if many websites redirect to one target, as it will be crawled only once. The index_redirect is then the job.id corresponding to the redirect target.job_num_starts - amount of crawlers that started this job (counts restarts in case of unsuccessful crawl, max is 3)job_from_static - whether this job was included in the static selection (see Sec. 3.3 of the paper)job_from_dynamic - whether this job was included in the dynamic selection (see Sec. 3.3 of the paper) - this is not exclusive with from_static - both can be true when the lists overlap.job_crawl_name - our name of the crawl, contains year and month (e.g., 'regular-2024-12' for regular crawls, in Dec 2024)policy_url_id - ID of the URL this policy haspolicy_keyword_score - score (higher is better) according to the crawler's keywords list that given document is a policypolicy_ml_probability - probability assigned by the BERT model that given document is a policypolicy_consideration_basis - on which basis we decided that this url is policy. The following three options are executed by the crawler in this order:
policy_url - full URL to the policypolicy_content_hash - used as identifier - if the document remained the same between crawls, it won't create a new entrypolicy_content - contains the text of policies and terms extracted to Markdown using Mozilla's readability librarypolicy_lang - Language detected by fasttext of the contentAnalogous to policy data, just substitute policy to terms.
Check this Google Docs for an updated version of this README.md.
Not seeing a result you expected?
Learn how you can add new datasets to our index.
Facebook
TwitterUnited States agricultural researchers have many options for making their data available online. This dataset aggregates the primary sources of ag-related data and determines where researchers are likely to deposit their agricultural data. These data serve as both a current landscape analysis and also as a baseline for future studies of ag research data. Purpose As sources of agricultural data become more numerous and disparate, and collaboration and open data become more expected if not required, this research provides a landscape inventory of online sources of open agricultural data. An inventory of current agricultural data sharing options will help assess how the Ag Data Commons, a platform for USDA-funded data cataloging and publication, can best support data-intensive and multi-disciplinary research. It will also help agricultural librarians assist their researchers in data management and publication. The goals of this study were to establish where agricultural researchers in the United States-- land grant and USDA researchers, primarily ARS, NRCS, USFS and other agencies -- currently publish their data, including general research data repositories, domain-specific databases, and the top journals compare how much data is in institutional vs. domain-specific vs. federal platforms determine which repositories are recommended by top journals that require or recommend the publication of supporting data ascertain where researchers not affiliated with funding or initiatives possessing a designated open data repository can publish data Approach The National Agricultural Library team focused on Agricultural Research Service (ARS), Natural Resources Conservation Service (NRCS), and United States Forest Service (USFS) style research data, rather than ag economics, statistics, and social sciences data. To find domain-specific, general, institutional, and federal agency repositories and databases that are open to US research submissions and have some amount of ag data, resources including re3data, libguides, and ARS lists were analysed. Primarily environmental or public health databases were not included, but places where ag grantees would publish data were considered. Search methods We first compiled a list of known domain specific USDA / ARS datasets / databases that are represented in the Ag Data Commons, including ARS Image Gallery, ARS Nutrition Databases (sub-components), SoyBase, PeanutBase, National Fungus Collection, i5K Workspace @ NAL, and GRIN. We then searched using search engines such as Bing and Google for non-USDA / federal ag databases, using Boolean variations of “agricultural data” /“ag data” / “scientific data” + NOT + USDA (to filter out the federal / USDA results). Most of these results were domain specific, though some contained a mix of data subjects. We then used search engines such as Bing and Google to find top agricultural university repositories using variations of “agriculture”, “ag data” and “university” to find schools with agriculture programs. Using that list of universities, we searched each university web site to see if their institution had a repository for their unique, independent research data if not apparent in the initial web browser search. We found both ag specific university repositories and general university repositories that housed a portion of agricultural data. Ag specific university repositories are included in the list of domain-specific repositories. Results included Columbia University – International Research Institute for Climate and Society, UC Davis – Cover Crops Database, etc. If a general university repository existed, we determined whether that repository could filter to include only data results after our chosen ag search terms were applied. General university databases that contain ag data included Colorado State University Digital Collections, University of Michigan ICPSR (Inter-university Consortium for Political and Social Research), and University of Minnesota DRUM (Digital Repository of the University of Minnesota). We then split out NCBI (National Center for Biotechnology Information) repositories. Next we searched the internet for open general data repositories using a variety of search engines, and repositories containing a mix of data, journals, books, and other types of records were tested to determine whether that repository could filter for data results after search terms were applied. General subject data repositories include Figshare, Open Science Framework, PANGEA, Protein Data Bank, and Zenodo. Finally, we compared scholarly journal suggestions for data repositories against our list to fill in any missing repositories that might contain agricultural data. Extensive lists of journals were compiled, in which USDA published in 2012 and 2016, combining search results in ARIS, Scopus, and the Forest Service's TreeSearch, plus the USDA web sites Economic Research Service (ERS), National Agricultural Statistics Service (NASS), Natural Resources and Conservation Service (NRCS), Food and Nutrition Service (FNS), Rural Development (RD), and Agricultural Marketing Service (AMS). The top 50 journals' author instructions were consulted to see if they (a) ask or require submitters to provide supplemental data, or (b) require submitters to submit data to open repositories. Data are provided for Journals based on a 2012 and 2016 study of where USDA employees publish their research studies, ranked by number of articles, including 2015/2016 Impact Factor, Author guidelines, Supplemental Data?, Supplemental Data reviewed?, Open Data (Supplemental or in Repository) Required? and Recommended data repositories, as provided in the online author guidelines for each the top 50 journals. Evaluation We ran a series of searches on all resulting general subject databases with the designated search terms. From the results, we noted the total number of datasets in the repository, type of resource searched (datasets, data, images, components, etc.), percentage of the total database that each term comprised, any dataset with a search term that comprised at least 1% and 5% of the total collection, and any search term that returned greater than 100 and greater than 500 results. We compared domain-specific databases and repositories based on parent organization, type of institution, and whether data submissions were dependent on conditions such as funding or affiliation of some kind. Results A summary of the major findings from our data review: Over half of the top 50 ag-related journals from our profile require or encourage open data for their published authors. There are few general repositories that are both large AND contain a significant portion of ag data in their collection. GBIF (Global Biodiversity Information Facility), ICPSR, and ORNL DAAC were among those that had over 500 datasets returned with at least one ag search term and had that result comprise at least 5% of the total collection. Not even one quarter of the domain-specific repositories and datasets reviewed allow open submission by any researcher regardless of funding or affiliation. See included README file for descriptions of each individual data file in this dataset. Resources in this dataset:Resource Title: Journals. File Name: Journals.csvResource Title: Journals - Recommended repositories. File Name: Repos_from_journals.csvResource Title: TDWG presentation. File Name: TDWG_Presentation.pptxResource Title: Domain Specific ag data sources. File Name: domain_specific_ag_databases.csvResource Title: Data Dictionary for Ag Data Repository Inventory. File Name: Ag_Data_Repo_DD.csvResource Title: General repositories containing ag data. File Name: general_repos_1.csvResource Title: README and file inventory. File Name: README_InventoryPublicDBandREepAgData.txt