Open Government Data (OGD) has the potential to support social and economic progress. However, this potential can be frustrated if this data remains unused. Although the literature suggests that OGD datasets' metadata quality is one of the main factors affecting their use, to the best of our knowledge, no quantitative study provided evidence of this relationship. Considering about 400,000 datasets of 28 national, municipal, and international OGD portals, we have programmatically analyzed their usage, their metadata quality, and the relationship between the two. Our analysis has highlighted three main findings. First of all, regardless of their size, the software platform adopted, and their administrative and territorial coverage, most OGD datasets are underutilized. Second, OGD portals pay varying attention to the quality of their datasets’ metadata. Third, we did not find clear evidence that datasets usage is positively correlated to better metadata publishing practices. Finally, we have considered other factors, such as datasets’ category, and some demographic characteristics of the OGD portals, and analyzed their relationship with datasets usage, obtaining partially affirmative answers.

The dataset consists of three zipped CSV files, containing the collected datasets' usage data, full metadata, and computed quality values, for about 400,000 datasets belonging to the 8 national, 4 international, and 16 US municipalities OGD portals considered in the study.

Data collection occurred in the period: 2019-12-19 -- 2019-12-23.

_

Portal #Datasets Platform

_

US 261,514 CKAN

France 39,412 Other

Colombia 9,795 Socrata

IE 9,598 CKAN

Slovenia 4,892 CKAN

Poland 1,032 Other

Latvia 336 CKAN

Puerto Rico 178 Socrata

New York, NY 2,771 Socrata

Baltimore, MD 2,617 Socrata

Austin, TX 2,353 Socrata

Chicago, IL 1,368 Socrata

San Francisco, CA 1,001 Socrata

Dallas, TX 1,001 Socrata

Los Angeles, CA 943 Socrata

Seattle, WA 718 Socrata

Providence, RI 288 Socrata

Honolulu, HI 244 Socrata

New Orleans, LA 215 Socrata

Buffalo, NY 213 Socrata

Nashville, TN 172 Socrata

Boston, MA 170 CKAN

Albuquerque, NM 60 CKAN

Albany, NY 50 Socrata

HDX 17,325 CKAN

EUODP 14,058 CKAN

NASA 9,664 Socrata

World Bank Finances 2,177 Socrata

_

The three datasets share the same table structure:

Table Fields

• portalid: portal identifier
• id: dataset identifier
• engine: identifier of the supporting portal platform: 1(CKAN), 2 (Socrata)
• admindomain: 1 (National), 2 (US), 3 (International)
• downloaddate: date of data collection
• views: number of total views for the dataset
• downloads: number of total downloads for the dataset
• overallq: overall quality values computed by applying the methodology presented by Neumaier et al. in [1]
• qvalues: json object containing the quality values computed for the 17 metrics presented in by Neumaier et al. [1]
• assessdate: date of quality assessment
• metadata: the overall dataset's metadata downloaded via API from the portal according to the supporting platform schema

[1] Neumaier, S.; Umbrich, J.; Polleres, A. Automated Quality Assessment of Metadata Across Open Data Portals.J. Data and Information Quality2016,8, 2:1–2:29. doi:10.1145/2964909

This statistic displays the findings of a survey on the awareness of data collection by government agencies for the purpose of national security in the European Union (EU-28) as of March 2015. During the survey, it was found that 49 percent of respondents had not heard of revelations about such data collection.

Government Open Data Management Platform Market Outlook

The global government open data management platform market size was valued at USD 2.5 billion in 2023 and is projected to reach USD 6.7 billion by 2032, growing at a compound annual growth rate (CAGR) of 11.5% during the forecast period. The rising emphasis on transparency, accountability, and citizen engagement by governments worldwide is a significant driving factor for this market's growth.

The proliferation of digital governance initiatives is one of the primary growth factors for the government open data management platform market. Governments across the globe are increasingly adopting digital platforms to improve public service delivery, enhance citizen engagement, and increase operational efficiency. By providing open access to data, these platforms enable better decision-making and foster innovation among various stakeholders, including businesses, researchers, and the general public. This trend is further accelerated by the growing demand for data-driven governance and public policies that are more responsive and accountable.

Moreover, advancements in data analytics and artificial intelligence (AI) are significantly contributing to the growth of the government open data management platform market. Modern open data platforms are increasingly incorporating sophisticated analytics tools and AI capabilities to offer more insightful and actionable data. These technological advancements enable governments to leverage large datasets for predictive analytics, enhancing their ability to anticipate and respond to public needs effectively. Additionally, the integration of AI in data management platforms helps in automating data processing tasks, thereby improving efficiency and reducing operational costs.

The increasing focus on smart city initiatives is another critical factor driving the demand for government open data management platforms. Smart cities rely heavily on data to optimize urban planning, improve traffic management, enhance public safety, and provide efficient public services. Open data platforms play a crucial role in these initiatives by providing a centralized repository for diverse data sets collected from various sensors and systems across the city. This data can be accessed and analyzed by different stakeholders to develop innovative solutions that address urban challenges and improve the quality of life for citizens.

Government Software plays a pivotal role in the development and implementation of open data management platforms. These software solutions are designed to meet the specific needs of government agencies, providing robust tools for data collection, analysis, and dissemination. By leveraging government software, agencies can ensure data accuracy, enhance transparency, and improve public service delivery. The integration of advanced features such as data visualization, predictive analytics, and machine learning within government software allows for more informed decision-making and policy formulation. As governments continue to prioritize digital transformation, the demand for specialized government software solutions is expected to rise, driving further growth in the open data management platform market.

From a regional perspective, North America holds a significant share of the government open data management platform market, driven by the early adoption of digital governance solutions and the presence of major technology providers in the region. Europe is also a prominent market, with several countries implementing open data policies to promote transparency and citizen participation. The Asia Pacific region is expected to witness substantial growth during the forecast period, supported by increasing government initiatives to digitize public services and the rising adoption of smart city projects. Latin America, the Middle East, and Africa are also anticipated to show promising growth, although at a comparatively slower pace due to varying levels of technological infrastructure and government investment in these regions.

Component Analysis

The government open data management platform market is segmented by component into software and services. Software components include the core data management platforms, which facilitate the collection, storage, and dissemination of open data. These software solutions are designed to handle large volumes of data and provide various functionalities such as data analytics, visualization, and integration. The increasi

This statistic displays the findings of a survey on the impact of online data collection by government agencies on residents' trust in the European Union (EU-28) in March 2015. During the survey, it was found that 11 percent of respondents reported that such data collection by government agencies had a positive impact on their trust in how their online personal data is used.

The United States Census Bureau conducts a Census of Governments every five years -- in years ending in "2" or "7" -- to collect information about governments in the United States. The Government Organization branch of the 1997 Census of Governments describes the organization and activities of local governments. The 1997 Local Government Directory Survey covered all county, municipal, town or township, school district, special district governments, school systems, and education service agencies that met the Census Bureau criteria for independent governments. The counts of local governments reflect those in operation in June 1997. This collection includes eight parts, each including information regarding a different type of government: (1) county governments, (2) municipal governments, (3) township governments, (4) special district governments, (5) school district governments, (6) state dependent school systems, (7) local dependent school systems, and (8) education service agencies. The data include information on various codes used to identify the government unit, government name, population in 1996 (or enrollment in 1996 for data collected from schools), and government functions.

Data file combines ESS Cumulative Data (Rounds 1-8) available here: https://www.europeansocialsurvey.org/downloadwizard/ and ParlGov data about (a) date of the national (legislative/parliamentary/lower chamber) elections and (b) government replacement dates in countries included in ESS both available here: http://www.parlgov.org/data/table/view_cabinet/ Note: ParlGov does not include information for Russia and Ukraine (both included in ESS).

This dataset contains the results of a survey about the use of open government data applied to public agents working in public institutions in Brazil. It has two sets, one with questionnaire responses and metadata and the second with a coding table with interview extracts: 1) In the first dataset, each row holds a response to a questionnaire about the public agent's perceptions of the use and reuse of open government data in Brazilian public institutions. Columns store the questionnaire questions. Data were collected between 8 June and 13 July 2021, and this sample is composed of responses from 40 federal, state, and municipal public administrators. Thus, this dataset contains 40 rows and 158 columns. Data were collected on the LimeSurvey platform, where it was screened for missing values and incomplete responses. After cleaning, data were exported to Excel in tabular format. Questionnaire responses are provided in two files ResultsSurvey_OGDUseBRPubInstitutions_DataSet_PT and ResultsSurvey_OGDUseBRPubInstitutions_DataSet_EN. They contain the same information in Portuguese and English. 2) The second dataset records the code table of the interviews about the benefits, barriers, enablers, and drivers of open government data (OGD) use in Brazilian public institutions. A questionnaire applied to public agents working in Brazilian public institutions was followed up by interviews to broaden an understanding of the use of OGD. Nine interviews were conducted between May 17-31, 2022. This dataset represents the perspective of these public agents. The dataset contains 97 lines and six columns. Each row of the dataset lists the factor code used in the questionnaire, the factor descriptions in Portuguese and English, the interviewee code, the transcription extract of an interviewee narration collected in Portuguese, and the English translation. After collection in Portuguese, interviews were automatically transcribed using the NVivo Transcription software. Then, they were anonymized, and a human reviewed the transcriptions. Interviews were coded using NVivo and used the questionnaire factors to guide coding. Coded extracts were translated to English using Google and Microsoft translators. Then, translated extracts were revised by a human and were used for reporting. The coding table was exported to Excel. Interviews extracts are provided in one file, InterviewsExtracts_OGDUseBR_PublicInstitutions_Dataset.

This dataset contains data collected during a study "Transparency of open data ecosystems in smart cities: Definition and assessment of the maturity of transparency in 22 smart cities" (Sustainable Cities and Society (SCS), vol.82, 103906) conducted by Martin Lnenicka (University of Pardubice), Anastasija Nikiforova (University of Tartu), Mariusz Luterek (University of Warsaw), Otmane Azeroual (German Centre for Higher Education Research and Science Studies), Dandison Ukpabi (University of Jyväskylä), Visvaldis Valtenbergs (University of Latvia), Renata Machova (University of Pardubice).

This study inspects smart cities’ data portals and assesses their compliance with transparency requirements for open (government) data by means of the expert assessment of 34 portals representing 22 smart cities, with 36 features.

It being made public both to act as supplementary data for the paper and in order for other researchers to use these data in their own work potentially contributing to the improvement of current data ecosystems and build sustainable, transparent, citizen-centered, and socially resilient open data-driven smart cities.

Purpose of the expert assessment The data in this dataset were collected in the result of the applying the developed benchmarking framework for assessing the compliance of open (government) data portals with the principles of transparency-by-design proposed by Lněnička and Nikiforova (2021)* to 34 portals that can be considered to be part of open data ecosystems in smart cities, thereby carrying out their assessment by experts in 36 features context, which allows to rank them and discuss their maturity levels and (4) based on the results of the assessment, defining the components and unique models that form the open data ecosystem in the smart city context.

Methodology Sample selection: the capitals of the Member States of the European Union and countries of the European Economic Area were selected to ensure a more coherent political and legal framework. They were mapped/cross-referenced with their rank in 5 smart city rankings: IESE Cities in Motion Index, Top 50 smart city governments (SCG), IMD smart city index (SCI), global cities index (GCI), and sustainable cities index (SCI). A purposive sampling method and systematic search for portals was then carried out to identify relevant websites for each city using two complementary techniques: browsing and searching. To evaluate the transparency maturity of data ecosystems in smart cities, we have used the transparency-by-design framework (Lněnička & Nikiforova, 2021)*. The benchmarking supposes the collection of quantitative data, which makes this task an acceptability task. A six-point Likert scale was applied for evaluating the portals. Each sub-dimension was supplied with its description to ensure the common understanding, a drop-down list to select the level at which the respondent (dis)agree, and a comment to be provided, which has not been mandatory. This formed a protocol to be fulfilled on every portal. Each sub-dimension/feature was assessed using a six-point Likert scale, where strong agreement is assessed with 6 points, while strong disagreement is represented by 1 point. Each website (portal) was evaluated by experts, where a person is considered to be an expert if a person works with open (government) data and data portals daily, i.e., it is the key part of their job, which can be public officials, researchers, and independent organizations. In other words, compliance with the expert profile according to the International Certification of Digital Literacy (ICDL) and its derivation proposed in Lněnička et al. (2021)* is expected to be met. When all individual protocols were collected, mean values and standard deviations (SD) were calculated, and if statistical contradictions/inconsistencies were found, reassessment took place to ensure individual consistency and interrater reliability among experts’ answers. *Lnenicka, M., & Nikiforova, A. (2021). Transparency-by-design: What is the role of open data portals?. Telematics and Informatics, 61, 101605 *Lněnička, M., Machova, R., Volejníková, J., Linhartová, V., Knezackova, R., & Hub, M. (2021). Enhancing transparency through open government data: the case of data portals and their features and capabilities. Online Information Review.

Test procedure (1) perform an assessment of each dimension using sub-dimensions, mapping out the achievement of each indicator (2) all sub-dimensions in one dimension are aggregated, and then the average value is calculated based on the number of sub-dimensions – the resulting average stands for a dimension value - eight values per portal (3) the average value from all dimensions are calculated and then mapped to the maturity level – this value of each portal is also used to rank the portals.

Description of the data in this data set Sheet#1 "comparison_overall" provides results by portal Sheet#2 "comparison_category" provides results by portal and category Sheet#3 "category_subcategory" provides list of categories and its elements

Format of the file .xls

Licenses or restrictions CC-BY

For more info, see README.txt

As of 2024, most of the government entities engaged in data collection efforts in Saudi Arabia collected datasets smaller than *** gigabytes, making up around ** percent of the total. In comparison, the share of such government entities that have collected datasets exceeding ************ was about ** percent as of the same year.

Accession / Acquisition data details groups of records that are transferred from City agencies and donors through a records management disposition process. It typically includes the originating agency/donor and a general summary on the content, formats, and size of the collections. Information is also available at: https://a860-collectionguides.nyc.gov/

(1) The Government Research Bulletin (GRB) contains research project data from government agencies since 1982, and provides query, browsing, and full-text download services for publicly accessible research project data. (2) The intellectual property of the included research reports belongs to the agencies or project leaders. For further understanding of the report content or providing improvement suggestions, please contact the organizing agency or project leader directly.

This data collection examines the effects of organizational environment and funding level on the utility of criminal justice research projects sponsored by the National Institute of Justice (NIJ). The data represent a unique source of information on factors that influence the quality and utility of criminal justice research. Variables describing the research grants include NIJ office responsible for monitoring the grant (e.g., courts, police, corrections, etc.), organization type receiving the grant (academic or nonacademic), type of data (collected originally, existing, merged), and priority area (crime, victims, parole, police). The studies are also classified by: (1) sampling method employed, (2) presentation style, (3) statistical analysis employed, (4) type of research design, (5) number of observation points, and (6) unit of analysis. Additional variables provided include whether there was a copy of the study report in the National Criminal Justice Archive, whether the study contained recommendations for policy or practice, and whether the project was completed on time. The data file provides two indices--one that represents quality and one that represents utility. Each measure is generated from a combination of variables in the dataset.

Digital objects are used to link born-digital or digitized content of archival collections. The data is collected from an archival management or archival preservation system. The data can be used to assess the extent of digitized content of archival collections. Digital Collections can also be accessed at https://a860-collectionguides.nyc.gov/.

This data collection contains financial data on state government revenues and expenditures for 16 states during 1933-1937. There are separate files for different levels of aggregation: (1) revenue and expenditure aggregates (1-digit codes), (2) revenues and expenditures classified by major 20th-century categories (2-digit codes), (3) revenues and expenditures classified by minor categories that correspond to special features of 19th- and/or 20th-century governments (3-digit codes), and (4) revenues and expenditures classified by idiosyncratic categories which differ from state to state (4-digit categories). Parts 1 through 4 contain expenditure data. Parts 5 through 8 comprise revenue data. Part 9 contains codes for the categories of expenditures and revenues.

Progress Needed on Identifying Expenditures, Building and Utilizing a Data Infrastructure, and Reducing Duplicative Efforts The federal government collects, maintains, and uses geospatial information—data linked to specific geographic locations—to help support varied missions, including national security and natural resources conservation. To coordinate geospatial activities, in 1994 the President issued an executive order to develop a National Spatial Data Infrastructure—a framework for coordination that includes standards, data themes, and a clearinghouse. GAO was asked to review federal and state coordination of geospatial data. GAO’s objectives were to (1) describe the geospatial data that selected federal agencies and states use and how much is spent on geospatial data; (2) assess progress in establishing the National Spatial Data Infrastructure; and (3) determine whether selected federal agencies and states invest in duplicative geospatial data. To do so, GAO identified federal and state uses of geospatial data; evaluated available cost data from 2013 to 2015; assessed FGDC’s and selected agencies’ efforts to establish the infrastructure; and analyzed federal and state datasets to identify duplication. What GAO Found Federal agencies and state governments use a variety of geospatial datasets to support their missions. For example, after Hurricane Sandy in 2012, the Federal Emergency Management Agency used geospatial data to identify 44,000 households that were damaged and inaccessible and reported that, as a result, it was able to provide expedited assistance to area residents. Federal agencies report spending billions of dollars on geospatial investments; however, the estimates are understated because agencies do not always track geospatial investments. For example, these estimates do not include billions of dollars spent on earth-observing satellites that produce volumes of geospatial data. The Federal Geographic Data Committee (FGDC) and the Office of Management and Budget (OMB) have started an initiative to have agencies identify and report annually on geospatial-related investments as part of the fiscal year 2017 budget process. FGDC and selected federal agencies have made progress in implementing their responsibilities for the National Spatial Data Infrastructure as outlined in OMB guidance; however, critical items remain incomplete. For example, the committee established a clearinghouse for records on geospatial data, but the clearinghouse lacks an effective search capability and performance monitoring. FGDC also initiated plans and activities for coordinating with state governments on the collection of geospatial data; however, state officials GAO contacted are generally not satisfied with the committee’s efforts to coordinate with them. Among other reasons, they feel that the committee is focused on a federal perspective rather than a national one, and that state recommendations are often ignored. In addition, selected agencies have made limited progress in their own strategic planning efforts and in using the clearinghouse to register their data to ensure they do not invest in duplicative data. For example, 8 of the committee’s 32 member agencies have begun to register their data on the clearinghouse, and they have registered 59 percent of the geospatial data they deemed critical. Part of the reason that agencies are not fulfilling their responsibilities is that OMB has not made it a priority to oversee these efforts. Until OMB ensures that FGDC and federal agencies fully implement their responsibilities, the vision of improving the coordination of geospatial information and reducing duplicative investments will not be fully realized. OMB guidance calls for agencies to eliminate duplication, avoid redundant expenditures, and improve the efficiency and effectiveness of the sharing and dissemination of geospatial data. However, some data are collected multiple times by federal, state, and local entities, resulting in duplication in effort and resources. A new initiative to create a national address database could potentially result in significant savings for federal, state, and local governments. However, agencies face challenges in effectively coordinating address data collection efforts, including statutory restrictions on sharing certain federal address data. Until there is effective coordination across the National Spatial Data Infrastructure, there will continue to be duplicative efforts to obtain and maintain these data at every level of government.https://www.gao.gov/assets/d15193.pdfWhat GAO Recommends GAO suggests that Congress consider assessing statutory limitations on address data to foster progress toward a national address database. GAO also recommends that OMB improve its oversight of FGDC and federal agency initiatives, and that FGDC and selected agencies fully implement initiatives. The agencies generally agreed with the recommendations and identified plans to implement them.

Harness AI-Driven Precision for Global Company Insights Leverage cutting-edge AI agents to fetch and validate company registry data in real-time, bypassing obsolete databases. Unlike traditional providers, our service dynamically retrieves data directly from government registries worldwide, ensuring up-to-the-minute accuracy and eliminating outdated records.

Key Features 1. AI-Powered Real-Time Access: Deploy autonomous AI agents to collect and structure data from any national registry, even those with dynamic layouts or authentication barriers.

1. Universal Registry Compatibility: Seamlessly extract data from 250+ countries, including hard-to-access regions, with automatic translation and normalization.

2. Document Processing: Parse financial filings, annual reports, and legal documents (PDF, DOCX) using NLP-driven analysis. Extract key attributes like ownership structures, director details, and compliance status.

3. Format Flexibility: Receive data via API, CSV, JSON, or custom formats (e.g., PostgreSQL DB, Google Sheets) with hourly/daily refresh options.

4. 99% Accuracy Guarantee: Multi-layer validation via AI cross-referencing and human audits ensures error-free datasets.

Data Sourcing & Coverage 1. Sources: Direct integration with 1,800+ government registries of your choice on demand, supplemented by AI-enhanced verification of public filings and regulatory submissions.

1. Attributes: Company name, registration number, directors, shareholders, financials, litigation history, and industry-specific certifications (e.g., ISO, NAICS).

2. Historical Data: 10+ years of archived records, updated in real-time.

Use Cases 1. Due Diligence: Verify company legitimacy for mergers, acquisitions, or partnerships.

1. Compliance: Streamline KYC/AML workflows with automated registry checks.

2. Market Research: Track competitor expansions, ownership changes, or industry trends.

3. Risk Management: Monitor regulatory violations or financial instability signals.

4. Credit Reporting: Automate end-to-end credit report creation process.

Technical Specifications 1. Delivery: API (REST/GraphQL), SFTP, cloud sync (AWS S3, Google Cloud).

1. Integration: Custom connectors for Salesforce, HubSpot, and BI tools (Tableau, Power BI).

2. Latency: Sub-5-second to 60 mins response time for on-demand queries based on the complexity and response time of registry.

Why Choose Us? 1. Pioneers in AI Agent Technology: Outperform static datasets with live registry scraping.

1. GDPR/CCPA Compliance: Data sourced ethically from public registries, with audit trails on output.

2. Free Sample: Test 100 records at zero cost.