This report is written evidence compiled by the National Data Guardian (NDG) and submitted to the UK Parliament’s Science and Technology Committee in February 2022 for consideration as part of its inquiry: The right to privacy: digital data. This evidence was also published by the committee on its website.

This response does not address all areas set out for exploration in the inquiry, only those that fall under the NDG’s remit. This inquiry asked about sharing data across a wide range of different organisations such as ‘government departments, other public bodies, research institutions and commercial organisations’. Other questions asked about sharing within discrete contexts such as ‘health and care contexts’. Given the NDG’s remit, this response only addresses the sharing of health and adult social care information.

This report gives an overview of the digital age across society and economy in the UK. It shows the extent to which people, education, business and government have taken up information and communication technology (ICT), and how it is changing leisure, working and business practices. Source agency: Office for National Statistics Designation: National Statistics Language: English Alternative title: Focus On The Digital Age

Measures, analysis, and research into the digital economy key.

UK businesses' engagement in the digital economy including e-commerce sales and purchases, and use of other information and communication technology (ICT).

The UK Government manages the .gov.uk domain name registry in order to signify digital services that are part of the administration of the state, so that they can be identified as authoritative and trustworthy.

The rules governing which organisations are eligible to register .gov.uk domain names and those names that may be used are set out in Naming and registering websites and social media channels.

Public sector bodies may register .gov.uk domain names for a variety of purposes:

Email only purposes Websites, including those for campaigns, education, information and transactions Page redirection (e.g. for alternative spellings of domain names) To maintain access to information on the UK Government Website Archive Domain registration is requested via an Internet Service Provider (ISP) to the technical administrators of the .gov.uk second level domain, JANET(UK). JANET(UK) acts as the domain name registry, holding information on which domain names are registered and who owns them on behalf of Cabinet Office.

Cabinet Office is responsible for approving requests for new domain names and any appeals. The .gov.uk domain name approvals and appeals process is described online.

The database contains satellite images of the UK purchased by the BGS or on its behalf by NERC. It includes data from the Landsat, Spot and Radarsat satellites. The images are stored in proprietary formats on various types of magnetic media. Estimate of extent of coverage. Entire country is covered by the dataset, however, there are gaps in coverage from individual sensors.

A dataset of all the meta-data for all of the datasets available through the data.gov.uk service. This is provided as a zipped CSV or JSON file. It is published nightly.

Updates: 27 Sep 2017: we've moved all the previous dumps to an S3 bucket at https://dgu-ckan-metadata-dumps.s3-eu-west-1.amazonaws.com/ - This link is now listed here as a data file.

From 13/10/16 we added .v2.jsonl dump, which is set to replace the .json dump (which will be discontinued after a 3 month transition). This is produced using 'ckanapi dump'. It provides an enhanced version of each dataset ('validated', or what you get from package_show in CKAN API v3 - the old json was the unvalidated version). This now includes full details of the organization the dataset is in, rather than just the owner_id. Plus it includes the results of the archival & qa for each dataset and resource, showing whether the link is broken, detected format and stars of openness. It also benefits from being json lines http://jsonlines.org/ format, so you don't need to load the whole thing into memory to parse the json - just a line at a time.

On 12/1/2015 the organizations of the CSV was changed:

• Before this date, each dataset was one line, and resources added as numbered columns. Since a dataset may have up to 300 resources, it ends up with 1025 columns, which is wider than many versions of Excel and Libreoffice will open. And the uncompressed size of 170Mb is more than most will deal with too. It is suggested you load it into a database, ahandle it with a python or ruby script, or use tools such as Refine or Google Fusion Tables.

• After this date, the datasets are provided in one CSV and resources in another. On occasions that you want to join them, you can join them using the (dataset) "Name" column. These are now manageable in spreadsheet software.

You can also use the standard CKAN API if you want to search or get a small section of the data. Please respect the traffic limits in the API: http://data.gov.uk/terms-and-conditions

Annual estimates of aspects of UK digital trade (Information Communication and Technology (ICT) goods and potentially digitally delivered services) including trade inside and outside the EU.

This dataset has been withdrawn and is no longer avaible from BGS The database contains satellite images of the UK purchased by the BGS or on its behalf by NERC. It includes data from the Lansdat, SPOT, Radarsat and ERS satellites. The images are stored in proprietary format on various types of magnetic media. The data are currently stored by path-row scene numbers and as mosaics on tapes, CDs and drives. Entire UK is covered by the dataset, however, there are gaps in coverage from individual sensors. Coverage exists for countries (or parts of countries) where work has been carried out.

Data on signature growth and trending information for the UK Cabinet Office petition platform (2011-2015). This dataset consists of the number of signatures to petitions over time (at an hourly resolution) along with petition metadata (title, category, time of creation) and trending information, directly crawled from the website. Further details are available at the links supplied in the references.

A Digital Terrain Model (DTM) is a digital file consisting of a grid of regularly spaced points of known height which, when used with other digital data such as maps or orthophotographs, can provide a 3D image of the land surface. 10m and 50m DTM’s are available. This is a large dataset and will take sometime to download. Please be patient. By download or use of this dataset you agree to abide by the LPS Open Government Data Licence.

A range of reports that helped shape the development and direction of the Mayor's Digital Talent programme .

GOV.UK Registers was a service provided by the Government Digital Service which retired on March 15 2021. The service provided a central point for canonical data to support the development of Government services. The historic data is provided by the National Archives Government web archive.

The list of releases of organograms via data.gov.uk's organogram system.

These are also listed in datasets like https://data.gov.uk/dataset/organogram-cabinet-office but it's helpful to have this aggregated list of organogram releases here in one CSV file.

NB a few organograms have been released in other ways, such as PDFs from 2010/11, some lingering RDFs 2010-2016 and the occasional CSV 2010-2016 that didn't go through the data.gov.uk system. Many of those are listed in data.gov.uk datasets alongside the ones that have gone through the system (and a few are just on https://www.gov.uk/government/publications ). So this is not complete, but not far off.

NB this dataset doesn't contain the actual organogram data - it is just a list of the releases and the URLs to get the data. This is helpful to understand how well government bodies are meeting their commitments. listed at https://data.gov.uk/apps

This is exported from data.gov.uk's database. It is refreshed nightly.

The fields should be relatively self-explanatory when comparing the data with data.gov.uk organogram datasets: https://data.gov.uk/data/search?collection=Organogram

A digital strategy is a plan for focusing and scaling up the benefits to the council, the place, and/or its customers (residents and businesses) of data assets and technology-focused projects and programmes. Digital strategies show how to break down silos between council CIOs and directors of delivery-focused service units to deliver a better digital customer experience and evidence-based policy making.

The NSTA has recently purchased digital well data from CGG for an additional 2235 E&A wells. These have been selected from across the UKCS to complement the existing joined digital well logs that the NSTA has previously released either in support of licence rounds (e.g. 30th Licensing Round, Greater Buchan Area Supplementary Round) or as part of the Government Seismic Data initiatives. Where available, the NSTA has purchased joined digital well logs, deviation data and checkshot data for these additional wells. These data have been loaded to the National Data Repository (NDR). The NSTA’s Well Data Availability layer has also been updated to reflect which well data has been purchased. These data are being released under the OGA Licence (OGAL), the terms of which are available on download from the NDR.