Last updated on 22 Feb 2025

Introduction

This dataset provides comprehensive information on property sales in England and Wales, sourced from the UK government's HM Land Registry. Although the government site claims to update on the same day each month, actual updates can vary. To bridge this update variation gap, our fully automated ETL pipeline retrieves the official government data on a daily basis. This ensures that the dataset always reflects the most current transaction data available.

ETL Process

Our ETL (Extract, Transform, Load) process is designed to automate the data update and publishing workflow: 1. Extract:
The pipeline uses web scraping to retrieve the latest data from the official government website. This step is necessary as the site does not offer an API. 2. Transform:
Before loading the data, the ETL pipeline processes the dataset to ensure consistency and usability. As part of the transformation stage, the first column (Transaction_unique_identifier) is removed. This column is dropped during staging to focus on the most relevant transactional information. The column removal successfully reduces the data file size from almost 6GB to 3.1GB, and therefore will greatly increase the data analysis efficiency, and reduces the chance of kernal error/restart. 3. Load:
Finally, the transformed data is loaded into the dataset.

The transformed data is loaded into the dataset in two parts: - Complete Data (pp-complete.csv): This file encompasses all records from January 1995 to the present. The complete data file is replaced during each update to reflect any corrections or additional historical data. The first column is price. - Monthly Data: A separate monthly file is amended each month. This monthly archive ensures a complete record of updates over time, allowing users to track changes and trends more granularly.

Summary of Results

The dataset (pp-complete.csv) contains records of property sales dating back to January 1995, up to the most recent monthly data. It covers various types of transactions—from residential to commercial properties—providing a holistic view of the real estate market in England and Wales.

Column Descriptions

The original data includes the following columns: - Transaction_unique_identifier
- price
- Date_of_Transfer
- postcode
- Property_Type
- Old/New
- Duration
- PAON
- SAON
- Street
- Locality
- Town/City
- District
- County
- PPDCategory_Type
- Record_Status - monthly_file_only

Note: As part of the transformation process, the Transaction_unique_identifier column is removed from the final published pp-complete.csv data file. Therefore the first column of the pp-complete.csv file is price.

Address data Explanation - Postcode: The postal code where the property is located. - PAON (Primary Addressable Object Name): Typically the house number or name. - SAON (Secondary Addressable Object Name): Additional information if the building is divided into flats or sub-buildings. - Street: The street name where the property is located. - Locality: Additional locality information. - Town/City: The town or city where the property is located. - District: The district in which the property resides. - County: The county where the property is located. - Price Paid: The price for which the property was sold.

Legal and Ethical Considerations

Ownership and Attribution This dataset is the property of HM Land Registry and is released under the Open Government Licence (OGL). If you use or publish this dataset, you are required to include the following attribution statement:

>"Contains HM Land Registry data © Crown copyright and database right 2021. This data is licensed under the Open Government Licence v3.0."

Usage Guidelines

The data can be used for both commercial and non-commercial purposes.

The OGL does not cover third-party rights, which HM Land Registry is not authorized to license. For any other use of the Address Data, you must contact Royal Mail.

Suggested Usages

Market Trend Analysis: Understand the ups and downs of the property market over time. Investment Research: Identify potential areas for property investment. Academic Studies: Use the data for economic research and studies related to the housing market. Policy Making: Assist government agencies in making informed decisions regarding housing policies. Real Estate Apps: Integrate the data into apps that provide property price information services.

By using this dataset, you agree to abide by the terms and conditions as specified by HM Land Registry. Failure to do so may result in legal consequences.

This is a subset of UK House Price Index data published by Land Registry and ONS.

Enhance Property also includes home mover data, providing information around its current listing status, property purchase dates as well as a predicted move date.

What is it? Enhance Property is an address level product built using our trusted, comprehensive data asset repository enabling you to better understand characteristics at a local area level. It provides everything you would want to know about a property.

Sourcing data from Royal Mail’s PAF (Postcode Address) file, Land Registry feeds, Property Listing feeds, and household &address-level segmentations.

Use cases - Understand your customers further through their property characteristics, allowing you to review your customer base at household levels - Identify which households within your customer base are the best kind of customers, using that to then identify new, promising prospects - Trigger communications based on home-moves - Property characteristics can drive product relevance, i.e. a business trying to sell Lawnmowers would only want to focus on households with gardens

Additional Insights Enhance Core, Property & Geo (Individual, Property & Postcode level data) can all be used modularly, allowing you to understand the full picture of your customer base, considering not only their individual variance but also where they live & those around them.

This is a spatial dataset containing polygons representing areas of vegetation mapped within the Moor House National Nature Reserve in the northern Pennines, England. The map was created by staff of The Nature Conservancy in the 1960s. Full details about this dataset can be found at https://doi.org/10.5285/bb703113-3428-483c-858a-2a85cafd9821

Data comprise a set of broadleaf afforestation scenarios (provided as netCDF files) that may be run with the Joint UK Land Environment Simulator (JULES), a community land surface model. The scenarios are based on the CEH Land Cover 2000 classification. Afforestation takes place according to catchment structure and existing land cover. Scenarios cover twelve river catchments in Great Britain: Dee, Tay, Ouse, Ure, Derwent, Thames, Avon, Tamar, Severn at Bewdley , Severn at Haw Bridge, Ribble and Clyde. Afforestation scenarios relate to two catchment properties: - (1) River network structure and (2) Land use. By using these two catchment properties, in conjunction with different extents of afforestation, up to 288 afforestation scenarios per catchment are generated. This dataset was created as part of the NERC doctoral training partnerships (grant number NE/L002612/1).

This is a 25m pixel data set representing the land surface of Northern Ireland, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2021 products are available from the LCM2021 product documentation accompanying this dataset.

This is a 25m pixel data set representing the land surface of Great Britain, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2022 products are available from the LCM2022 product documentation.

This dataset contains information about soil near-surface physical and hydrological properties, vegetation observations and land use & management information across the Thames catchment (UK). It was collected during the ‘Landwise' project's ‘Broad-scale field survey' which sampled 1836 location points across a total of 164 fields/land parcels. The aim of the survey was to quantify the impact of innovative land use and management on soil properties, with implications for natural flood management. The surveyed fields were selected to represent four broad land use and management classes (arable with and without grass in rotation, permanent grassland and broadleaf woodland) and five generalised soil/geology classes. Approximately eight fields were sampled for each of the twenty combinations of land use and soil/geology class. The sampled fields cover a range of traditional and innovative agricultural practices. Within each field/parcel, representative sampling locations were selected to cover the anticipated range of soil variability, including typical infield, untrafficked margins and trafficked headlands/tramlines etc. Sampling was undertaken once during the period 2018-2021. Samples were measured and analysed using a range of field and laboratory techniques (see Data Lineage). Point data include: 1. Survey point location (British National Grid coordinates) 2. Soil quantitative measurements (near-surface: 0 – 50 mm below ground level): dry bulk density, volumetric water content, organic matter, derived porosity, derived porosity accounting for variable organic matter, particle size distribution and texture classification 3. Vegetation quantitative measurements: maximum and minimum height 4. Soil qualitative measurements: hand texture classification, aggregate stability test slaking and dispersion results, hydrochloric acid test for calcareous soil, and for a subset of locations Visual Evaluation of Soil Structure (VESS) score 5. Observations (also classified into groups): soil surface condition (e.g. slaked/unslaked/capped/poached etc.), vegetation type Field contextual data include: 1. Land owner/manager responses to a land use and management questionnaire (primary data) including information on: crop types/rotation, cover crops, herbal leys, organic or conventional, organic amendments, lime additions, tillage, last ploughed, tramlines, buffer strips, field drainage, grass species, livestock, last grazed, stocking density, grazing weeks per year, stock out-wintering, mob or paddock grazing, woodland management, tree species, woodland age, path management, land use history, flooding history, waterlogging, water or sediment runoff 2. Classification of selected questionnaire free text responses into categories (derived secondary data) 3. General field observations (primary data) including: slope gradient and shape, surface form, surface water, surface condition (slaking, capped, ruts, wheelings, poaching etc.), soil erosion or deposition features As agreed with the survey participants, this dataset has been anonymised by removing location specific information, such as farm and field names, along with any other personally identifiable information. As also agreed, point data location coordinates have been degraded to the nearest 1 km grid point. The dataset was co-produced by the UK Centre for Ecology and Hydrology and Landwise Partners as part of the Landwise Natural Flood Management project, supported by the Natural Environment Research Council (Grant NE/R004668/1). The participation and assistance of the land owners and managers is gratefully acknowledged. Full details about this dataset can be found at https://doi.org/10.5285/9ab5285f-e9c4-4588-ba21-476e79e87668

This is a 25m pixel data set representing the land surface of Northern Ireland, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2020 products are available from the LCM2020 product documentation.

This is a 25m pixel data set representing the land surface of Great Britain, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2023 products are available from the LCM2023 product documentation.

This is a 25m pixel data set representing the land surface of Northern Ireland, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2022 products are available from the LCM2022 product documentation.

This is a 25m pixel data set representing the land surface of Northern Ireland, classified into 21 UKCEH land cover classes, based upon Biodiversity Action Plan broad habitats. It is a three-band raster in GeoTiff format, produced by rasterising three properties of the classified land parcels dataset. The first band gives the most likely land cover type; the second band gives the per-parcel probability of the land cover, the third band is a measure of parcel purity. The probability and purity bands (scaled 0 to 100) combine to give an indication of uncertainty. A full description of this and all UKCEH LCM2023 products are available from the LCM2023 product documentation.