• Name: GoiEner smart meters data
• Summary: The dataset contains hourly time series of electricity consumption (kWh) provided by the Spanish electricity retailer GoiEner. The time series are arranged in four compressed files:
  • raw.tzst, contains raw time series of all GoiEner clients (any date, any length, may have missing samples).
  • imp-pre.tzst, contains processed time series (imputation of missing samples), longer than one year, collected before March 1, 2020.
  • imp-in.tzst, contains processed time series (imputation of missing samples), longer than one year, collected between March 1, 2020 and May 30, 2021.
  • imp-post.tzst, contains processed time series (imputation of missing samples), longer than one year, collected after May 30, 2020.
  • metadata.csv, contains relevant information for each time series.
• License: CC-BY-SA
• Acknowledge: These data have been collected in the framework of the WHY project. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 891943.
• Disclaimer: The sole responsibility for the content of this publication lies with the authors. It does not necessarily reflect the opinion of the Executive Agency for Small and Medium-sized Enterprises (EASME) or the European Commission (EC). EASME or the EC are not responsible for any use that may be made of the information contained therein.
• Collection Date: From November 2, 2014 to June 8, 2022.
• Publication Date: December 1, 2022.
• DOI: 10.5281/zenodo.7362094
• Other repositories: None.
• Author: GoiEner, University of Deusto.
• Objective of collection: This dataset was originally used to establish a methodology for clustering households according to their electricity consumption.
• Description: The meaning of each column is described next for each file.
  • raw.tzst: (no column names provided)
    • timestamp;
    • electricity consumption in kWh.
  • imp-pre.tzst, imp-in.tzst, imp-post.tzst:
    • “timestamp”: timestamp;
    • “kWh”: electricity consumption in kWh;
    • “imputed”: binary value indicating whether the row has been obtained by imputation.
  • metadata.csv:
    • “user”: 64-character identifying a user;
    • “start_date”: initial timestamp of the time series;
    • “end_date”: final timestamp of the time series;
    • “length_days”: number of days elapsed between the initial and the final timestamps;
    • “length_years”: number of years elapsed between the initial and the final timestamps;
    • “potential_samples”: number of samples that should be between the initial and the final timestamps of the time series if there were no missing values;
    • “actual_samples”: number of actual samples of the time series;
    • “missing_samples_abs”: number of potential samples minus actual samples;
    • “missing_samples_pct”: potential samples minus actual samples as a percentage;
    • “contract_start_date”: contract start date; “contract_end_date”: contract end date;
    • “contracted_tariff”: type of tariff contracted (2.X: households and SMEs, 3.X: SMEs with high consumption, 6.X: industries, large commercial areas, and farms);
    • “self_consumption_type”: the type of self-consumption to which the users are subscribed;
    • “p1”, “p2”, “p3”, “p4”, “p5”, “p6”: contracted power (in kW) for each of the six time slots;
    • “province”: province where the user is located;
    • “municipality”: municipality where the user is located (municipalities below 50.000 inhabitants have been removed);
    • “zip_code”: post code (post codes of municipalities below 50.000 inhabitants have been removed);
    • “cnae”: CNAE (Clasificación Nacional de Actividades Económicas) code for economic activity classification.
• 5 star: ⭐⭐⭐
• Preprocessing steps: Data cleaning (imputation of missing values using the Last Observation Carried Forward algorithm using weekly seasons); data integration (combination of multiple SIMEL files, i.e. the data sources); data transformation (anonymization, unit conversion, metadata generation).
• Reuse: This dataset is related to datasets:
  • "A database of features extracted from different electricity load profiles datasets" (DOI 10.5281/zenodo.7382818), where time series feature extraction has been performed.
  • "Measuring the flexibility achieved by a change of tariff" (DOI 10.5281/zenodo.7382924), where the metadata has been extended to include the results of a socio-economic characterization and the answers to a survey about barriers to adapt to a change of tariff.
• Update policy: There might be a single update in mid-2023.
• Ethics and legal aspects: The data provided by GoiEner contained values of the CUPS (Meter Point Administration Number), which are personal data. A pre-processing step has been carried out to replace the CUPS by random 64-character hashes.
• Technical aspects:
  • raw.tzst contains a 15.1 GB folder with 25,559 CSV files;
  • imp-pre.tzst contains a 6.28 GB folder with 12,149 CSV files;
  • imp-in.tzst contains a 4.36 GB folder with 15.562 CSV files; and
  • imp-post.tzst contains a 4.01 GB folder with 17.519 CSV files.
• Other: None.

"Smart meter roll-outs provide easy access to granular meter measurements, enabling advanced energy services, ranging from demand response measures, tailored energy feedback and smart home/building automation. To design such services, train and validate models, access to data that resembles what is expected of smart meters, collected in a real-world setting, is necessary. The REFIT electrical load measurements dataset includes whole house aggregate loads and nine individual appliance measurements at 8-second intervals per house, collected continuously over a period of two years from 20 houses. During monitoring, the occupants were conducting their usual routines. At the time of publishing, the dataset has the largest number of houses monitored in the United Kingdom at less than 1-minute intervals over a period greater than one year. The dataset comprises 1,194,958,790 readings, that represent over 250,000 monitored appliance uses. The data is accessible in an easy-to-use comma-separated format, is time-stamped and cleaned to remove invalid measurements, correctly label appliance data and fill in small gaps of missing data."

Data set: https://pureportal.strath.ac.uk/en/publications/an-electrical-load-measurements-dataset-of-united-kingdom-househo

Banner photo by @thejmoore on unsplash.com

Kaggle London Smart Meters dataset contains 5560 half hourly time series that represent the energy consumption readings of London households in kilowatt hour (kWh) from November 2011 to February 2014.

The original dataset contains missing values. They have been replaced by carrying forward the corresponding last observations (LOCF method).

This dataset is maintained by Steven Firth (s.k.firth@lboro.ac.uk), Building Energy Research Group (BERG), School of Civil and Building Engineering, Loughborough University. The REFIT project (www.refitsmarthomes.org) carried out a study from 2013 to 2015 in which 20 UK homes were upgraded to Smart Homes through the installation of devices including Smart Meters, programmable thermostats, programmable radiator valves, motion sensors, door sensors and window sensors.Data was collected using building surveys, sensor placements and household interviews.The REFIT Smart Home dataset is one of the datasets made publically available by the project. This dataset includes: - Building survey data for the 20 homes. - Sensor measurements made before the Smart Home equipment was installed. - Sensor measurements made after the Smart Home equipment was installed. - Climate data recorded at a nearby weather station.--- This work has been carried out as part of the REFIT project (‘Personalised Retrofit Decision Support Tools for UK Homes using Smart Home Technology’, Grant Reference EP/K002457/1). REFIT is a consortium of three universities - Loughborough, Strathclyde and East Anglia - and ten industry stakeholders funded by the Engineering and Physical Sciences Research Council (EPSRC) under the Transforming Energy Demand in Buildings through Digital Innovation (BuildTEDDI) funding programme. For more information see: www.epsrc.ac.uk and www.refitsmarthomes.org---The references below provide links to the REFIT project website, the TEDDINET website, a journal article which uses the dataset, and three additional datasets collected as part of the REFIT project by the University of Strathclyde and the University of East Anglia.

This dataset builds upon the publicly available work by Taranvee, who collected household energy consumption data at one-minute resolution from a smart meter monitoring multiple appliances (e.g., dishwasher, home office, fridge, kitchen). The original release also includes regional weather information (humidity, temperature, atmospheric pressure, etc.), providing a rich contextual layer for understanding how environmental factors influence residential power usage.

In this augmented version, we introduce additional consumption columns representing distinct IoT devices, Car charger,Water heater,Air conditioning,Home Theater,Outdoor lights,microwave,Laundry,Pool Pump

Each of these new columns tracks an appliance's energy usage in kilowatts ([kW]), effectively broadening the dataset’s scope for modeling complex, multi-device scenarios within a single smart home

Original work: https://www.kaggle.com/datasets/taranvee/smart-home-dataset-with-weather-information

All DNOs are required by licence condition to comply with Ofgem’s Data Best Practice (DBP) Guidance. The DBP Guidance was updated by Ofgem on 7th August 2023, and the DBP Guidance now requires DNOs to openly publish Aggregated Smart Meter Consumption Data from 28th February 2024. This data set contains Aggregated Smart Meter Consumption Data at a Low Voltage Feeder level. Presented in gz format due to file size considerations. If you're using Windows, you can extract GZ files using the "tar" command in Command Prompt or by installing the 7-Zip program. On a Mac, just double-click the file to extract it, or use the command gunzip filename. gz in a Terminal window. If you're using Linux, use the gzip -d filename.

The PREMISES Smart Meter Measurement dataset contains electrical consumption data, as well as water and gas consumption when available, for 3 households in Belgium. The measurements were obtained using the local P1-Port interface and the open software tool: https://github.com/ejpalacios/p1-reader-premises. Electricity measurements are sampled at 1-second intervals, while gas and water cumulative values are given every 5 minutes. Metadata on household composition and energy efficiency (EPC and E levels) are provided. Likewise, we include details on whether the house has on-site PV generation, electric vehicle charger, and heat pump. This work was supported by the Research Foundation Flanders (FWO) Marie Skłodowska-Curie Actions - Seal of Excellence Postdoctoral Fellowships, under the project: "PREMISES: PRoviding Energy Metering Infrastructures with Secure Extended Services" Grant number: 12ZZV22N Data donated with the explicit permission of the subjects.

Zip files enertalk-dataset-{house_number} contain a directory for each houses. Each directory holds a set of subdirectories that contain Parquet files for the daily aggregate and appliance-level data. The naming convention for these subdirectories is “” (e.g. “20161124” for November 24, 2016). The Parquet files are named “_.parquet.gzip” (e.g. “01_fridge.parquet.gzip”). In these names, the two-digit integer is uniquely associated with a distinct measuring device in a house. Each Parquet file consists of three columns: “timestamp,” “active_power,” and “reactive_power.” The “timestamp” column contains Unix timestamps in milliseconds, such that 1000 corresponds to one second. The “active_power” column represents active power in watts and the “reactive_power” column represents reactive power in VAR (volt-ampere reactive) units.

This dataset contains half hourly smart meter measurements of 4443 households, obtained during the Low Carbon London project, during 2013.

It is a refactored version of the data released by UK Power Networks under CC-BY license. The following filters have been applied:

• Only readings for households on the 'Std' tariff (i.e., non-time-of-use) have been retained
• 'Null' readings have been dropped
• Only data pertaining to energy consumption in 2013 has been retained. The first measurement has time stamp '2013-01-01 00:30:00.0000000' and the final measurement has time stamp '2014-01-01 00:00:00.0000000'.

Description of the data format:

• Zipped CSV file (utf-8 encoding, comma deliminator)
• First column is DateTime (GMT; no adjustment for British Summer Time)
• First row contains household labels
• Measurements are in kWh (energy consumption) for the preceding half hour. Note that to generate power profiles (with interpretation as average power in kW per half hour) requires multiplication by two.

Note: a cleaner version of the same data set, accompanied by survey data, is available under a more restrictive license at DOI: http://doi.org/10.5255/UKDA-SN-7857-2.

Overview    This dataset offers valuable insights into yearly domestic water consumption across various Lower Super Output Areas (LSOAs) or Data Zones, accompanied by the count of water meters within each area. It is instrumental for analysing residential water use patterns, facilitating water conservation efforts, and guiding infrastructure development and policy making at a localised level. Key Definitions    Aggregation   The process of summarising or grouping data to obtain a single or reduced set of information, often for analysis or reporting purposes.     AMR Meter Automatic meter reading (AMR) is the technology of automatically collecting consumption, diagnostic, and status data from a water meter remotely and periodically. Dataset   Structured and organised collection of related elements, often stored digitally, used for analysis and interpretation in various fields.  Data Zone Data zones are the key geography for the dissemination of small area statistics in Scotland Dumb Meter A dumb meter or analogue meter is read manually. It does not have any external connectivity. Granularity   Data granularity is a measure of the level of detail in a data structure. In time-series data, for example, the granularity of measurement might be based on intervals of years, months, weeks, days, or hours   ID   Abbreviation for Identification that refers to any means of verifying the unique identifier assigned to each asset for the purposes of tracking, management, and maintenance.    LSOA Lower Layer Super Output Areas (LSOA) are a geographic hierarchy designed to improve the reporting of small area statistics in England and Wales. Open Data Triage   The process carried out by a Data Custodian to determine if there is any evidence of sensitivities associated with Data Assets, their associated Metadata and Software Scripts used to process Data Assets if they are used as Open Data.    Schema   Structure for organising and handling data within a dataset, defining the attributes, their data types, and the relationships between different entities. It acts as a framework that ensures data integrity and consistency by specifying permissible data types and constraints for each attribute.    Smart Meter A smart meter is an electronic device that records information and communicates it to the consumer and the supplier. It differs from automatic meter reading (AMR) in that it enables two-way communication between the meter and the supplier. Units   Standard measurements used to quantify and compare different physical quantities.  Water Meter Water metering is the practice of measuring water use. Water meters measure the volume of water used by residential and commercial building units that are supplied with water by a public water supply system. Data History    Data Origin    Domestic consumption data is recorded using water meters. The consumption recorded is then sent back to water companies. This dataset is extracted from the water companies. Data Triage Considerations    This section discusses the careful handling of data to maintain anonymity and addresses the challenges associated with data updates, such as identifying household changes or meter replacements. Identification of Critical Infrastructure  This aspect is not applicable for the dataset, as the focus is on domestic water consumption and does not contain any information that reveals critical infrastructure details. Commercial Risks and Anonymisation Individual Identification Risks There is a potential risk of identifying individuals or households if the consumption data is updated irregularly (e.g., every 6 months) and an out-of-cycle update occurs (e.g., after 2 months), which could signal a change in occupancy or ownership. Such patterns need careful handling to avoid accidental exposure of sensitive information. Meter and Property Association Challenges arise in maintaining historical data integrity when meters are replaced but the property remains the same. Ensuring continuity in the data without revealing personal information is crucial. Interpretation of Null Consumption Instances of null consumption could be misunderstood as a lack of water use, whereas they might simply indicate missing data. Distinguishing between these scenarios is vital to prevent misleading conclusions. Meter Re-reads The dataset must account for instances where meters are read multiple times for accuracy. Joint Supplies & Multiple Meters per Household Special consideration is required for households with multiple meters as well as multiple households that share a meter as this could complicate data aggregation. Schema Consistency with the Energy Industry: In formulating the schema for the domestic water consumption dataset, careful consideration was given to the potential risks to individual privacy. This evaluation included examining the frequency of data updates, the handling of property and meter associations, interpretations of null consumption, meter re-reads, joint suppliers, and the presence of multiple meters within a single household as described above. After a thorough assessment of these factors and their implications for individual privacy, it was decided to align the dataset's schema with the standards established within the energy industry. This decision was influenced by the energy sector's experience and established practices in managing similar risks associated with smart meters. This ensures a high level of data integrity and privacy protection. Schema The dataset schema is aligned with those used in the energy industry, which has encountered similar challenges with smart meters. However, it is important to note that the energy industry has a much higher density of meter distribution, especially smart meters. Aggregation to Mitigate Risks The dataset employs an elevated level of data aggregation to minimise the risk of individual identification. This approach is crucial in maintaining the utility of the dataset while ensuring individual privacy. The aggregation level is carefully chosen to remove identifiable risks without excluding valuable data, thus balancing data utility with privacy concerns. Data Freshness  Users should be aware that this dataset reflects historical consumption patterns and does not represent real-time data. Publish Frequency  Annually Data Triage Review Frequency    An annual review is conducted to ensure the dataset's relevance and accuracy, with adjustments made based on specific requests or evolving data trends. Data Specifications   For the domestic water consumption dataset, the data specifications are designed to ensure comprehensiveness and relevance, while maintaining clarity and focus. The specifications for this dataset include: Each dataset encompasses recordings of domestic water consumption as measured and reported by the data publisher. It excludes commercial consumption. Where it is necessary to estimate consumption, this is calculated based on actual meter readings. Meters of all types (smart, dumb, AMR) are included in this dataset. The dataset is updated and published annually. Historical data may be made available to facilitate trend analysis and comparative studies, although it is not mandatory for each dataset release. Context   Users are cautioned against using the dataset for immediate operational decisions regarding water supply management. The data should be interpreted considering potential seasonal and weather-related influences on water consumption patterns. The geographical data provided does not pinpoint locations of water meters within an LSOA. The dataset aims to cover a broad spectrum of households, from single-meter homes to those with multiple meters, to accurately reflect the diversity of water use within an LSOA.

Data HistoryData OriginDomestic consumption data is recorded using water meters. The consumption recorded is then sent back to water companies. This dataset is extracted from the water companies.Data Triage ConsiderationsThis section discusses the careful handling of data to maintain anonymity and addresses the challenges associated with data updates, such as identifying household changes or meter replacements.Identification of Critical InfrastructureThis aspect is not applicable for the dataset, as the focus is on domestic water consumption and does not contain any information that reveals critical infrastructure details.Individual Identification RisksThere is a potential risk of identifying individuals or households if the consumption data is updated irregularly (e.g., every 6 months) and an out-of-cycle update occurs (e.g., after 2 months), which could signal a change in occupancy or ownership. Such patterns need careful handling to avoid accidental exposure of sensitive information.Meter and Property AssociationChallenges arise in maintaining historical data integrity when meters are replaced but the property remains the same. Ensuring continuity in the data without revealing personal information is crucial.Interpretation of Null ConsumptionInstances of null consumption could be misunderstood as a lack of water use, whereas they might simply indicate missing data. Distinguishing between these scenarios is vital to prevent misleading conclusions.Meter Re-readsThe dataset must account for instances where meters are read multiple times for accuracy.Joint Supplies & Multiple Meters per HouseholdSpecial consideration is required for households with multiple meters as well as multiple households that share a meter as this could complicate data aggregation.Schema Consistency with the Energy Industry:In formulating the schema for the domestic water consumption dataset, careful consideration was given to the potential risks to individual privacy. This evaluation included examining the frequency of data updates, the handling of property and meter associations, interpretations of null consumption, meter re-reads, joint suppliers, and the presence of multiple meters within a single household as described above.After a thorough assessment of these factors and their implications for individual privacy, it was decided to align the dataset"s schema with the standards established within the energy industry. This decision was influenced by the energy sector"s experience and established practices in managing similar risks associated with smart meters. This ensures a high level of data integrity and privacy protection.SchemaThe dataset schema is aligned with those used in the energy industry, which has encountered similar challenges with smart meters. However, it is important to note that the energy industry has a much higher density of meter distribution, especially smart meters.Aggregation to Mitigate RisksThe dataset employs an elevated level of data aggregation to minimise the risk of individual identification. This approach is crucial in maintaining the utility of the dataset while ensuring individual privacy. The aggregation level is carefully chosen to remove identifiable risks without excluding valuable data, thus balancing data utility with privacy concerns.Data FreshnessUsers should be aware that this dataset reflects historical consumption patterns and does not represent real-time data.Publish FrequencyAnnuallyData Triage Review FrequencyAn annual review is conducted to ensure the dataset"s relevance and accuracy, with adjustments made based on specific requests or evolving data trends.Data SpecificationsFor the domestic water consumption dataset, the data specifications are designed to ensure comprehensiveness and relevance, while maintaining clarity and focus. The specifications for this dataset include:Each dataset encompasses recordings of domestic water consumption as measured and reported by the data publisher. It excludes commercial consumption.Consumption for calendar year dates is calculated based on actual meter readings.​Meters of all types (smart, dumb, AMR) are included in this dataset.​The dataset is updated and published annually.​The dataset includes LSOAs with 10 or more meters. Any LSOAs with less than 10 meters have been excluded.​The dataset includes only meters that are currently shown as active.​The dataset excludes any meters where consumption is recorded as null.ContextUsers are cautioned against using the dataset for immediate operational decisions regarding water supply management. The data should be interpreted considering potential seasonal and weather-related influences on water consumption patterns.​The geographical data provided does not pinpoint locations of water meters within an LSOA.​​The dataset aims to cover a broad spectrum of households, from single-meter homes to those with multiple meters, to accurately reflect the diversity of water use within an LSOA.​This dataset has been calculated using actual read data. To align reads to calendar year, our approach uses a combination of previous and next usage readings, along with the number of days between these readings to calculate the total consumption.Supplementary Information​Below is a curated selection of links for additional reading, which provide a deeper understanding of this dataset:Ofwat guidance on water meters Data SchemaDATA_SOURCE: Company that provided the dataYEAR: The calendar year covered by the dataLSOA_CODE: LSOA or Data Zone converted code of the meter locationNUMBER_OF_METERS: Number of meters within an LSOATOTAL_CONSUMPTION: Total consumption within the LSOATOTAL_CONSUMPTION_UNITS: Units for total consumption

Overview

This dataset offers valuable insights into yearly domestic water consumption across various Lower Super Output Areas (LSOAs) or Data Zones, accompanied by the count of water meters within each area. It is instrumental for analysing residential water use patterns, facilitating water conservation efforts, and guiding infrastructure development and policy making at a localised level.

Key Definitions

Aggregation

The process of summarising or grouping data to obtain a single or reduced set of information, often for analysis or reporting purposes.

AMR Meter

Automatic meter reading (AMR) is the technology of automatically collecting consumption, diagnostic, and status data from a water meter remotely and periodically.

Dataset

Structured and organised collection of related elements, often stored digitally, used for analysis and interpretation in various fields.

Data Zone

Data zones are the key geography for the dissemination of small area statistics in Scotland

Dumb Meter

A dumb meter or analogue meter is read manually. It does not have any external connectivity.

Granularity

Data granularity is a measure of the level of detail in a data structure. In time-series data, for example, the granularity of measurement might be based on intervals of years, months, weeks, days, or hours

ID

Abbreviation for Identification that refers to any means of verifying the unique identifier assigned to each asset for the purposes of tracking, management, and maintenance.

LSOA

Lower Layer Super Output Areas (LSOA) are a geographic hierarchy designed to improve the reporting of small area statistics in England and Wales.

Open Data Triage

The process carried out by a Data Custodian to determine if there is any evidence of sensitivities associated with Data Assets, their associated Metadata and Software Scripts used to process Data Assets if they are used as Open Data.

Schema

Structure for organising and handling data within a dataset, defining the attributes, their data types, and the relationships between different entities. It acts as a framework that ensures data integrity and consistency by specifying permissible data types and constraints for each attribute.

Smart Meter

A smart meter is an electronic device that records information and communicates it to the consumer and the supplier. It differs from automatic meter reading (AMR) in that it enables two-way communication between the meter and the supplier.

Units

Standard measurements used to quantify and compare different physical quantities.

Water Meter

Water metering is the practice of measuring water use. Water meters measure the volume of water used by residential and commercial building units that are supplied with water by a public water supply system.

Data History

Data Origin

Domestic consumption data is recorded using water meters. The consumption recorded is then sent back to water companies. This dataset is extracted from the water companies.

Data Triage Considerations

This section discusses the careful handling of data to maintain anonymity and addresses the challenges associated with data updates, such as identifying household changes or meter replacements.

Identification of Critical Infrastructure

This aspect is not applicable for the dataset, as the focus is on domestic water consumption and does not contain any information that reveals critical infrastructure details.

Commercial Risks and Anonymisation

Individual Identification Risks

There is a potential risk of identifying individuals or households if the consumption data is updated irregularly (e.g., every 6 months) and an out-of-cycle update occurs (e.g., after 2 months), which could signal a change in occupancy or ownership. Such patterns need careful handling to avoid accidental exposure of sensitive information.

Meter and Property Association

Challenges arise in maintaining historical data integrity when meters are replaced but the property remains the same. Ensuring continuity in the data without revealing personal information is crucial.

Interpretation of Null Consumption

Instances of null consumption could be misunderstood as a lack of water use, whereas they might simply indicate missing data. Distinguishing between these scenarios is vital to prevent misleading conclusions.

Meter Re-reads

The dataset must account for instances where meters are read multiple times for accuracy.

Joint Supplies & Multiple Meters per Household

Special consideration is required for households with multiple meters as well as multiple households that share a meter as this could complicate data aggregation.

Schema Consistency with the Energy Industry:

In formulating the schema for the domestic water consumption dataset, careful consideration was given to the potential risks to individual privacy. This evaluation included examining the frequency of data updates, the handling of property and meter associations, interpretations of null consumption, meter re-reads, joint suppliers, and the presence of multiple meters within a single household as described above.

After a thorough assessment of these factors and their implications for individual privacy, it was decided to align the dataset's schema with the standards established within the energy industry. This decision was influenced by the energy sector's experience and established practices in managing similar risks associated with smart meters. This ensures a high level of data integrity and privacy protection.

Schema

The dataset schema is aligned with those used in the energy industry, which has encountered similar challenges with smart meters. However, it is important to note that the energy industry has a much higher density of meter distribution, especially smart meters.

Aggregation to Mitigate Risks

The dataset employs an elevated level of data aggregation to minimise the risk of individual identification. This approach is crucial in maintaining the utility of the dataset while ensuring individual privacy. The aggregation level is carefully chosen to remove identifiable risks without excluding valuable data, thus balancing data utility with privacy concerns.

Data Freshness

Users should be aware that this dataset reflects historical consumption patterns and does not represent real-time data.

Publish Frequency

Annually

Data Triage Review Frequency

An annual review is conducted to ensure the dataset's relevance and accuracy, with adjustments made based on specific requests or evolving data trends.

Data Specifications

For the domestic water consumption dataset, the data specifications are designed to ensure comprehensiveness and relevance, while maintaining clarity and focus. The specifications for this dataset include:

·
Each dataset encompasses recordings of domestic water consumption as measured and reported by the data publisher. It excludes commercial consumption.

· Where it is necessary to estimate consumption, this is calculated based on actual meter readings.

· Meters of all types (smart, dumb, AMR) are included in this dataset.

·
The dataset is updated and published annually.

·
Historical data may be made available to facilitate trend analysis and comparative studies, although it is not mandatory for each dataset release.

Context

Users are cautioned against using the dataset for immediate operational decisions regarding water supply management. The data should be interpreted considering potential seasonal and weather-related influences on water consumption patterns.

The geographical data provided does not pinpoint locations of water meters within an LSOA.

The dataset aims to cover a broad spectrum of households, from single-meter homes to those with multiple meters, to accurately reflect the diversity of water use within an LSOA.

Supplementary Information

1. Below is a curated selection of links for additional reading, which provide a deeper understanding of this dataset.

2. Ofwat guidance on water meters

3. https://www.ofwat.gov.uk/wp-content/uploads/2015/11/prs_lft_101117meters.pdf

Here are a few use cases for this project:

1. Utility Management: This model could be used to automate the process of reading utility meters such as gas, water, or electricity in homes and businesses. By identifying the numbers and position of the pointer tip on the meter, the model could provide real-time, accurate data to utility providers for billing purpose.

2. Industrial Machinery Monitoring: The model would be used in factories or industrial settings to monitor machinery gauges and meters. Automated regular readings would allow for proactive maintenance and immediate response to irregularities or failures, preventing costly downtime.

3. Vehicular Data Collection: An application could be development of a system for fleet vehicles, enabling automatic collection and reporting of data such as fuel consumption, speed, and distance traveled. This would enhance operational efficiency and maintenance planning.

4. Health Monitoring Systems: The model could be integrated into medical devices or instruments like blood pressure or glucose meters, to streamline the process of reading and recording health data, making it easier both for health professionals and patients managing their conditions at home.

5. Home Energy Management: The model could be used in smart home systems to read home energy usage from various appliances. The data could then be analyzed to provide tips on energy conservation, potentially helping homeowners to reduce their energy bills.

The dataset refers to measurements and data collected from and for the CERTH Smart House nanogrid infrastructure in Thessaloniki, Greece. This infrastructure is a living lab that belongs to the Centre for Research and Technology-Hellas (www.certh.gr) and has been designed, deployed and operated by the Information Technology Institute (www.iti.gr). Mainly energy-related aspects are included in the datasets uploaded which are in the form of .csv files, covering Electricity Energy Consumption, Generation, and Storage, as well as Weather and Electricity Price information from external APIs (a local weather station has just been installed and will be included in future versions).

Energy consumption readings for a sample of 5,567 London Households that took part in the UK Power Networks led Low Carbon London project between November 2011 and February 2014.

Readings were taken at half hourly intervals. The customers in the trial were recruited as a balanced sample representative of the Greater London population.

The dataset contains energy consumption, in kWh (per half hour), unique household identifier, date and time. The CSV file is around 10GB when unzipped and contains around 167million rows.

Within the data set are two groups of customers. The first is a sub-group, of approximately 1100 customers, who were subjected to Dynamic Time of Use (dToU) energy prices throughout the 2013 calendar year period. The tariff prices were given a day ahead via the Smart Meter IHD (In Home Display) or text message to mobile phone. Customers were issued High (67.20p/kWh), Low (3.99p/kWh) or normal (11.76p/kWh) price signals and the times of day these applied. The dates/times and the price signal schedule is availaible as part of this dataset. All non-Time of Use customers were on a flat rate tariff of 14.228pence/kWh.

The signals given were designed to be representative of the types of signal that may be used in the future to manage both high renewable generation (supply following) operation and also test the potential to use high price signals to reduce stress on local distribution grids during periods of stress.

The remaining sample of approximately 4500 customers energy consumption readings were not subject to the dToU tariff.

More information can be found on the Low Carbon London webpage

Some analysis of this data can be seen here.

This is grouped and aggregated electricity consumption data from the EtudELEC study conducted by the Observatoire du Transition Energétique Grenoble (OTE-UGA). If you find this dataset useful and like different groupings to be published or have any questions, please feel free to comment on the discussion dedicated to this dataset on the OTE forum . The EtudELEC study only studies electricity consumption from residential dwellings around France. The study involved over 400 homes (individual houses and apartments) and spanned ~11 months from 25th October 2022 to 1st October 2023. The data is collected from the smart meter data (Linky data) which is only available with a time-step of 30 minutes. The data is in average watts consumed in a half-hour period. For reasons for privacy in line with GDPR laws, personal data such as individual home consumption will be shared as aggregated datasets as opposed to individual data points. The data from the participants were aggregated based on the following groupings: - Type of heating used and type of residence (stand-alone house vs apartment) This dataset is best viewed in the "Tree" view below. A folder is created for each of the groupings and sub-folders exist for all the subsequent groups. Each group folder contains: - a table of the minimum, mean, and maximum of the average power consumed for each 30-minute period (W), and - a JSON file with aggregated demographics information (number of inhabitants in different age backets, socio-professional category, year of construction etc.) of the group The datasets will be updated on a yearly basis following the renewal of consent of the panel members. Il s'agit de données de consommation électriques groupées et agrégées issues de l'étude EtudELEC menée par l'Observatoire de la Transition Energétique (OTE-UGA). Si vous trouvez ce jeu de données utile et souhaitez que différents regroupements soient publiés, n'hésitez pas à écrire dans le topic sur le forum OTE. L'étude EtudELEC est une étude sur la consommation d'électricité des logements résidentiels en France. L'étude porte sur plus de 400 logements (maisons individuelles et appartements) et s'étend sur 11 mois du 25 octobre 2022 au 1 octobre 2023. Les données sont collectées à partir des données des compteurs intelligents (données Linky) qui ne sont disponibles qu'avec un pas de temps de 30 minutes. Les données sont exprimées en watts consommés en moyenne sur une période d'une demi-heure. Pour des raisons de confidentialité conformes aux lois RGPD, les données personnelles telles que la consommation individuelle des maisons seront partagées sous forme d'ensembles de données agrégées plutôt que de points de données individuels. Les données des participants ont été agrégées sur la base des regroupements suivants : - Type de chauffage utilisé et type de résidence (maison individuelle ou appartement). Cet ensemble de données est mieux visualisé dans l'arborescence ci-dessous. Un dossier est créé pour chaque groupe et des sous-dossiers existent pour tous les groupes suivants. Chaque dossier de groupe contient : - un tableau du minimum, de la moyenne et du maximum de la puissance moyenne consommée pour chaque période de 30 minutes (W), et - un fichier JSON avec des informations démographiques agrégées (nombre d'habitants dans différentes tranches d'âge, catégorie socioprofessionnelle, année de construction, etc. Les jeux de données seront mis à jour chaque année après le renouvellement du consentement des membres du panel.

For processed data files more suitable for use, please refer to the GoiEner Smart Meters Data dataset: DOI 10.5281/zenodo.7362094

The GoiEner Raw Files dataset contains a complete set of raw data from the customer database of the Spanish renewable energy cooperative GoiEner, obtained from smart meters. Founded in the Basque Country in 2012, GoiEner has made available this extensive dataset, which includes the raw electricity consumption data (and self-generation) for all its customers. The supply points provided comprise a diverse range of customers, such as households, offices, small and medium-sized enterprises (SMEs), industrial buildings, and public facilities.

This dataset spans the entire period from the first records using smart meters in late 2014 until June 2022. It consists of 71,048 files containing information on consumption, generation, contracted power, pricing, and other related data for each supply point in the grid. These files serve as the primary source for managing electricity supply contracts between distributors and retailers, as well as for user billing. The formats and specifications of the files adhere to the guidelines set by the Spanish electricity market regulator, the National Commission of Markets and Competition (CNMC), which establishes the Electricity Metering Information System (SIMEL).

Data Between September 2021 and September 2022, we collected the cumulative water consumption data of 17 households in Germany using the commercially available smart meters Hydrus 1.3, DN 20,00 from Diehl Metering. Further information on data collection and description of the dataset can be found in section 3 of the original article, which is available at the following DOI: https://doi.org/10.3390/jsan12030046

Structure The root folder of the Zenodo archive contains the documents readme.md, info.txt, and info.json, as well as the 17 folders for the individual households. The readme.md gives some basic information on the data set and its use. The documents main.txt and main.json contains the metadata for the measurements of all households in both human and machine-readable format. For each test household, there is a separate folder with the documents info.txt and info.json. These contain household-specific metadata. The total water consumption measurements are stored in smartmeter.csv.

The Smart Energy Research Lab (SERL) Observatory facilitates a broad range of energy demand research and is a unique data resource for research where access to high resolution, large scale energy data linked to relevant contextual data is required. Further information about SERL can be found on the Smart Energy Research Lab website.

This dataset of aggregated statistics is available under standard Safeguarded (End User Licence) access conditions. It contains over 2.5 million rows of data and describes domestic gas and electricity energy use in Great Britain 2020-2023 based on data from the Smart Energy Research Lab (SERL) Observatory, which consists of smart meter and contextual data from approximately 13,000 homes that are broadly representative of the GB population in terms of region and Index of Multiple Deprivation (IMD) quintile. This aggregated dataset can be used, for example, to show how residential energy use in GB varies over time (monthly over the year and half-hourly over the course of the day); and can be broken down by occupant characteristics (number of occupants, tenure), property characteristics (age, size, form, and Energy Performance Certificate (EPC)), by type of heating system, presence of solar panels and of electric vehicles, and by weather, region and IMD quintile.

Secure Access data
A more detailed set of SERL data, including smart meter data and additional contextual data, is available under restricted Secure access conditions under SN 8666: Smart Energy Research Lab Observatory Data: Secure Access. It is a longitudinal dataset containing records from August 2019, with updates provided to researchers on a (roughly) quarterly basis. Users should download this safeguarded access statistical study first to see whether it is suitable for their needs before considering an application for the Secure dataset.

The second edition (May 2024) includes summaries of daily average energy use in a data file for 2020-2023, and summaries of half-hourly average energy use in four data files for 2020-2023, as well as an accompanying technical document.

Here are a few use cases for this project:

1. Utility Management: This tool would be particularly beneficial for utility companies managing large scale operations that include reading countless meters for gas, electricity, or water. It would reduce the need for human labor and minimize errors.

2. Sustainability Audits: Organizations or individuals conducting energy audits could use this model to efficiently analyze utility consumption over specified periods without manual inputs, aiding in creating detailed sustainability reports.

3. Smart Homes: It could be integrated within the tools used to manage smart homes or buildings. By automatically reading and recording utility readings, the model would allow for real-time monitoring of energy consumption, contributing to energy-efficient management.

4. In IoT Devices: This model could be implemented into IoT devices designed for energy monitoring and management, providing real-time updates on utility usage, and contributing to conservation efforts.

5. Research and Survey: For researchers and surveyors who are gathering data on energy consumption, this automated tool would significantly streamline the process, making large-scale data collection faster and more accurate.