Raw US Energy Information Administration (EIA) Form 861 Data, archived from https://www.eia.gov/electricity/data/eia861/

This archive contains raw input data for the Public Utility Data Liberation (PUDL) software developed by Catalyst Cooperative. It is organized into Frictionless Data Packages. For additional information about this data and PUDL, see the following resources:

• The PUDL Repository on GitHub
• PUDL Documentation
• Other Catalyst Cooperative data archives

Annual data on electricity generating capacity, electricity generation and useful thermal output, fuel receipts, fuel stocks, sales, consumption, and emissions in the United States. Based on Form EIA-861 and Form EIA-860 data. Annual time series extend back to 1994.

Hourly Electricity Demand by State

This archive contains the output of the Public Utility Data Liberation (PUDL) Project state electricity demand allocation analysis, as of the v0.4.0 release of the PUDL Python package. Here is the script that produced this output. It was run using the Docker container and processed data that are included in PUDL Data Release v2.0.0.

The analysis uses hourly electricity demand reported at the balancing authority and utility level in the FERC 714 (data archive), and service territories for utilities and balancing authorities inferred from the counties served by each utility, and the utilities that make up each balancing authority in the EIA 861 (data archive), to estimate the total hourly electricity demand for each US state.

We used the total electricity sales by state reported in the EIA 861 as a scaling factor to ensure that the magnitude of electricity sales is roughly correct, and obtains the shape of the demand curve from the hourly planning area demand reported in the FERC 714. The scaling is necessary partly due to imperfections in the historical utility and balancing authority service territory maps which we have been able to reconstruct from the data reported in the EIA 861 Service Territories and Balancing Authority tables.

The compilation of historical service territories based on the EIA 861 data is somewhat manual and could be improved, but overall the results seem reasonable. Additional predictive spatial variables will be required to obtain more granular electricity demand estimates (e.g. at the county level).

FERC 714 Respondents

The file ferc714_respondents.csv links FERC Form 714 respondents to what we believe to be their corresponding EIA utilities or balancing authorities.

• eia_code: An integer ID reported in the FERC Form 714 corresponding to the respondent's EIA ID. In some cases this is a Utility ID, and in others it is a Balancing Authority ID, but which is not specified and so we have had to infer the type of entity which is responding. Note that in many cases the same company acts as both a utility and a balancing authority, and the integer ID associated with the company is often the same in both roles, but it does not need to be.
• respondent_type: Either balancing_authority or utility depending on which type of entity we believe was responding to the FERC 714.
• respondent_id_ferc714: The integer ID of the responding entity within the FERC 714.
• respondent_name_ferc714: The name provided by the respondent in the FERC 714.
• balancing_authority_id_eia: If the respondent was identified as a balancing authority, the EIA ID for that balancing authority, taken from the EIA Form 861.
• balancing_authority_code_eia: If the respondent was identified as a balancing authority, the EIA short code used to identify the balancing authority, taken from the EIA Form 861.
• balancing_authority_name_eia: If the respondent was identified as a balancing authority, the name of the balancing authority, taken from the EIA Form 861.
• utility_id_eia: If the respondent was identified as a utility, the EIA utility ID, taken from the EIA Form 861.
• utility_name_eia: If the respondent was identified as a utility, the name of the utility, taken from the EIA 861.

FERC 714 Respondent Service Territories

The file ferc714_service_territories.csv describes the historical service territories for FERC 714 respondents for the years 2006-2019. For each respondent and year, their service territory is composed of a collection of counties, identified by their 5-digit FIPS codes. The file contains the following columns, with each row associating a single county with a FERC 714 respondent in a particular year:

• respondent_id_ferc714: The FERC Form 714 respondent ID, which is also found in ferc714_respondents.csv
• report_date: The first day of the year for which the service territory is being described.
• state: Two letter abbreviation for the state containing the county, for human readability.
• county: The name of the county, for human readability.
• state_id_fips: The 2-digit FIPS state code.
• county_id_fips: The 5-digit FIPS county code for use with other geospatial data resources, like the US Census DP1 geodatabase.

State Hourly Electricity Demand Estimates

The file demand.csv contains hourly electricity demand estimates for each US state from 2006-2019. It contains the following columns:

• state_id_fips: The 2-digit FIPS state code.
• utc_datetime: UTC time at hourly resolution.
• demand_mwh: Electricity demand for that state and hour in MWh. This is an allocation of the electricity demand reported directly in the FERC Form 714.
• scaled_demand_mwh: Estimated total electricity demand for that state and hour, in MWh. This is the reported FERC Form 714 hourly demand scaled up or down linearly such that the total annual electricity demand matches the total annual electricity sales reported at the state level in the EIA Form 861.

A collection of plots are also included, comparing the original and scaled demand time series for each state.

Acknowledgements

This analysis was funded largely by GridLab, and done in collaboration with researchers at the Lawrence Berkeley National Laboratory, including Umed Paliwal and Nikit Abhyankar.

• Ethan Welty wrote the final code and most of the algorithms.
• Yash Kumar did initial data explorations and geospatial analyses.

The data screening methods were originally designed to identify unrealistic data in the electricity demand timeseries reported to EIA on Form 930, and have been applied here to data form the FERC Form 714.

They are adapted from code published and modified by:

• Tyler Ruggles
• Greg Schivley

And described at:

• Developing reliable hourly electricity demand data through screening and imputation
• EIA Cleaned Hourly Electricity Demand Code (Zenodo)
• EIA Cleaned Hourly Electricity Demand Code (GitHub)

The imputation methods were designed for multivariate time series forecasting.

They are adapted from code published by:

• Xinyu Chen chenxy346@gmail.com

And described at:

• Low-Rank Autoregressive Tensor Completion for Multivariate Time Series Forecasting
• Scalable Low-Rank Tensor Learning for Spatiotemporal Traffic Data Imputation
• Tensor Learning (张量学习)

About PUDL & Catalyst Cooperative

For additional information about this data and PUDL, see the following resources:

• The PUDL Repository on GitHub
• The PUDL Documentation
• Other Catalyst Cooperative data archives on Zenodo

These files contain data that are submitted to the Energy Information Administration (EIA) by selected electric utilities on Form-826, "Monthly Electric Utility Sales and Revenue Report with State distributions." Form EIA-826 is used to collect retail sales of electricity (in megawatthours) and associated revenue (in thousands of dollars). Respondents are chosen from a sample of electric utilities in the United States that complete Form EIA-861, "Annual Electric Utility Report." (See CISER codebook number ECON-070).

The Annual Electric Utility Report, taken from Form EIA-861, collects information on the status of electric utilities and their generation, transmission, and distribution of electric energy in the United States, its territories, and Puerto Rico. The data from this form are used to accurately maintain the EIA electric utility frame, to draw samples for other electric power surveys, and to provide input for other EIA publications. The data include average revenue per kilowatthour, average monthly consumption, and operating revenue from sales.

This API provides data on retail sales of electricity by major end-use sectors, i.e., residential, commercial, industrial, and transportation. Based on Form EIA-826 and Form EIA-861 data. Annual, quarterly, and monthly data available.

Data on annual emissions of Carbon Dioxide (CO2), Sulfur Dioxide (SO2), and Nitrogen Oxides (NOx). Data organized by type of electric power producer, by energy source, and by U.S. state. Annual time series extend back to 1990. Based on Form EIA-861 data. Electric Power Producer: Commercial Cogen, Commercial Non-Cogen, Electric Utility, Industrial Cogen, Industrial Non-Cogen, IPP NAICS-22 Cogen, IPP NAICS-22 Non-Cogen, and Total Electric Power Industry Energy Source: Coal, Geothermal, Natural Gas, Other, Other Biomass, Other Gases, Wood and Wood Derived Fuels, Petroleum, and All Energy Sources

This data deposit constructs data on monthly generation costs and capacities in the United States from 1999-2012 in preparation for their use in "Imperfect Markets versus Imperfect Regulation in U.S. Electricity Generation" (openicpsr-115467).It builds panel data files from the following EIA forms:"Form EIA-860: Annual Electric Generator Report""Form EIA-861: Annual Electric Power Industry Report""Form EIA-767: Annual Steam-Electric Plant Operation and Design Report""Form EIA-923: Power Plant Operations Report""Form EIA-759/906/920/923: Power Plant Report""Form EIA-423: Monthly Cost and Quality of Fuels for Electric Plants Report"and the EPA's Continuous Emissions Monitor System.It constructs a crosswalk that connects the EPA's boilers to the EIA's generator identifiers.

Annual data back to 2003 at the national level for electricity generation; capacity; consumption and cost of fossil fuels; sales, price and revenue; emissions; demand-side management; and operating revenues, expenses, and income. Based on Form EIA-860 and Form EIA-861 data.

Annual data on the average price of retail electricity to consumers. Data organized by U.S. state and by provider, i.e., total electric industry, full-service providers, restructured retail service providers, energy-only providers, and delivery-only service. Annual time series extend back to 1990. Based on Form EIA-861 data.

Historical U.S. electric utility data. Data on generation, electric purchases, peak load, sales, revenues, customer counts, demand-side management programs, green pricing, net metering programs, and distributed generation capacity. Based on EIA Form-861 data.

The U.S. Energy Information Administration complies this calendar-year data to show a comparison for high voltage service. Source: Form EIA-861 2016 Data.

Data on sales, costs, rates, area, rates of return and equity, and illustrative bills for power providers in California. Source data documents and cleaned tables are included.

Datasets included: 1. CCA JRC PDFs: Joint Rate Comparisons for Community Choice Aggregators, scraped from IOU website via the Wayback Machine 2. EIA 861 Tables: collected EIA Form 861 datasets and associated datacleaning code 3. POU Financial Statements: annual audited financials of LADWP and SMUD 4. Residential and Commercial Rate Source PDFs: for IOUs and POUs, source files for rate time series 5. Appendix Data Part 1 - Rates, Usage, Area, Costs: 5a: Core CPI rates 5b: Residential and Commercial Rate time series 5c: IOU rate base 5d: IOU O&M 5e: POU utility plant 5f: POU O&M 5g: CCA rate comparisons 5h: detailed IOU O&M data 5i: consolidated EIA sales data (all CA LSEs) 5j: consolidated EIA sales data with average prices (selected CA and non-CA LSEs) 5k: CCA service area calculation 5l: interstate baseline residential rate comparison 5m: extended sample of CA POUs O&M and utility plant 6. Appendix Data Part 2 - Illustrative Monthly Bills: comparisons between CCA and IOUs, with customer load shape and intermediate calculations 7. Appendix Data Part 3 - ROE and ROR: time series of rates of return and return on equity for selected IOUs 8. PUDL FERC1 Processing Code: Python code to pull latest stable PUDL dataset and extract IOU O&M data as required

The U.S. Energy Information Administration complies this calendar-year data to show a comparison for commercial rates. Source: Form EIA-861 2016 Data.

The U.S. Energy Information Administration complies this calendar-year data to show a comparison for residential rates. Source: Form EIA-861 2016 Data.

Historical U.S. electric utility data. Data on generation, electric purchases, peak load, sales, revenues, customer counts, demand-side management programs, green pricing, net metering programs, and distributed generation capacity. Based on EIA Form-861 data. Data contained in a zip file.

The Annual Electric Utility Report, taken from Form EIA-861, collects information on the status of electric utilities and their generation, transmission, and distribution of electric energy in the United States, its territories, and Puerto Rico. The data from this form are used to accurately maintain the EIA electric utility frame, to draw samples for other electric power surveys, and to provide input for other EIA publications. The data include average revenue per kilowatthour, average monthly consumption, and operating revenue from sales.