The dataset is an excerpt of the validation dataset used in:

Ruiz-Arias JA, Gueymard CA. Review and performance benchmarking of 1-min solar irradiance components separation methods: The critical role of dynamically-constrained sky conditions. Submitted for publication to Renewable and Sustainable Energy Reviews.

and it is ready to use in the Python package splitting_models developed during that research. See the documentation in the Python package for usage details. Below, there is a detailed description of the dataset.

The data is in a single parquet file that contains 1-min time series of solar geometry, clear-sky solar irradiance simulations, solar irradiance observations and CAELUS sky types for 5 BSRN sites, one per primary Köppen-Geiger climate, namely: Minamitorishima (mnm), JP, for equatorial climate; Alice Springs (asp), AU, for dry climate; Carpentras (car), FR, for temperate climate; Bondville (bon), US, for continental climate; and Sonnblick (son), AT, for cold/polar/snow climate. It includes one calendar year per site. The BSRN data is publicly available. See download instructions in https://bsrn.awi.de/data.

The specific variables included in the dataset are:

• climate: primary Köppen-Geiger climate. Values are: A (equatorial), B (dry), C (temperate), D (continental) and E (polar/snow).
• longitude: longitude, in degrees east.
• latitude: latitude, in degrees north.
• sza: solar zenith angle, in degrees.
• eth: extraterrestrial solar irradiance (i.e., top of atmosphere solar irradiance), in W/m2.
• ghics: clear-sky global solar irradiance, in W/m2. It is evaluated with the SPARTA clear-sky model and MERRA-2 clear-sky atmosphere.
• difcs: clear-sky diffuse solar irradiance, in W/m2.It is evaluated with the SPARTA clear-sky model and MERRA-2 clear-sky atmosphere.
• ghicda: clean-and-dry clear-sky global solar irradiance, in W/m2. It is evaluated with the SPARTA clear-sky model and MERRA-2 clear-sky atmosphere, prescribing zero aerosols and zero precipitable water.
• ghi: observed global horizontal irradiance, in W/m2.
• dif: observed diffuse irradiance, in W/m2.
• sky_type: CAELUS sky type. Values are: 1 (unknown), 2 (overcast), 3 (thick clouds), 4 (scattered clouds), 5 (thin clouds), 6 (cloudless) and 7 (cloud enhancement).

The dataset can be easily loaded in a Python Pandas DataFrame as follows:

import pandas as pd

data = pd.read_parquet(

The dataframe has a multi-index with two levels: times_utc and site. The former are the UTC timestamps at the center of each 1-min interval. The latter is each site's label.

Monthly 30-year "normal" dataset covering the conterminous U.S., averaged over the climatological period 1991-2020. Contains spatially gridded average annual daily total clear-sky global shortwave radiation on a horizontal plane at 4km grid cell resolution. Clear sky radiation was modeled with the Image Processing Workbench (IPW) and MERRA-2 satellite data. Distribution of effective cloud transmittance point measurements to the spatial grid was accomplished using the PRISM model. Total shortwave radiation on horizontal and sloped ground surfaces was modeled with IPW, using the clear-sky and transmittance grids as inputs. Terrain data was derived from a 30-arc-sec version of the US National Elevation Database. Modeled values incorporate the effects of elevation, shading, and reflection from nearby terrain on solar radiation. This dataset is available free-of-charge on the PRISM website.

The National Solar Radiation Database (NSRDB) is a serially complete collection of meteorological and solar irradiance data sets for the United States and a growing list of international locations for 1998-2023. The NSRDB is updated annually and provides foundational information to support U.S. Department of Energy programs, research, industry and the general public.

The NSRDB provides time-series data at 30-minute resolution of resource averaged over surface cells of 0.038 degrees in both latitude and longitude, or nominally 4 km in size. Additionally time series data at 5 minutes for the US and 10 minutes for North, Central and South America at 2 km resolution are produced from the next generation of GOES satellites and made available from 2019. The solar radiation values represent the resource available to solar energy systems. The data was created using cloud properties which are generated using the AVHRR Pathfinder Atmospheres-Extended (PATMOS-x) algorithms developed by the University of Wisconsin. Fast all-sky radiation model for solar applications (FARMS) in conjunction with the cloud properties, and aerosol optical depth (AOD) and precipitable water vapor (PWV) from ancillary source are used to estimate solar irradiance (GHI, DNI, and DHI). The Global Horizontal Irradiance (GHI) is computed for clear skies using the REST2 model. For cloud scenes identified by the cloud mask, FARMS is used to compute GHI and FARMS DNI is used to compute the Direct Normal Irradiance (DNI). The PATMOS-X model uses radiance images in visible and infrared channels from the Geostationary Operational Environmental Satellite (GOES) series of geostationary weather satellites. Ancillary variables needed to run REST2 and FARMS (e.g., aerosol optical depth, precipitable water vapor, and albedo) are derived from NASA's Modern Era-Retrospective Analysis (MERRA-2) dataset. Temperature and wind speed data are also derived from MERRA-2 and provided for use in NREL's System Advisor Model (SAM) to compute PV generation.

This dataset provides historical values of global, direct and diffuse solar irradiation, as well as direct normal irradiation, on a latitude/longitude grid covering land surfaces and coastal areas of Europe, Africa, Oceania, Eastern South America, the Middle East and South-East Asia. It is created from 15 minute resolved timeseries at each grid point. These timeseries were calculated by the CAMS Solar Radiation Time Series Service and use information on aerosol, ozone and water vapour from the CAMS global forecasting system. Other properties, such as ground albedo and ground elevation, are also taken into account. Data is provided for both clear-sky and observed cloud conditions. For cloudy conditions high-resolution cloud information is directly inferred from satellite observations provided by the Meteosat Second Generation (MSG) and Himawari 8 satellites. It is the Himawari satellite that provides the Asian coverage, which is only available from 2016 and v4.6 (rev2) onwards. The aim of the dataset is to fulfil the needs of European and national policy development and the requirements of both commercial and public downstream services, e.g. for planning, monitoring, efficiency improvements and the integration of solar energy systems into energy supply grids. Data is offered in monthly netCDF formatted files.

Data repository for measurements from a solar measurement station in Bangladesh. Data will be uploaded in batches, on a monthly basis, and will transmit daily reports on 1 minute average values for solar radiation levels, temperature, air pressure and wind speed. For download access to GIS layers, please visit the Global Solar Atlas: http://globalsolaratlas.info/

The text file "Solar radiation.txt" contains hourly solar radiation data in Langleys and associated data-source flags from January 1, 1948, to September 30, 2021. The primary data for water year 2021 (a water year is the 12-month period, October 1 through September 30, designated by the calendar year in which it ends) were downloaded from the Argonne National Laboratory (ANL) (Argonne National Laboratory, 2022) and processed following the guidelines documented in Over and others (2010). The processed data were appended to ARGN20.WDM (Bera and Over, 2021) and renamed as ARGN21.WDM. Missing and apparently erroneous data values were replaced with adjusted values from nearby weather stations used as "backup." Temporal variations in the statistical properties of the data resulting from changes in measurement and data storage methodologies were adjusted to match the statistical properties resulting from the data collection procedures that have been in place since January 1, 1989 (Over and others, 2010). The adjustments were computed based on the regressions between the primary data series from ANL and the backup series using data obtained during common periods. The statistical properties of the regressions were used to assign estimated standard errors to values that were adjusted or filled from other series. The Illinois Climate Network (Water and Atmospheric Resources Monitoring Program, 2022) stations at St. Charles and DeKalb, Illinois, were used as "backup." Each data source flag is of the form "xyz", which allows the user to determine its source and the methods used to process the data (Over and others, 2010). References Cited: Argonne National Laboratory, 2022, Meteorological data, accessed on January 17, 2022, at https://www.atmos.anl.gov/ANLMET/numeric/. Bera, M., 2021, Meteorological Database, Argonne National Laboratory, Illinois, January 1, 1948 - September 30, 2020: U.S. Geological Survey data release, https://doi.org/10.5066/P9GP8COF. Over, T.M., Price, T.H., and Ishii, A.L., 2010, Development and analysis of a meteorological database, Argonne National Laboratory, Illinois: U.S. Geological Survey Open-File Report 2010-1220, 67 p., http://pubs.usgs.gov/of/2010/1220/. Water and Atmospheric Resources Monitoring Program. Illinois Climate Network, 2022. Illinois State Water Survey, 2204 Griffith Drive, Champaign, IL 61820-7495. Data accessed on January 4, 2022, at http://dx.doi.org/10.13012/J8MW2F2Q.

The UK hourly solar radiation data contain the amount of solar irradiance received during the hour ending at the specified time. All sites report 'global' radiation amounts. This is also known as 'total sky radiation' as it includes both direct solar irradiance and 'diffuse' irradiance as a result of light scattering. Some sites also provide separate diffuse and direct irradiation amounts, depending on the instrumentation at the site. For these the sun's path is tracked with two pyrometers - one where the path to the sun is blocked by a suitable disc to allow the scattered sunlight to be measured to give the diffuse measurement, while the other has a tube pointing at the sun to measure direct solar irradiance whilst blanking out scattered sun light.

For details about the different measurements made and the limited number of sites making them please see the MIDAS Solar Irradiance table linked to in the online resources section of this record.

This version supersedes the previous version of this dataset and a change log is available in the archive, and in the linked documentation for this record, detailing the differences between this version and the previous version. The change logs detail new, replaced and removed data. These include the addition of data for calendar year 2023.

The data were collected by observation stations operated by the Met Office across the UK and transmitted within the following message types: SYNOP, HCM, AWSHRLY, MODLERAD, ESAWRADT and DRADR35 messages. The data spans from 1947 to 2023.

This dataset is part of the Midas-open dataset collection made available by the Met Office under the UK Open Government Licence, containing only UK mainland land surface observations owned or operated by the Met Office. It is a subset of the fuller, restricted Met Office Integrated Data Archive System (MIDAS) Land and Marine Surface Stations dataset, also available through the Centre for Environmental Data Analysis - see the related dataset section on this record.

Generating estimates of daily reference photosynthetically Active Radiation (PAR). We show the procedure to generate estimates of daily reference PAR using solar radiation data. The input for the R script (CalculateDailyPAR.R) is a raw time series of hourly solar radiation (stored in variable ‘ws’) that for our case was obtained from the CIMIS website (station id: 105) [California Department of Water Resources, 2015]. The script processes the data set to format the date and time columns, and to identify missing data points reporting their position within the time series (variable ‘na.id’). The user fills the gaps using adequate strategies and creates a new input file (stored in variable ‘fill.points’) containing the values to fill in within the time series. A reference PAR estimate is obtained as a constant fraction of solar radiation using the conversion factor proposed by [Meek et al., 1984]. The script then calculates an average daily value of solar radiation and integrates the reference PAR over the daytime period to obtain a daily value. The script ends by generating a final table (‘ws.results’) reporting daily values of solar radiation (maximum and mean in W m-2), and maximum, mean, and minimum reference PAR values in units of (μmol m-2 d-1) and (mol m-2 d-1). DOI:10.6084/m9.figshare.3412765

This data provides monthly average daily total solar resource averaged over surface cells of 0.038 degrees in both latitude and longitude, or nominally 4 km in size. The solar radiation values represent the resource available to solar energy systems. The data was created using cloud properties which are generated using the AVHRR Pathfinder Atmospheres-Extended (PATMOS-x) algorithms. Fast all-sky radiation model for solar applications (FARMS) in conjunction with the cloud properties, and aerosol optical depth (AOD) and precipitable water vapor (PWV) from ancillary source are used to estimate direct normal irradiance (DNI) and global horizontal irradiance (GHI). The DNI and GHI are computed for clear skies using the REST2 model. For cloud scenes identified by the cloud mask, the FARMS is used to compute the GHI. The DNI for cloud scenes is then computed using the DISC model. The data are averaged from hourly model output over 19 years (1998-2016). The PATMOS-X model uses half-hourly radiance images in visible and infrared channels from the GOES series of geostationary weather satellites, daily snow cover data from the NSIDC and mixing ratio, temperature and pressure profiles from the Modern Era-Retrospective Analysis (MERRA-2) dataset. The REST2 model uses hourly aerosol optical depth from MERRA-2 to calculate GHI and DNI; water vapor and other inputs for REST 2 are obtained from the MERRA-2. This dataset was derived from the NSRDB and may be used with the following citation: Sengupta, M., Xie, Y., Lopez, A., Habte, A., Maclaurin, G., & Shelby, J. (2018). The national solar radiation data base (NSRDB). Renewable and Sustainable Energy Reviews, 89, 51-60.

Released to the public as part of the Department of Energy's Open Energy Data Initiative, the National Solar Radiation Database (NSRDB) is a serially complete collection of hourly and half-hourly values of the three most common measurements of solar radiation – global horizontal, direct normal, and diffuse horizontal irradiance — and meteorological data. These data have been collected at a sufficient number of locations and temporal and spatial scales to accurately represent regional solar radiation climates.

Data on daily global solar radiation (Please visit the reference link for other climate information). The multiple file formats are available for datasets download in API.

The text file "Solar radiation.txt" contains hourly data and associated data-source flag from January 1, 1948, to September 30, 2015. The primary source of the data is the Argonne National Laboratory, Illinois. The first four columns give year, month, day and hour of the observation. Column 5 is the data in Langleys. Column 6 is the three-digit data-source flag to identify the solar radiation data processing and they indicate if the data are original or missing, the method that was used to fill the missing periods, and any other transformations of the data. Bera (2014) describes in detail an addition of a new flag based on the regression analysis of the backup data series at St. Charles (STC) for water years (WY) 2008–10. The user of the data should consult Over and others (2010) and Bera (2014) for the detailed documentation of this hourly data-source flag series. Reference Cited: Over, T.M., Price, T.H., and Ishii, A.L., 2010, Development and analysis of a meteorological database, Argonne National Laboratory, Illinois: U.S. Geological Survey Open File Report 2010-1220, 67 p., http://pubs.usgs.gov/of/2010/1220/. Bera, M., 2014, Watershed Data Management (WDM) database for Salt Creek streamflow simulation, DuPage County, Illinois, water years 2005-11: U.S. Geological Survey Data Series 870, 18 p., http://dx.doi.org/10.3133/ds870.

Ground measurement data from 3 solar meteorological stations in Zambia. Data contains 1 minute average values for solar radiation, air temperature, relative humidity, barometric pressure, precipitation, wind speed and wind direction, cleaned and soiled radiance sensor (soiling measurement) and cleaning events.

The National Solar Radiation Data Base (NSRDB) is the most comprehensive collection of solar data freely available. It is a serially complete collection of meteorological and solar irradiance data sets for the United States and a growing list of international locations. The data are publicly available at no cost to the user. The NSRDB provides foundational information to support U.S. Department of Energy programs, research, and the general public. Comparable products are also available from commercial vendors.

Mean annual solar radiation data layer used in the creation of Land Environments of New Zealand (LENZ) classification. The classification layers have been made publicly available by the Ministry for the Environment (see https://data.mfe.govt.nz/layers/?q=LENZ for to access these layers). Development of surfaces for annual and monthly solar radiation required substantially more data preparation than for the other climate surfaces, reflecting the small number of stations at which solar radiation has traditionally been measured. Monthly estimates of average daily solar radiation to 1980 were available for 22 meteorological stations, but measurements of sunshine hours were available for a total of 98 stations, including 18 of the stations for which solar radiation measurements were available. To extract as much information as possible from these data, a surface was fitted first that predicted for each month the ratio of solar radiation reaching the earth's surface to that reaching the top of the atmosphere, with the latter calculated from solar geometry equations. In fitting this surface, only the 18 data points where measurements were made of both solar radiation and sunshine hours were used. In addition to NZMG coordinates, it used as an additional predictor the ratio of measured sunshine hours for each month to the maximum possible sunshine hours given no cloud. This surface was then used to estimate the monthly solar radiation received at each of the 80 sites for which measurements of sunshine hours alone were available. Using a total of 98 sites for which solar radiation data were either measured directly or estimated from sunshine hours, surfaces predicting annual and monthly solar radiation were then fitted. Data describing monthly humidity was used as a surrogate measure of cloudiness to improve the fit of the surface to the underlying data. This also increases the local accuracy of the surface predictions, as the number of meteorological stations used to fit the humidity surface is more than three times greater than the number of sites used to fit the solar radiation surface. The units for this layer are in MJ/m2/day, higher values signifiy areas that have higher levels of solar radiation. This layer has been multiplied by a factor of 10 (i.e. converted into an integer grid) to save space and make the grids more responsive. A value of 123 is actually 12.3 MJ/m2/day. Additional details such as the climate station locations used in the creation of the layer and error maps are defined in the attached LENZ Technical Guide.

Data repository for ground measurements and satellite data from 4 meteorological stations in Maldives. Data contains 1 minute average values for solar radiation levels, air temperature, relative humidity, wind speed and atmospheric pressure. Delivered files: Solar-Measurements_Maldives_sitename_WB-ESMAP_Header metadata for ground measurements files. Solar-Measurements_Maldives_sitename_WB-ESMAP_Raw raw ground measurements from datalogger. Do not use for further development. Solar-Measurements_Maldives_sitename_WB-ESMAP_QC quality checked ground measurements from dataloger. Solar-Measurements_Maldives_WB-ESMAP_SatelliteTS site adapted time series of satellite data. Solar-Measurements_Maldives_WB-ESMAP_SatelliteTMY Typical meteorological year data file (P50) based on site adapted time series of satellite data. For more information and additional outputs, please visit: http://esmap.org/re_mapping_madlvies For download access to GIS layers, please visit the Global Solar Atlas: http://globalsolaratlas.info/ Please cite as: [Data/information/map obtained from the] “World Bank via ENERGYDATA.info, under a project funded by the Energy Sector Management Assistance Program (ESMAP). For more information: Maldives-Solar Radiation Measurement Data, 2018,

Ground measurement data from 3 solar meteorological stations in Zambia. Data contains 1 minute average values for solar radiation, air temperature, relative humidity, barometric pressure, precipitation, wind speed and wind direction, cleaned and soiled radiance sensor (soiling measurement) and cleaning events.

Data repository for measurements from 9 automated solar stations in Pakistan. Data will be uploaded in batches, on a monthly basis, and will transmit daily reports on 10 minute average values for solar radiation levels, temperature, air pressure and wind speed. From 2018 onward, the measurement stations are being operated by the Government of Pakistan, together with NREL and USAID. For more information and additional outputs, please visit: https://esmap.org/node/3058. For download access to GIS layers, please visit the Global Solar Atlas: http://globalsolaratlas.info/ Please cite as: [Data/information/map obtained from the] “World Bank via ENERGYDATA.info, under a project funded by the Energy Sector Management Assistance Program (ESMAP). For more information: Pakistan-Solar Radiation Measurement Data, 2017,

This text file "Solar radiation.txt" contains hourly data in Langleys and associated data-source flag from January 1, 1948, to September 30, 2016. The primary source of the data is the Argonne National Laboratory, Illinois. The data-source flag consist of a three-digit sequence in the form "xyz" that describe the origin and transformations of the data values. They indicate if the data are original or missing, the method that was used to fill the missing periods, and any other transformations of the data. Bera (2014) describes in detail an addition of a new data-source flag based on the regression analysis of the backup data series at St. Charles (STC) for water years (WY) 2008-10. The user of the data should consult Over and others (2010) and Bera (2014) for the detailed documentation of the data-source flag. Reference Cited: Over, T.M., Price, T.H., and Ishii, A.L., 2010, Development and analysis of a meteorological database, Argonne National Laboratory, Illinois: U.S. Geological Survey Open File Report 2010-1220, 67 p., http://pubs.usgs.gov/of/2010/1220/. Bera, M., 2014, Watershed Data Management (WDM) database for Salt Creek streamflow simulation, DuPage County, Illinois, water years 2005-11: U.S. Geological Survey Data Series 870, 18 p., http://dx.doi.org/10.3133/ds870.

Context

Space Apps Moscow was held on April 29th & 30th. Thank you to the 175 people who joined the International Space Apps Challenge at this location!

Content

The dataset contains such columns as: "wind direction", "wind speed", "humidity" and temperature. The response parameter that is to be predicted is: "Solar_radiation". It contains measurements for the past 4 months and you have to predict the level of solar radiation. Just imagine that you've got solar energy batteries and you want to know will it be reasonable to use them in future?

Acknowledgements

Thanks NASA for the dataset.

Inspiration

Predict the level of solar radiation. Here are some intersecting dependences that i have figured out: 1. Humidity & Solar_radiation. 2.Temeperature & Solar_radiation.

The best result of accuracy I could get using cross-validation was only 55%.