This dataset contains information on the Surface Soil Moisture (SM) content derived from satellite observations in the microwave domain.

Dataset Paper (Open Access)

A description of this dataset, including the methodology and validation results, is available at:

Preimesberger, W., Stradiotti, P., and Dorigo, W.: ESA CCI Soil Moisture GAPFILLED: an independent global gap-free satellite climate data record with uncertainty estimates, Earth Syst. Sci. Data, 17, 4305–4329, https://doi.org/10.5194/essd-17-4305-2025, 2025.

Abstract

ESA CCI Soil Moisture is a multi-satellite climate data record that consists of harmonized, daily observations coming from 19 satellites (as of v09.1) operating in the microwave domain. The wealth of satellite information, particularly over the last decade, facilitates the creation of a data record with the highest possible data consistency and coverage.
However, data gaps are still found in the record. This is particularly notable in earlier periods when a limited number of satellites were in operation, but can also arise from various retrieval issues, such as frozen soils, dense vegetation, and radio frequency interference (RFI). These data gaps present a challenge for many users, as they have the potential to obscure relevant events within a study area or are incompatible with (machine learning) software that often relies on gap-free inputs.
Since the requirement of a gap-free ESA CCI SM product was identified, various studies have demonstrated the suitability of different statistical methods to achieve this goal. A fundamental feature of such gap-filling method is to rely only on the original observational record, without need for ancillary variable or model-based information. Due to the intrinsic challenge, there was until present no global, long-term univariate gap-filled product available. In this version of the record, data gaps due to missing satellite overpasses and invalid measurements are filled using the Discrete Cosine Transform (DCT) Penalized Least Squares (PLS) algorithm (Garcia, 2010). A linear interpolation is applied over periods of (potentially) frozen soils with little to no variability in (frozen) soil moisture content. Uncertainty estimates are based on models calibrated in experiments to fill satellite-like gaps introduced to GLDAS Noah reanalysis soil moisture (Rodell et al., 2004), and consider the gap size and local vegetation conditions as parameters that affect the gapfilling performance.

Summary

• Gap-filled global estimates of volumetric surface soil moisture from 1991-2023 at 0.25° sampling
• Fields of application (partial): climate variability and change, land-atmosphere interactions, global biogeochemical cycles and ecology, hydrological and land surface modelling, drought applications, and meteorology
• Method: Modified version of DCT-PLS (Garcia, 2010) interpolation/smoothing algorithm, linear interpolation over periods of frozen soils. Uncertainty estimates are provided for all data points.
• More information: See Preimesberger et al. (2025) and https://doi.org/10.5281/zenodo.8320869" target="_blank" rel="noopener">ESA CCI SM Algorithm Theoretical Baseline Document [Chapter 7.2.9] (Dorigo et al., 2023)

Programmatic Download

You can use command line tools such as wget or curl to download (and extract) data for multiple years. The following command will download and extract the complete data set to the local directory ~/Download on Linux or macOS systems.

#!/bin/bash

# Set download directory
DOWNLOAD_DIR=~/Downloads

base_url="https://researchdata.tuwien.at/records/3fcxr-cde10/files"

# Loop through years 1991 to 2023 and download & extract data
for year in {1991..2023}; do
  echo "Downloading $year.zip..."
  wget -q -P "$DOWNLOAD_DIR" "$base_url/$year.zip"
  unzip -o "$DOWNLOAD_DIR/$year.zip" -d $DOWNLOAD_DIR
  rm "$DOWNLOAD_DIR/$year.zip"
done

Data details

The dataset provides global daily estimates for the 1991-2023 period at 0.25° (~25 km) horizontal grid resolution. Daily images are grouped by year (YYYY), each subdirectory containing one netCDF image file for a specific day (DD), month (MM) in a 2-dimensional (longitude, latitude) grid system (CRS: WGS84). The file name has the following convention:

ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_GAPFILLED-YYYYMMDD000000-fv09.1r1.nc

Data Variables

Each netCDF file contains 3 coordinate variables (WGS84 longitude, latitude and time stamp), as well as the following data variables:

• sm: (float) The Soil Moisture variable reflects estimates of daily average volumetric soil moisture content (m3/m3) in the soil surface layer (~0-5 cm) over a whole grid cell (0.25 degree).
• sm_uncertainty: (float) The Soil Moisture Uncertainty variable reflects the uncertainty (random error) of the original satellite observations and of the predictions used to fill observation data gaps.
• sm_anomaly: Soil moisture anomalies (reference period 1991-2020) derived from the gap-filled values (`sm`)
• sm_smoothed: Contains DCT-PLS predictions used to fill data gaps in the original soil moisture field. These values are also provided for cases where an observation was initially available (compare `gapmask`). In this case, they provided a smoothed version of the original data.
• gapmask: (0 | 1) Indicates grid cells where a satellite observation is available (1), and where the interpolated (smoothed) values are used instead (0) in the 'sm' field.
• frozenmask: (0 | 1) Indicates grid cells where ERA5 soil temperature is <0 °C. In this case, a linear interpolation over time is applied.

Additional information for each variable is given in the netCDF attributes.

Version Changelog

Changes in v9.1r1 (previous version was v09.1):

• This version uses a novel uncertainty estimation scheme as described in Preimesberger et al. (2025).

Software to open netCDF files

These data can be read by any software that supports Climate and Forecast (CF) conform metadata standards for netCDF files, such as:

• https://github.com/pydata/xarray" target="_blank" rel="noopener">Xarray (python)
• https://unidata.github.io/netcdf4-python/" target="_blank" rel="noopener">netCDF4 (python)
• https://github.com/TUW-GEO/esa_cci_sm">esa_cci_sm (python)
• Similar tools exists for other programming languages (Matlab, R, etc.)
• Software packages and GIS tools can open netCDF files, e.g. CDO, NCO, QGIS, ArCGIS
• You can also use the GUI software Panoply to view the contents of each file

References

• Preimesberger, W., Stradiotti, P., and Dorigo, W.: ESA CCI Soil Moisture GAPFILLED: an independent global gap-free satellite climate data record with uncertainty estimates, Earth Syst. Sci. Data, 17, 4305–4329, https://doi.org/10.5194/essd-17-4305-2025, 2025.
• Dorigo, W., Preimesberger, W., Stradiotti, P., Kidd, R., van der Schalie, R., van der Vliet, M., Rodriguez-Fernandez, N., Madelon, R., & Baghdadi, N. (2023). ESA Climate Change Initiative Plus - Soil Moisture Algorithm Theoretical Baseline Document (ATBD) Supporting Product Version 08.1 (version 1.1). Zenodo. https://doi.org/10.5281/zenodo.8320869
• Garcia, D., 2010. Robust smoothing of gridded data in one and higher dimensions with missing values. Computational Statistics & Data Analysis, 54(4), pp.1167-1178. Available at: https://doi.org/10.1016/j.csda.2009.09.020
• Rodell, M., Houser, P. R., Jambor, U., Gottschalck, J., Mitchell, K., Meng, C.-J., Arsenault, K., Cosgrove, B., Radakovich, J., Bosilovich, M., Entin, J. K., Walker, J. P., Lohmann, D., and Toll, D.: The Global Land Data Assimilation System, Bulletin of the American Meteorological Society, 85, 381 – 394, https://doi.org/10.1175/BAMS-85-3-381, 2004.

Related Records

The following records are all part of the ESA CCI Soil Moisture science data records community

PV-gradient tropopause time series

General description

These datasets contain time series of the PV-gradient tropopause (PVG tropopause) introduced by A. Kunz (2011, doi:10.1029/2010JD014343) and calculated by K. Turhal (2024, paper " Variability and Trends in the PVG Tropopause", preprint in EGUsphere: https://doi.org/10.5194/egusphere-2024-471).

Data and methods

The PVG tropopause has been computed by means of the Eddy Tracking Toolkit (developed by J. Clemens and K. Turhal, to be published):

• from four reanalyses: ERA5, ERA-Interim, MERRA-2 and JRA-55
• for the time range 1980/01/01 -- 2017/12/31 in time steps of the according reanalyses, i.e. four times daily at 00h, 06h, 12h and 18h
• on each isentropic level, with potential temperatures (theta) ranging from 320 K to 380 K, in steps of 5 K for ERA5 and 10 K for the other reanalyses.

Contents

Datasets are provided for each year and isentropic level in NetCDF4 format, every file consisting of two groups for the northern and southern hemisphere. Each group contains the following variables, with time as dimension:

• time in seconds since 2000/01/01 00:00 UTC
• u_lim: Zonal wind speed at the PVG tropopause
• vh_lim: Horizontal wind speed at the PVG tropopause
• q_lim: Maximum of Q = vh * Grad PV
• eqlat_lim: Location of the PVG tropopause in equivalent latitudes
• latmean_lim: Location of the PVG tropopause in latitudes
• pv_lim: PV value at the PVG tropopause

In this upload, the PVG tropopause time series are included as *.zip files:

• ERA5 dataset: "pvg-tp_era5_ts.zip"
• ERA-Interim dataset: "pvg-tp_eraint_ts.zip"
• MERRA-2 dataset: "pvg-tp_merra2_ts.zip"
• JRA-55 dataset: "pvg-tp_jra55_ts.zip"
• Plots of time series for each reanalysis of the variables eqlat_lim, latmean_lim and pv_lim: "pvg_tropopause_timeseries_plots.zip".

How to use

The variables in these netCDF files are grouped by hemisphere. To read in the data, specify the group first ("NorthernHemisphere" or "SouthernHemisphere") and then the variable name (see list above). In Python, this can be done as follows:

import netCDF4 as nc

file="

If you would like to read in all variables in both hemispheres, you can loop e.g. as follows:

import netCDF4 as nc

file = "

Funding

This project has been funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – TRR 301 – Project-ID 428312742, TPChange: The Tropopause Region in a Changing Atmosphere (https://tpchange.de/).

Data Summary: US states grid mask file and NOAA climate regions grid mask file, both compatible with the 12US1 modeling grid domain. Note:The datasets are on a Google Drive. The metadata associated with this DOI contain the link to the Google Drive folder and instructions for downloading the data. These files can be used with CMAQ-ISAMv5.3 to track state- or region-specific emissions. See Chapter 11 and Appendix B.4 in the CMAQ User's Guide for further information on how to use the ISAM control file with GRIDMASK files. The files can also be used for state or region-specific scaling of emissions using the CMAQv5.3 DESID module. See the DESID Tutorial and Appendix B.4 in the CMAQ User's Guide for further information on how to use the Emission Control File to scale emissions in predetermined geographical areas. File Location and Download Instructions: Link to GRIDMASK files Link to README text file with information on how these files were created File Format: The grid mask are stored as netcdf formatted files using I/O API data structures (https://www.cmascenter.org/ioapi/). Information on the model projection and grid structure is contained in the header information of the netcdf file. The output files can be opened and manipulated using I/O API utilities (e.g. M3XTRACT, M3WNDW) or other software programs that can read and write netcdf formatted files (e.g. Fortran, R, Python). File descriptions These GRIDMASK files can be used with the 12US1 modeling grid domain (grid origin x = -2556000 m, y = -1728000 m; N columns = 459, N rows = 299). GRIDMASK_STATES_12US1.nc - This file containes 49 variables for the 48 states in the conterminous U.S. plus DC. Each state variable (e.g., AL, AZ, AR, etc.) is a 2D array (299 x 459) providing the fractional area of each grid cell that falls within that state. GRIDMASK_CLIMATE_REGIONS_12US1.nc - This file containes 9 variables for 9 NOAA climate regions based on the Karl and Koss (1984) definition of climate regions. Each climate region variable (e.g., CLIMATE_REGION_1, CLIMATE_REGION_2, etc.) is a 2D array (299 x 459) providing the fractional area of each grid cell that falls within that climate region. NOAA Climate regions: CLIMATE_REGION_1: Northwest (OR, WA, ID) CLIMATE_REGION_2: West (CA, NV) CLIMATE_REGION_3: West North Central (MT, WY, ND, SD, NE) CLIMATE_REGION_4: Southwest (UT, AZ, NM, CO) CLIMATE_REGION_5: South (KS, OK, TX, LA, AR, MS) CLIMATE_REGION_6: Central (MO, IL, IN, KY, TN, OH, WV) CLIMATE_REGION_7: East North Central (MN, IA, WI, MI) CLIMATE_REGION_8: Northeast (MD, DE, NJ, PA, NY, CT, RI, MA, VT, NH, ME) + Washington, D.C.* CLIMATE_REGION_9: Southeast (VA, NC, SC, GA, AL, GA) *Note that Washington, D.C. is not included in any of the climate regions on the website but was included with the “Northeast” region for the generation of this GRIDMASK file.

# ERA-NUTS (1980-2018)

This dataset contains a set of time-series of meteorological variables based on Copernicus Climate Change Service (C3S) ERA5 reanalysis. The data files can be downloaded from here while notebooks and other files can be found on the associated Github repository.

This data has been generated with the aim of providing hourly time-series of the meteorological variables commonly used for power system modelling and, more in general, studies on energy systems.

An example of the analysis that can be performed with ERA-NUTS is shown in this video.

Important: this dataset is still a work-in-progress, we will add more analysis and variables in the near-future. If you spot an error or something strange in the data please tell us sending an email or opening an Issue in the associated Github repository.

## Data
The time-series have hourly/daily/monthly frequency and are aggregated following the NUTS 2016 classification. NUTS (Nomenclature of Territorial Units for Statistics) is a European Union standard for referencing the subdivisions of countries (member states, candidate countries and EFTA countries).

This dataset contains NUTS0/1/2 time-series for the following variables obtained from the ERA5 reanalysis data (in brackets the name of the variable on the Copernicus Data Store and its unit measure):

- t2m: 2-meter temperature (`2m_temperature`, Celsius degrees)
- ssrd: Surface solar radiation (`surface_solar_radiation_downwards`, Watt per square meter)
- ssrdc: Surface solar radiation clear-sky (`surface_solar_radiation_downward_clear_sky`, Watt per square meter)
- ro: Runoff (`runoff`, millimeters)

There are also a set of derived variables:
- ws10: Wind speed at 10 meters (derived by `10m_u_component_of_wind` and `10m_v_component_of_wind`, meters per second)
- ws100: Wind speed at 100 meters (derived by `100m_u_component_of_wind` and `100m_v_component_of_wind`, meters per second)
- CS: Clear-Sky index (the ratio between the solar radiation and the solar radiation clear-sky)
- HDD/CDD: Heating/Cooling Degree days (derived by 2-meter temperature the EUROSTAT definition.

For each variable we have 350 599 hourly samples (from 01-01-1980 00:00:00 to 31-12-2019 23:00:00) for 34/115/309 regions (NUTS 0/1/2).

The data is provided in two formats:

- NetCDF version 4 (all the variables hourly and CDD/HDD daily). NOTE: the variables are stored as `int16` type using a `scale_factor` of 0.01 to minimise the size of the files.
- Comma Separated Value ("single index" format for all the variables and the time frequencies and "stacked" only for daily and monthly)

All the CSV files are stored in a zipped file for each variable.

## Methodology

The time-series have been generated using the following workflow:

1. The NetCDF files are downloaded from the Copernicus Data Store from the ERA5 hourly data on single levels from 1979 to present dataset
2. The data is read in R with the climate4r packages and aggregated using the function `/get_ts_from_shp` from panas. All the variables are aggregated at the NUTS boundaries using the average except for the runoff, which consists of the sum of all the grid points within the regional/national borders.
3. The derived variables (wind speed, CDD/HDD, clear-sky) are computed and all the CSV files are generated using R
4. The NetCDF are created using `xarray` in Python 3.7.

NOTE: air temperature, solar radiation, runoff and wind speed hourly data have been rounded with two decimal digits.

## Example notebooks

In the folder `notebooks` on the associated Github repository there are two Jupyter notebooks which shows how to deal effectively with the NetCDF data in `xarray` and how to visualise them in several ways by using matplotlib or the enlopy package.

There are currently two notebooks:

- exploring-ERA-NUTS: it shows how to open the NetCDF files (with Dask), how to manipulate and visualise them.
- ERA-NUTS-explore-with-widget: explorer interactively the datasets with [jupyter]() and ipywidgets.

The notebook `exploring-ERA-NUTS` is also available rendered as HTML.

## Additional files

In the folder `additional files`on the associated Github repository there is a map showing the spatial resolution of the ERA5 reanalysis and a CSV file specifying the number of grid points with respect to each NUTS0/1/2 region.

## License

This dataset is released under CC-BY-4.0 license.

Data Summary:Community Multiscale Air Quality (CMAQ) Model Version 5.1 output data from a 01/01/2013 - 12/31/2013 CONUS simulation. Note:The datasets are on a Google Drive. The metadata associated with this DOI contain the link to the Google Drive folder and instructions for downloading the data. File Location and Download Instructions: The 2013 model output are available in two forms. The hourly datasets are a set of monthly files with surface-layer hourly concentrations for a model domain that encompasses the contiguous U.S. with a horizontal grid resolution of 12km x 12km. The daily average dataset is a single file with a year of daily average data for the same domain. Link to hourly data Link to daily average data Download instructions File Format:The 2013 model output are stored as netcdf formatted files using I/O API data structures (https://www.cmascenter.org/ioapi/). Information on the model projection and grid structure is contained in the header information of the netcdf file. The output files can be opened and manipulated using I/O API utilities (e.g. M3XTRACT, M3WNDW) or other software programs that can read and write netcdf formatted files (e.g. Fortran, R, Python). Model Variables Variable names in hourly data files: Variable Name, Units, Variable Description CO, ppb, carbon monoxide NO, ppb, nitric oxide NO2, ppb, nitrogen dioxide O3, ppb, ozone SO2, ppb, sulfur dioxide SO2_UGM3, micrograms/m^3, sulfur dioxide AECIJ, micrograms/m^3, aerosol elemental carbon (sum of i-mode and j-mode)* AOCIJ, micrograms/m^3, aerosol organic carbon (sum of i-mode and j-mode)* ANO3IJ, micrograms/m^3, aerosol nitrate (sum of i-mode and j-mode)* TNO3, micrograms/m^3, total nitrate= NO3 (ANO3IJ)+ nitric acid (HNO3) ANH4IJ,micrograms/m^3, aerosol ammonium (sum of i-mode and j-mode)* ASO4IJ,micrograms/m^3, aerosol sulfate (sum of i-mode and j-mode)* PMIJ**, micrograms/m^3, total fine particulate matter (sum of i-mode and j-mode)* PM10**, micrograms/m^3, total particulate matter (sum of i-mode, j-mode, k-mode)* Variable names in daily data files: Note: All daily averages are computed using Local Standard Time (LST) Variable Name, Units, Variable Description CO_AVG, ppb, 24-hr average carbon monoxide NO_AVG, ppb, 24-hr average nitric oxide NO2_AVG, ppb, 24-hr average nitrogen dioxide O3_AVG, ppb, 24-hr average ozone O3_MDA8, ppb, Maximum daily 8-hr average ozone + SO2_AVG, ppb, 24-hr average sulfur dioxide SO2_UGM3_AVG, micrograms/m^3, 24-hr average sulfur dioxide AECIJ_AVG, micrograms/m^3, 24-hr average aerosol elemental carbon (sum of i-mode and j-mode)* AOCIJ_AVG, micrograms/m^3, 24-hr average aerosol organic carbon (sum of i-mode and j-mode)* ANO3IJ_AVG, micrograms/m^3, 24-hr average aerosol nitrate (sum of i-mode and j-mode)* TNO3_AVG, micrograms/m^3, 24-hr average total nitrate= NO3 (ANO3IJ)+ nitric acid (HNO3) ANH4IJ_AVG,micrograms/m^3, 24-hr average aerosol ammonium (sum of i-mode and j-mode)* ASO4IJ_AVG,micrograms/m^3, 24-hr average aerosol sulfate (sum of i-mode and j-mode)* PMIJ_AVG**, micrograms/m^3, 24-hr average total fine particulate matter (sum of i-mode and j-mode)* PM10_AVG**, micrograms/m^3, 24-hr average total particulate matter (sum of i-mode, j-mode, k-mode)* +The calculation of the MDA8 O3 variable is based on the current ozone NAAQS and is derived from the highest of the 17 consecutive 8-hr averages beginning with the 8-hr period from 7:00am to 3:00pm LST and ending with the 8-hr period from 11pm to 7am the following day. *CMAQ represents PM using three interacting lognormal distributions, or modes. Two modes, Aitken (i-mode) and accumulation (j-mode) are generally less than 2.5 microns in diameter while the coarse mode (k-mode) contains significant amounts of mass above 2.5 microns. **Note that modeled size distributions can also be used to output PM species that represent the aerosol mass that falls below a specific diameter, e.g. 2.5 um or 10um. The output variables that are based on the sharp cut-off method are typically very similar to the aggregate PMIJ (i+j mode) and PM10 (i+j+k mode) variables included in these files. Further information on particle size-composition distributions in CMAQv5.0 can be found in Nolte et al. (2015), https://doi.org/10.5194/gmd-8-2877-2015. Simulation Settings and Inputs: CMAQ Model Model version: 5.1Bi-directional NH3 air-surface exchange: Massad formulationChemical mechanism: CB05e51Aerosol module: aero6 Domain: Continental U.S. (CONUS) using a 12 km grid size and a Lambert Conformal projection assuming a spherical earth with radius 6370.0 km. Vertical Resolution: 35 layers from the surface to the top of the free troposphere with layer 1 nominally 19 m tall. Boundary Condition Inputs Hourly values from 2013 simulation of GEOS-Chem v9-01-02 with GEOS-5 meteorology inputs. Emissions Inputs Anthropogenic emissions: Emissions inventory label 2013ej. 2011 modeling platform version 6.2 with 2013 updates for fires, mobile sources, and mobile source (link...

Abstract

Summary

• Global estimates of root-zone soil moisture from 01-1980 to 12-2024 at ~25 km spatial sampling based on the COMBINED product of ESA CCI SM v9.2.
• Method: Exponential filter model, calibrated with in situ measurements for 3 depth layers: 0-10, 10-40, 40-100 cm with uncertainty estimates. Additionally, one layer representing the average condition from 0-1 m depth is provided. See Pasik et al. (2023) for more details.
• Good agreement with independent reanalysis data (R ~0.6 [-] and ubRMSD <0.04 m³/m³), decreasing performance for deeper layers due to weaker coupling with surface SM.

Programmatic (bulk) download

You can use command-line tools such as wget or curl to download (and extract) data for multiple years. The following command will download and extract the complete data set to the local directory ~/Download on Linux or macOS systems.

#!/bin/bash

# Set download directory
DOWNLOAD_DIR=~/Downloads

base_url="https://researchdata.tuwien.at/records/tqrwj-t7r58/files"

# Loop through years 1980 to 2024 and download & extract data
for year in {1980..2024}; do
  echo "Downloading $year.zip..."
  wget -q -P "$DOWNLOAD_DIR" "$base_url/$year.zip"
  unzip -o "$DOWNLOAD_DIR/$year.zip" -d $DOWNLOAD_DIR
  rm "$DOWNLOAD_DIR/$year.zip"
done

Data details

Version Changelog

Changes in v9.2:

• The COMBINED product of v9.2 is used as input.
• The period was extended to 12-2024.

Software to open netCDF files

These data can be read by any software that supports Climate and Forecast (CF) conform metadata standards for netCDF files, such as:

• https://github.com/pydata/xarray" target="_blank" rel="noopener">Xarray (Python)
• https://unidata.github.io/netcdf4-python/" target="_blank" rel="noopener">netCDF4 (Python)
• https://github.com/TUW-GEO/esa_cci_sm" target="_blank" rel="noopener">esa_cci_sm (Python)
• Similar tools exist for other programming languages (Matlab, R, etc.)
• Software packages and GIS tools can open netCDF files, e.g. CDO, NCO, QGIS, ArcGIS
• You can also use the GUI software Panoply to view the contents of each file

Related Records

This record and all related records are part of the ESA CCI Soil Moisture science data records community.

This data repository contains the accompanying data for the study by Stradiotti et al. (2025). Developed as part of the ESA Climate Change Initiative (CCI) Soil Moisture project. Project website: https://climate.esa.int/en/projects/soil-moisture/

Journal Article (Open Access)

This dataset was created as part of the following study, which contains a description of the algorithm and validation results.

Stradiotti, P., Gruber, A., Preimesberger, W., & Dorigo, W. (2025). Accounting for seasonal retrieval errors in the merging of multi-sensor satellite soil moisture products. Science of Remote Sensing, 12, 100242. https://doi.org/10.1016/j.srs.2025.100242

Summary

This repository contains the final, merged soil moisture and uncertainty values from Stradiotti et al. (2025), derived using a novel uncertainty quantification and merging scheme. In the accompanying study, we present a method to quantify the seasonal component of satellite soil moisture observations, based on Triple Collocation Analysis. Data from three independent satellite missions are used (from ASCAT, AMSR2, and SMAP). We observe consistent intra-annual variations in measurement uncertainties across all products (primarily caused by dynamics on the land surface such as seasonal vegetation changes), which affect the quality of the received signals. We then use these estimates to merge data from the three missions into a single consistent record, following the approach described by Dorigo et al. (2017). The new (seasonal) uncertainty estimates are propagated through the merging scheme, to enhance the uncertainty characterization of the final merged product provided here.

Evaluation against in situ data suggests that the estimated uncertainties of the new product are more representative of their true seasonal behaviour, compared to the previously used static approach. Based on these findings, we conclude that using a seasonal TCA approach can provide a more realistic characterization of dataset uncertainty, in particular its temporal variation. However, improvements in the merged soil moisture values are constrained, primarily due to correlated uncertainties among the sensors.

Technical details

The dataset provides global daily gridded soil moisture estimates for the 2012-2023 period at 0.25° (~25 km) resolution. Daily images are grouped by year (YYYY), each subdirectory containing one netCDF image file for a specific day (DD), month (MM) in a 2-dimensional (longitude, latitude) grid system (CRS: WGS84). All file names follow the naming convention:

L3S-SSMS-MERGED-SOILMOISTURE-YYYYMMDD000000-fv0.1.nc

Data Variables

Each netCDF file contains 3 coordinate variables (WGS84 longitude, latitude and time stamp), as well as the following data variables:

• sm: (float) The Soil Moisture variable contains the daily average volumetric soil moisture content (m3/m3) in the soil surface layer (~0-5 cm) over a whole grid cell (0.25 degree). Based on (merged) observations from ASCAT, AMSR2 and SMAP using the new merging scheme described in our study.
• sm_uncertainty: (float) The Soil Moisture Uncertainty variable contains the uncertainty estimates (random error) for the ‘sm’ field. Based on the uncertainty estimation and propagation scheme described in our study.
• dnflag: (int) Indicator for satellite orbit(s) used in the retrieval (day/nighttime). 1=day, 2=night, 3=both
• flag: (int) Indicator for data quality / missing data indicator. For more details, see netcdf attributes.
• freqbandID: (int) Indicator for frequency band(s) used in the retrieval. For more details, see netcdf attributes.
• mode: (int) Indicator for satellite orbit(s) used in the retrieval (ascending, descending)
• sensor: (int) Indicator for satellite sensor(s) used in the retrieval. For more details, see netcdf attributes.
• t0: (float) Representative time stamp, based on overpass times of all merged satellites.

Software to open netCDF files

After extracting the .nc files from the downloaded zip archived, they can read by any software that supports Climate and Forecast (CF) standard conform netCDF files, such as:

• Xarray (python)
• netCDF4 (python)
• esa_cci_sm (python)
• Similar tools exists for other programming languages (Matlab, R, etc.)
• GIS and netCDF tools such as CDO, NCO, QGIS, ArCGIS.
• You can also use the GUI software Panoply to view the contents of each file

Funding

This dataset was produced with funding from the European Space Agency (ESA) Climate Change Initiative (CCI) Plus Soil Moisture Project (CCN 3 to ESRIN Contract No: 4000126684/19/I-NB "ESA CCI+ Phase 1 New R&D on CCI ECVS Soil Moisture"). Project website: https://climate.esa.int/en/projects/soil-moisture/

QLKNN11D training set This dataset contains a large-scale run of ~1 billion flux calculations of the quasilinear gyrokinetic transport model QuaLiKiz. QuaLiKiz is applied in numerous tokamak integrated modelling suites, and is openly available at https://gitlab.com/qualikiz-group/QuaLiKiz/. This dataset was generated with the 'QLKNN11D-hyper' tag of QuaLiKiz, equivalent to 2.8.1 apart from the negative magnetic shear filter being disabled. See https://gitlab.com/qualikiz-group/QuaLiKiz/-/tags/QLKNN11D-hyper for the in-repository tag. The dataset is appropriate for the training of learned surrogates of QuaLiKiz, e.g. with neural networks. See https://doi.org/10.1063/1.5134126 for a Physics of Plasmas publication illustrating the development of a learned surrogate (QLKNN10D-hyper) of an older version of QuaLiKiz (2.4.0) with a 300 million point 10D dataset. The paper is also available on arXiv https://arxiv.org/abs/1911.05617 and the older dataset on Zenodo https://doi.org/10.5281/zenodo.3497066. For an application example, see Van Mulders et al 2021 https://doi.org/10.1088/1741-4326/ac0d12, where QLKNN10D-hyper was applied for ITER hybrid scenario optimization. For any learned surrogates developed for QLKNN11D, the effective addition of the alphaMHD input dimension through rescaling the input magnetic shear (s) by s = s - alpha_MHD/2, as carried out in Van Mulders et al., is recommended. Related repositories: General QuaLiKiz documentation https://qualikiz.com QuaLiKiz/QLKNN input/output variables naming scheme https://qualikiz.com/QuaLiKiz/Input-and-output-variables Training, plotting, filtering, and auxiliary tools https://gitlab.com/Karel-van-de-Plassche/QLKNN-develop QuaLiKiz related tools https://gitlab.com/qualikiz-group/QuaLiKiz-pythontools FORTRAN QLKNN implementation with wrapper for Python and MATLAB https://gitlab.com/qualikiz-group/QLKNN-fortran Weights and biases of 'hyperrectangle style' QLKNN https://gitlab.com/qualikiz-group/qlknn-hype Data exploration The data is provided in 43 netCDF files. We advise opening single datasets using xarray or multiple datasets out-of-core using dask. For reference, we give the load times and sizes of a single variable that just depends on the scan size dimx below. This was tested single-core on a Intel Xeon 8160 CPU at 2.1 GHz and 192 GB of DDR4 RAM. Note that during loading, more memory is needed than the final number. Timing of dataset loading Amount of datasets Final in-RAM memory (GiB) Loading time single var (M:SS) 1 10.3 0:09 5 43.9 1:00 10 63.2 2:01 16 98.0 3:25 17 Out Of Memory x:xx Full dataset The full dataset of QuaLiKiz in-and-output data is available on request. Note that this is 2.2 TiB of netCDF files!

This is a gridded dataset of monthly industrial water withdrawal (IWW) for China, namely, the China industrial water withdrawal dataset (CIWW). The dataset begins in January 1965 and is ongoing (currently up to December 2020) with a temporal resolution of a month and a spatial resolution of 0.1°/0.25°. The CIWW dataset, together with its auxiliary data, will be useful for water resource management and hydrological models.Version history:V1.1_20240403Update the seasonal variability.Compared to version 1.0, we estimated the seasonality of the subsector (Electricity and Heating Power Production and Supply,) based on spatial classification and then recreated the CIWW data with the updated seasonal variability. More details are described in Hou et al. (2023). The seasonal variation in the updated version is less different from the previous one.V1.0_20230209Using notes：Updated notes about opening the data with ArcGIS and other software (Jan 13, 2025)When opening the CIWW dataset (NetCDF format) in ArcGIS, the following issues may appear, as reported by users:1) The file cannot be successfully opened in ArcGIS.2) The time dimension value could not be properly displayed (e.g., time fixed to January 1, 1965).a) For ArcGIS users, it is recommended to utilize the Multidimension Tools in the toolbox and select the Make NetCDFRaster Layer tool. During the import process:Choose iww_layer as the variable.Select time as the third dimension in addition to longitude and latitude.After importing, open the Properties of the layer and navigate to the Symbology tab. You will see 672 different bands, representing the monthly data from January 1965 to December 2020.If the dataset is directly dragged into ArcGIS, the variable cell_area will be opened by default. This variable represents the area of each grid cell at a resolution of 0.25°/0.1° within the longitude and latitude range of China. The industrial water withdrawal is provided in units of mm/month. If needed, you can convert this to m³/month by multiplying the values by the corresponding grid cell area. For detailed variable descriptions, refer to the readme.txt file.b) The CIWW dataset can be opened using QGIS. Users can select the relevant dimensions and drag the dataset directly into QGIS. The time dimension includes 672 bands, with each band representing the number of days since January 1, 1965.c) The NetCDF format CIWW dataset can be easily opened by any programing language with NetCDF capabilities, for example, the xarry package in Python, Matlab, R, and others).Authors: Chengcheng Hou (cch@mail.bnu.edu.cn), Yan Li (yanli@bnu.edu.cn).Reference: Hou, C., Li, Y., Sang, S., Zhao, X., Liu, Y., Liu, Y., and Zhao, F.: High-resolution mapping of monthly industrial water withdrawal in China from 1965 to 2020, Earth Syst. Sci. Data Discuss. [preprint], https://doi.org/10.5194/essd-2023-66, in review, 2023.