NDWI as netcdf file

No description was included in this Dataset collected from the OSF

Overview

Zip file contains two netCDF files with a subset of data from the "No Change 2" (NC2) experiment conducted by Savi et al., 2020 and published in Earth Surface Dynamics (https://doi.org/10.5194/esurf-8-303-2020) with the original data available via the Sediment Experimentalists Network Project Space SEAD Internal Repository (https://doi.org/10.26009/s0ZOQ0S6). Topographic scan data were re-formatted into the netCDF file "T_NC2_scans.nc", and overhead imagery was extracted from the video of the experiment approximately once every minute of experimental time and RGB band data is provided in the formatted netCDF file "T_NC2_images.nc". These data were formatted into netCDF files for easy loading into the "deltametrics" analysis toolbox.

Additional Details

Re-packaging the scan data from the .tif files was straightforward. From the metadata spreadsheet, we know the times at which the scans were taken (and can eliminate the redundant scan). From the paper itself we know the resolution of the topographic scans is 1 mm in the horizontal and vertical. We also know the input discharges, both water and sediment, through both the main channel and tributary, from the paper. We provide these values as metadata in the netCDF files. The scans form the 'eta' field representing the topography in the file. The packaged up netCDF file is called 'T_NC2_scans.nc'.

Overhead imagery from the T_NC2_Complete21fps.wmv video file was extracted using the following command:

ffmpeg -i T_NC2_Complete21fps.wmv -r 21 T_NC2_frames/%04d.png

This command utilizes the ffmpeg tool to extract the frames at a rate of 21 frames per second (-r 21) as the file name implies that is the rate at which the overhead photos were combined into a video. The NC designation indicates that this experiment was performed with no change in the input conditions in either the main or tributary channels.

The experiment ran for a total of 480 minutes. A total of 1466 images were obtained from the ffmpeg extraction. This translates to an image approximately every 20 seconds of real time (480 minutes / 1466 frames * 60 seconds/minute = 19.6453 seconds / frame). We sample every 3rd frame, which gives us images roughly once a minute (489 frames in all), to create the subset of data re-packaged as a netCDF file for deltametrics. Dimensions for the pixels were approximated based on our knowledge of the topographic scan resolution. Assuming the extents of the scans and overhead images are the same (although they are not), we calculate the number of millimeters per pixel in the x and y directions for the overhead images. We assume the pixels are more likely to be square than rectangular, so we average these values and assign this as the distance per pixel in both the x and y dimensions for these data.

References

Savi, Sara, et al. "Interactions between main channels and tributary alluvial fans: channel adjustments and sediment-signal propagation." Earth Surface Dynamics 8.2 (2020): 303-322.

Physical experiments on interactions between main-channels and tributary alluvial fans
S. Savi, Tofelde, A. Wickert, A. Bufe, T. Schildgen, and M. Strecker
https://doi.org/10.26009/s0ZOQ0S6

Data Summary: US states grid mask file and NOAA climate regions grid mask file, both compatible with the 12US1 modeling grid domain. Note:The datasets are on a Google Drive. The metadata associated with this DOI contain the link to the Google Drive folder and instructions for downloading the data. These files can be used with CMAQ-ISAMv5.3 to track state- or region-specific emissions. See Chapter 11 and Appendix B.4 in the CMAQ User's Guide for further information on how to use the ISAM control file with GRIDMASK files. The files can also be used for state or region-specific scaling of emissions using the CMAQv5.3 DESID module. See the DESID Tutorial and Appendix B.4 in the CMAQ User's Guide for further information on how to use the Emission Control File to scale emissions in predetermined geographical areas. File Location and Download Instructions: Link to GRIDMASK files Link to README text file with information on how these files were created File Format: The grid mask are stored as netcdf formatted files using I/O API data structures (https://www.cmascenter.org/ioapi/). Information on the model projection and grid structure is contained in the header information of the netcdf file. The output files can be opened and manipulated using I/O API utilities (e.g. M3XTRACT, M3WNDW) or other software programs that can read and write netcdf formatted files (e.g. Fortran, R, Python). File descriptions These GRIDMASK files can be used with the 12US1 modeling grid domain (grid origin x = -2556000 m, y = -1728000 m; N columns = 459, N rows = 299). GRIDMASK_STATES_12US1.nc - This file containes 49 variables for the 48 states in the conterminous U.S. plus DC. Each state variable (e.g., AL, AZ, AR, etc.) is a 2D array (299 x 459) providing the fractional area of each grid cell that falls within that state. GRIDMASK_CLIMATE_REGIONS_12US1.nc - This file containes 9 variables for 9 NOAA climate regions based on the Karl and Koss (1984) definition of climate regions. Each climate region variable (e.g., CLIMATE_REGION_1, CLIMATE_REGION_2, etc.) is a 2D array (299 x 459) providing the fractional area of each grid cell that falls within that climate region. NOAA Climate regions: CLIMATE_REGION_1: Northwest (OR, WA, ID) CLIMATE_REGION_2: West (CA, NV) CLIMATE_REGION_3: West North Central (MT, WY, ND, SD, NE) CLIMATE_REGION_4: Southwest (UT, AZ, NM, CO) CLIMATE_REGION_5: South (KS, OK, TX, LA, AR, MS) CLIMATE_REGION_6: Central (MO, IL, IN, KY, TN, OH, WV) CLIMATE_REGION_7: East North Central (MN, IA, WI, MI) CLIMATE_REGION_8: Northeast (MD, DE, NJ, PA, NY, CT, RI, MA, VT, NH, ME) + Washington, D.C.* CLIMATE_REGION_9: Southeast (VA, NC, SC, GA, AL, GA) *Note that Washington, D.C. is not included in any of the climate regions on the website but was included with the “Northeast” region for the generation of this GRIDMASK file.

This dataset contains information on the Surface Soil Moisture (SM) content derived from satellite observations in the microwave domain.

Dataset paper (public preprint)

A description of this dataset, including the methodology and validation results, is available at:

Preimesberger, W., Stradiotti, P., and Dorigo, W.: ESA CCI Soil Moisture GAPFILLED: An independent global gap-free satellite climate data record with uncertainty estimates, Earth Syst. Sci. Data Discuss. [preprint], https://doi.org/10.5194/essd-2024-610, in review, 2025.

Abstract

ESA CCI Soil Moisture is a multi-satellite climate data record that consists of harmonized, daily observations coming from 19 satellites (as of v09.1) operating in the microwave domain. The wealth of satellite information, particularly over the last decade, facilitates the creation of a data record with the highest possible data consistency and coverage.
However, data gaps are still found in the record. This is particularly notable in earlier periods when a limited number of satellites were in operation, but can also arise from various retrieval issues, such as frozen soils, dense vegetation, and radio frequency interference (RFI). These data gaps present a challenge for many users, as they have the potential to obscure relevant events within a study area or are incompatible with (machine learning) software that often relies on gap-free inputs.
Since the requirement of a gap-free ESA CCI SM product was identified, various studies have demonstrated the suitability of different statistical methods to achieve this goal. A fundamental feature of such gap-filling method is to rely only on the original observational record, without need for ancillary variable or model-based information. Due to the intrinsic challenge, there was until present no global, long-term univariate gap-filled product available. In this version of the record, data gaps due to missing satellite overpasses and invalid measurements are filled using the Discrete Cosine Transform (DCT) Penalized Least Squares (PLS) algorithm (Garcia, 2010). A linear interpolation is applied over periods of (potentially) frozen soils with little to no variability in (frozen) soil moisture content. Uncertainty estimates are based on models calibrated in experiments to fill satellite-like gaps introduced to GLDAS Noah reanalysis soil moisture (Rodell et al., 2004), and consider the gap size and local vegetation conditions as parameters that affect the gapfilling performance.

Summary

• Gap-filled global estimates of volumetric surface soil moisture from 1991-2023 at 0.25° sampling
• Fields of application (partial): climate variability and change, land-atmosphere interactions, global biogeochemical cycles and ecology, hydrological and land surface modelling, drought applications, and meteorology
• Method: Modified version of DCT-PLS (Garcia, 2010) interpolation/smoothing algorithm, linear interpolation over periods of frozen soils. Uncertainty estimates are provided for all data points.
• More information: See Preimesberger et al. (2025) and https://doi.org/10.5281/zenodo.8320869" target="_blank" rel="noopener">ESA CCI SM Algorithm Theoretical Baseline Document [Chapter 7.2.9] (Dorigo et al., 2023)

Programmatic Download

You can use command line tools such as wget or curl to download (and extract) data for multiple years. The following command will download and extract the complete data set to the local directory ~/Download on Linux or macOS systems.

#!/bin/bash

# Set download directory
DOWNLOAD_DIR=~/Downloads

base_url="https://researchdata.tuwien.at/records/3fcxr-cde10/files"

# Loop through years 1991 to 2023 and download & extract data
for year in {1991..2023}; do
  echo "Downloading $year.zip..."
  wget -q -P "$DOWNLOAD_DIR" "$base_url/$year.zip"
  unzip -o "$DOWNLOAD_DIR/$year.zip" -d $DOWNLOAD_DIR
  rm "$DOWNLOAD_DIR/$year.zip"
done

Data details

The dataset provides global daily estimates for the 1991-2023 period at 0.25° (~25 km) horizontal grid resolution. Daily images are grouped by year (YYYY), each subdirectory containing one netCDF image file for a specific day (DD), month (MM) in a 2-dimensional (longitude, latitude) grid system (CRS: WGS84). The file name has the following convention:

ESACCI-SOILMOISTURE-L3S-SSMV-COMBINED_GAPFILLED-YYYYMMDD000000-fv09.1r1.nc

Data Variables

Each netCDF file contains 3 coordinate variables (WGS84 longitude, latitude and time stamp), as well as the following data variables:

• sm: (float) The Soil Moisture variable reflects estimates of daily average volumetric soil moisture content (m3/m3) in the soil surface layer (~0-5 cm) over a whole grid cell (0.25 degree).
• sm_uncertainty: (float) The Soil Moisture Uncertainty variable reflects the uncertainty (random error) of the original satellite observations and of the predictions used to fill observation data gaps.
• sm_anomaly: Soil moisture anomalies (reference period 1991-2020) derived from the gap-filled values (`sm`)
• sm_smoothed: Contains DCT-PLS predictions used to fill data gaps in the original soil moisture field. These values are also provided for cases where an observation was initially available (compare `gapmask`). In this case, they provided a smoothed version of the original data.
• gapmask: (0 | 1) Indicates grid cells where a satellite observation is available (1), and where the interpolated (smoothed) values are used instead (0) in the 'sm' field.
• frozenmask: (0 | 1) Indicates grid cells where ERA5 soil temperature is <0 °C. In this case, a linear interpolation over time is applied.

Additional information for each variable is given in the netCDF attributes.

Version Changelog

Changes in v9.1r1 (previous version was v09.1):

• This version uses a novel uncertainty estimation scheme as described in Preimesberger et al. (2025).

Software to open netCDF files

These data can be read by any software that supports Climate and Forecast (CF) conform metadata standards for netCDF files, such as:

• https://github.com/pydata/xarray" target="_blank" rel="noopener">Xarray (python)
• https://unidata.github.io/netcdf4-python/" target="_blank" rel="noopener">netCDF4 (python)
• https://github.com/TUW-GEO/esa_cci_sm">esa_cci_sm (python)
• Similar tools exists for other programming languages (Matlab, R, etc.)
• Software packages and GIS tools can open netCDF files, e.g. CDO, NCO, QGIS, ArCGIS
• You can also use the GUI software Panoply to view the contents of each file

References

• Preimesberger, W., Stradiotti, P., and Dorigo, W.: ESA CCI Soil Moisture GAPFILLED: An independent global gap-free satellite climate data record with uncertainty estimates, Earth Syst. Sci. Data Discuss. [preprint], https://doi.org/10.5194/essd-2024-610, in review, 2025.
• Dorigo, W., Preimesberger, W., Stradiotti, P., Kidd, R., van der Schalie, R., van der Vliet, M., Rodriguez-Fernandez, N., Madelon, R., & Baghdadi, N. (2023). ESA Climate Change Initiative Plus - Soil Moisture Algorithm Theoretical Baseline Document (ATBD) Supporting Product Version 08.1 (version 1.1). Zenodo. https://doi.org/10.5281/zenodo.8320869
• Garcia, D., 2010. Robust smoothing of gridded data in one and higher dimensions with missing values. Computational Statistics & Data Analysis, 54(4), pp.1167-1178. Available at: https://doi.org/10.1016/j.csda.2009.09.020
• Rodell, M., Houser, P. R., Jambor, U., Gottschalck, J., Mitchell, K., Meng, C.-J., Arsenault, K., Cosgrove, B., Radakovich, J., Bosilovich, M., Entin, J. K., Walker, J. P., Lohmann, D., and Toll, D.: The Global Land Data Assimilation System, Bulletin of the American Meteorological Society, 85, 381 – 394, https://doi.org/10.1175/BAMS-85-3-381, 2004.

Related Records

The following records are all part of the Soil Moisture Climate Data Records from satellites community

The Global Forecast System (GFS) is a weather forecast model produced by the National Centers for Environmental Prediction (NCEP). Dozens of atmospheric and land-soil variables are available through this dataset, from temperatures, winds, and precipitation to soil moisture and atmospheric ozone concentration. The GFS data files stored here can be immediately used for OAR/ARL’s NOAA-EPA Atmosphere-Chemistry Coupler Cloud (NACC-Cloud) tool, and are in a Network Common Data Form (netCDF), which is a very common format used across the scientific community. These particular GFS files contain a comprehensive number of global atmosphere/land variables at a relatively high spatiotemporal resolution (approximately 13x13 km horizontal, vertical resolution of 127 levels, and hourly), are not only necessary for the NACC-Cloud tool to adequately drive community air quality applications (e.g., U.S. EPA’s Community Multiscale Air Quality model; https://www.epa.gov/cmaq), but can be very useful for a myriad of other applications in the Earth system modeling communities (e.g., atmosphere, hydrosphere, pedosphere, etc.). While many other data file and record formats are indeed available for Earth system and climate research (e.g., GRIB, HDF, GeoTIFF), the netCDF files here are advantageous to the larger community because of the comprehensive, high spatiotemporal information they contain, and because they are more scalable, appendable, shareable, self-describing, and community-friendly (i.e., many tools available to the community of users). Out of the four operational GFS forecast cycles per day (at 00Z, 06Z, 12Z and 18Z) this particular netCDF dataset is updated daily (/inputs/yyyymmdd/) for the 12Z cycle and includes 24-hr output for both 2D (gfs.t12z.sfcf$0hh.nc) and 3D variables (gfs.t12z.atmf$0hh.nc).

Also available are netCDF formatted Global Land Surface Datasets (GLSDs) developed by Hung et al. (2024). The GLSDs are based on numerous satellite products, and have been gridded to match the GFS spatial resolution (~13x13 km). These GLSDs contain vegetation canopy data (e.g., land surface type, vegetation clumping index, leaf area index, vegetative canopy height, and green vegetation fraction) that are supplemental to and can be combined with the GFS meteorological netCDF data for various applications, including NOAA-ARL's canopy-app. The canopy data variables are climatological, based on satellite data from the year 2020, combined with GFS meteorology for the year 2022, and are created at a daily temporal resolution (/inputs/geo-files/gfs.canopy.t12z.2022mmdd.sfcf000.global.nc)

The twin satellites of the Gravity Recovery and Climate Experiment (GRACE), launched in March of 2002, are making detailed monthly measurements of Earth's gravity field changes. These observations can detect regional mass changes of Earth's water reservoirs over land, ice and oceans. GRACE measures gravity variations by relating it to the distance variations between the two satellites, which fly in the same orbit, separated by about 240 km at an altitude of ~450 km. The monthly land mass grids contain terrestrial water storage anomalies (in aquifers, river basins, etc.) from GRACE time-variable gravity data relative to a time-mean. The storage anomalies are given in 'equivalent water thickness' (in NetCDF format). The time coverage for the monthly grids are determined by GRACE months. For the list of GRACE month dates visit http://grace.jpl.nasa.gov/data/grace-months/ . For information please visit http://grace.jpl.nasa.gov/data/get-data/monthly-mass-grids-land/ .

HRES is a high-resolution (100m) synthetic bed elevation terrain for the whole Antarctic continent. The synthetic bed surface preserves topographic roughness characteristics of airborne and ground-based ice-penetrating radar data from the Bedmap1 compilation and the ICECAP consortium. Broad-scale features of the Antarctic landscape are incorporated from a lowpass filter of the Bedmap2 bed elevation data. The data are available in NetCDF classic format on a 100m resolution grid in a Polar Stereographic Projection (Central Meridian 0 degrees, Standard Parallel 71 degrees S) with respect to the WGS84 geoid. The 100m grid is 66661 rows by 66661 columns, where the corner of the lower left cell is located at a polar stereographic easting and northing of -3333000 m and -3333000 m, respectively. The value for missing data is -9999.

A minor update was made to the dataset on 2021-12-23 to correct an error in the data.

The issue was that while the x and y dimensions were defined in the netcdf file (i.e. how many points in each dimension) the corresponding variables defining the actual locations were not defined. Therefore, when the file was loaded, the software was defaulting to using the index count to be the coordinate location, which would look like 1m data spacing

To fix this, the coordinate variables have been defined, and also the order the data was written in to correspond to what viewing programs such as ferret expect has been swapped.

The actual post-processing on the original data (hres.nc) was

gdal_translate -of netCDF -co WRITE_BOTTOMUP=NO NETCDF:"hres.nc":bed_elevation hres_temp_1.nc

ncpdq -a -y,x hres_temp_1.nc hres_temp_2.nc ncap2 -S hres_nco.txt hres_temp_2.nc hres_revised.nc

Where the file hres_nco.txt contains the following 8 lines y=array(-3331000f,100,$y); x=array(-3331000f,100,$x); y@long_name="Northing"; y@Standard_name="Northing"; y@units="m"; x@long_name="Easting"; x@Standard_name="Easting"; x@units="m";

The second processing step (ncpdq) requires a system with a bit over 40GB of memory.

CO2_netCDF.zip includes 3 files, readme_nc.txt is the descrition file of the rest 2 files. The format of data file is .nc, and demension description of the netCDF file is .xlsx. CO2_mat.zip includes 5 files, readme_mat.txt is the descrition file of the rest 4 data files. The format of data files is .mat, which could be load using matlab.

From http://northweb.hpl.umces.edu/LTRANS.htm. CHECK FOR UPDATES. NEWER VERSION MAY BE AVAILABLE.

PDF of original LTRANS_v.2b website [2016-09-14]

LTRANS v.2b Model Description

The Larval TRANSport Lagrangian model (LTRANS v.2b) is an off-line particle-tracking model that runs with the stored predictions of a 3D hydrodynamic model, specifically the Regional Ocean Modeling System (ROMS). Although LTRANS was built to simulate oyster larvae, it can easily be adapted to simulate passive particles and other planktonic organisms. LTRANS v.2 is written in Fortran 90 and is designed to track the trajectories of particles in three dimensions. It includes a 4th order Runge-Kutta scheme for particle advection and a random displacement model for vertical turbulent particle motion. Reflective boundary conditions, larval behavior, and settlement routines are also included. A brief description of the LTRANS particle-tracking model can be found here (68 KB .pdf). For more information on LTRANS and the application of LTRANS to oyster larvae transport, see a summary web page with animations, the publications North et al. (2008, 2011), and the LTRANS v.2 User's Guide. Please cite North et al. (2011) when referring to LTRANS v.2b. The updates that were made for LTRANS v.2b are listed here.

The Lagrangian TRANSport (LTRANS v.2b) model is based upon LTRANS v.1 (formerly the Larval TRANSport Lagrangian model). Ian Mitchell made the bug fixes in LTRANS v.2b. Zachary Schlag completed signigicant updates to the code in LTRANS v.2 with input from Elizabeth North, Chris Sherwood, and Scott Peckham. LTRANS v.1 was built by Elizabeth North and Zachary Schlag of University of Maryland Center for Environmental Science Horn Point Laboratory. Funding was provided by the National Science Foundation Biological and Physical Oceanography Programs**, Maryland Department of Natural Resources, NOAA Chesapeake Bay Studies, NOAA Maryland Sea Grant College Program, and NOAA-funded UMCP Advanced Study Institute for the Environment.

A beta version of LTRANS v2b which uses predictions from the circulation model ADCIRC is available here.

LTRANS Code

LTRANS v.2b Open Source Code. We would appreciate knowing who is using LTRANS. If you would like to share this information with us, please send us your name, contact information, and a brief description of how you plan to use LTRANS to enorth@umces.edu. To refer to LTRANS in a peer-reviewed publication, please cite the publication(s) listed in the Description section above.

External Dependencies and Programs. LTRANS v.2b requires NetCDF libraries and uses the following programs to calculate random numbers (Mersenne Twister) and fit tension splines (TSPACK). Because LTRANS v.2 reads-in ROMS-generated NetCDF (.nc) files, it requires that the appropriate NetCDF libraries be installed on your computer (see files and links below). Also, please note that although the Mersenne Twister and TSPACK programs are included in the LTRANS v.2b in the Random_module.f90 and Tension_module.f90, respectively, they do not share the same license file as LTRANS v.2b. Please review and respect their permissions (links and instructions provided below).

LTRANS v.2b Example Input Files. These files can be used to test LTRANS v.2b. They include examples of particle location and habitat polygon input files (.csv) and ROMS grid and history files (.nc) that are needed to run LTRANS v.2b. Many thanks to Wen Long for sharing the ROMS .nc files. The LTRANS v.2b code above is configured to run with these input files. Note: please download the tar (LTRANSv2.tgz) history files (clippped_macroms_his_*.nc) files between the hours of 5 pm and 6 am Eastern Standard Time because of their large size.

The TIGER/Line shapefiles and related database files (.dbf) are an extract of selected geographic and cartographic information from the U.S. Census Bureau's Master Address File / Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) Database (MTDB). The MTDB represents a seamless national file with no overlaps or gaps between parts, however, each TIGER/Line shapefile is designed to stand alone as an independent data set, or they can be combined to cover the entire nation. The Feature Names Relationship File (FEATNAMES.dbf) contains a record for each feature name and any attributes associated with it. Each feature name can be linked to the corresponding edges that make up that feature in the All Lines Shapefile (EDGES.shp), where applicable to the corresponding address range or ranges in the Address Ranges Relationship File (ADDR.dbf), or to both files. Although this file includes feature names for all linear features, not just road features, the primary purpose of this relationship file is to identify all street names associated with each address range. An edge can have several feature names; an address range located on an edge can be associated with one or any combination of the available feature names (an address range can be linked to multiple feature names). The address range is identified by the address range identifier (ARID) attribute, which can be used to link to the Address Ranges Relationship File (ADDR.dbf). The linear feature is identified by the linear feature identifier (LINEARID) attribute, which can be used to relate the address range back to the name attributes of the feature in the Feature Names Relationship File or to the feature record in the Primary Roads, Primary and Secondary Roads, or All Roads Shapefiles. The edge to which a feature name applies can be determined by linking the feature name record to the All Lines Shapefile (EDGES.shp) using the permanent edge identifier (TLID) attribute. The address range identifier(s) (ARID) for a specific linear feature can be found by using the linear feature identifier (LINEARID) from the Feature Names Relationship File (FEATNAMES.dbf) through the Address Range / Feature Name Relationship File (ADDRFN.dbf).

The TIGER/Line shapefiles and related database files (.dbf) are an extract of selected geographic and cartographic information from the U.S. Census Bureau's Master Address File / Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) Database (MTDB). The MTDB represents a seamless national file with no overlaps or gaps between parts, however, each TIGER/Line shapefile is designed to stand alone as an independent data set, or they can be combined to cover the entire nation. The Feature Names Relationship File (FEATNAMES.dbf) contains a record for each feature name and any attributes associated with it. Each feature name can be linked to the corresponding edges that make up that feature in the All Lines Shapefile (EDGES.shp), where applicable to the corresponding address range or ranges in the Address Ranges Relationship File (ADDR.dbf), or to both files. Although this file includes feature names for all linear features, not just road features, the primary purpose of this relationship file is to identify all street names associated with each address range. An edge can have several feature names; an address range located on an edge can be associated with one or any combination of the available feature names (an address range can be linked to multiple feature names). The address range is identified by the address range identifier (ARID) attribute, which can be used to link to the Address Ranges Relationship File (ADDR.dbf). The linear feature is identified by the linear feature identifier (LINEARID) attribute, which can be used to relate the address range back to the name attributes of the feature in the Feature Names Relationship File or to the feature record in the Primary Roads, Primary and Secondary Roads, or All Roads Shapefiles. The edge to which a feature name applies can be determined by linking the feature name record to the All Lines Shapefile (EDGES.shp) using the permanent edge identifier (TLID) attribute. The address range identifier(s) (ARID) for a specific linear feature can be found by using the linear feature identifier (LINEARID) from the Feature Names Relationship File (FEATNAMES.dbf) through the Address Range / Feature Name Relationship File (ADDRFN.dbf).

This is the Coral Reef Watch Ocean Acidification Product Suite produced monthly in conjunction with NOAA CoastWatch. CoastWatch Utilities, including CoastWatch Data Analysis Tool (CDAT), v3.2.1 or higher (https://coastwatch.noaa.gov/cw_software.html) can be used for viewing, analyzing, and plotting the data. cdm_data_type=Grid cols=122 comment=This is the Coral Reef Watch Ocean Acidification Product Suite produced monthly in conjunction with NOAA CoastWatch. CoastWatch Utilities, including CoastWatch Data Analysis Tool (CDAT), v3.2.1 or higher (https://coastwatch.noaa.gov/cw_software.html) can be used for viewing, analyzing, and plotting the data. composite=false contact=NOAA Coral Reef Watch at coralreefwatch@noaa.gov Conventions=COARDS, CF-1.0, Unidata Dataset Discovery v1.0 data_source=NOAA OI AVHRR-AMSRE SST, NCEP Interp SLP, Forecast pCO2air Model, IASNFS SSS Easternmost_Easting=-59.875 easternmost_longitude=-60.0 geospatial_lat_max=30.125 geospatial_lat_min=14.875 geospatial_lat_resolution=0.25 geospatial_lat_units=degrees_north geospatial_lon_max=-59.875 geospatial_lon_min=-90.125 geospatial_lon_resolution=0.25 geospatial_lon_units=degrees_east history_of_appended_files=Tue Sep 26 10:29:40 2023: Appended file NETCDFIASNFS/METADATA/landmask_25km.nc had following "history" attribute: Mon Sep 29 08:40:42 2008: ncatted -a flag_values,surface_flag,c,b,0,1 landmask_25km.nc Mon Sep 29 08:39:43 2008: ncrename -v z,surface_flag landmask_25km.nc Mon Sep 29 08:39:40 2008: ncatted -a units,longitude,c,c,degrees-east landmask_25km.nc Mon Sep 29 08:39:35 2008: ncatted -a units,latitude,c,c,degress-north landmask_25km.nc Mon Sep 29 08:39:22 2008: ncatted -a long_name,latitude,m,c,latitude landmask_25km.nc Mon Sep 29 08:39:17 2008: ncatted -a long_name,longitude,m,c,longitude landmask_25km.nc Mon Sep 29 08:39:11 2008: ncatted -a flag_meanings,z,c,c,valid, non-valid (includes land and areas of no model output) landmask_25km.nc Mon Sep 29 08:39:00 2008: ncatted -a flag_values,z,m,b,0,1 landmask_25km.nc Mon Sep 29 08:38:56 2008: ncatted -a units,z,c,c,none landmask_25km.nc Mon Sep 29 08:38:41 2008: ncatted -a description,z,c,c,An array in the same dimension as the data array(s) classifies valid, non-valid (includes land and areas of no model outpt) and missing pixels that are all flaged by the same missing_value in the da array(s). landmask_25km.nc Mon Sep 29 08:38:08 2008: ncatted -a description,z,m,c,An array in the same dimension as the data array(s) classifies valid, non-valid (includes land and areas of no model outpt) and missing pixels that are all flaged by the same missing_value in the da array(s). landmask_25km.nc Mon Sep 29 08:37:28 2008: ncatted -a long_name,z,m,c,Pixel characteristics flag array landmask_25km.nc Mon Sep 29 08:35:17 2008: ncrename -v y,latitude landmask_25km.nc Mon Sep 29 08:35:13 2008: ncrename -v x,longitude landmask_25km.nc grdmath ../../landmask_25km.grd ISNAN -1 ADD 2 POW = kk.grd Tue Sep 26 10:29:40 2023: Appended file co3.nc had following "history" attribute: Tue Sep 26 10:29:40 2023: ncrename -v z,CO3 co3.nc Tue Sep 26 10:29:40 2023: ncatted -a palette_info_variable,z,c,c,Uncertain co3.nc Tue Sep 26 10:29:40 2023: ncatted -a variable_info,z,c,c,The values in this variable array and in the valid_range attribute are in the units of x10umol/kg . The data values should be multiplied by the value (=0.1) contained in thescale_factor attribute to obtain the actual values in the units of .umol/kg . The values are modeled according to Gledhill et al., 2008. Fields of TA & fCO2sw were were coupled to solve for the carbonic acid system using the CO2SYS program (Lewis & Wallace, 1998). Constants: K1,K2 from Mehrbach et al, 1973 refit by Dickson & Millero, 1987; fCO2 (versus pCO2); KSO4 from Dickson; pH = total scale co3.nc Tue Sep 26 10:29:40 2023: ncatted -a coordsys,z,c,c,geographic co3.nc Tue Sep 26 10:29:40 2023: ncatted -a units,z,c,c,umol/kg co3.nc Tue Sep 26 10:29:40 2023: ncatted -a long_name,z,m,c,NOAA Coral Reef Watch Experimental Ocean Acidification Product Suite - Carbonate Ion Concentration (CO3--) 25km co3.nc Tue Sep 26 10:29:40 2023: ncatted -a units,y,c,c,degrees_north co3.nc Tue Sep 26 10:29:40 2023: ncatted -a long_name,y,m,c,latitude co3.nc Tue Sep 26 10:29:40 2023: ncatted -a units,x,c,c,degrees_east co3.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,x,m,c,longitude co3.nc grdmath landmask_25km.grd kk.grd MUL = co3.nc Tue Sep 26 10:29:40 2023: Appended file hco.nc had following "history" attribute: Tue Sep 26 10:29:39 2023: ncrename -v z,HCO3 hco.nc Tue Sep 26 10:29:39 2023: ncatted -a palette_info_variable,z,c,c,Uncertain hco.nc Tue Sep 26 10:29:39 2023: ncatted -a variable_info,z,c,c,The values in this variable array and in the valid_range attribute are in the units of x10umol/kg . The data values should be multiplied by the value (=0.1) contained in thescale_factor attribute to obtain the actual values in the units of .umol/kg . The values are modeled according to Gledhill et al., 2008. Fields of TA & fCO2sw were were coupled to solve for the carbonic acid system using the CO2SYS program (Lewis & Wallace, 1998). Constants: K1,K2 from Mehrbach et al, 1973 refit by Dickson & Millero, 1987; fCO2 (versus pCO2); KSO4 from Dickson; pH = total scale hco.nc Tue Sep 26 10:29:39 2023: ncatted -a coordsys,z,c,c,geographic hco.nc Tue Sep 26 10:29:39 2023: ncatted -a units,z,c,c,umol/kg hco.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,z,m,c,NOAA Coral Reef Watch Experimental Ocean Acidification Product Suite - Bicarbonate Ion Concentration (HCO3-) 25km hco.nc Tue Sep 26 10:29:39 2023: ncatted -a units,y,c,c,degrees_north hco.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,y,m,c,latitude hco.nc Tue Sep 26 10:29:39 2023: ncatted -a units,x,c,c,degrees_east hco.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,x,m,c,longitude hco.nc grdmath landmask_25km.grd kk.grd MUL = hco.nc Tue Sep 26 10:29:40 2023: Appended file ara.nc had following "history" attribute: Tue Sep 26 10:29:39 2023: ncrename -v z,SSA ara.nc Tue Sep 26 10:29:39 2023: ncatted -a palette_info_variable,z,c,c,Uncertain ara.nc Tue Sep 26 10:29:39 2023: ncatted -a variable_info,z,c,c,The values in this variable array and in the valid_range attribute are in the units of x100Omega . The data values should be multiplied by the value (=0.01) contained in thescale_factor attribute to obtain the actual values in the units of .Omega . The values are modeled according to Gledhill et al., 2008. Fields of TA & fCO2sw were were coupled to solve for the carbonic acid system using the CO2SYS program (Lewis & Wallace, 1998). Constants: K1,K2 from Mehrbach et al, 1973 refit by Dickson & Millero, 1987; fCO2 (versus pCO2); KSO4 from Dickson; pH = total scale ara.nc Tue Sep 26 10:29:39 2023: ncatted -a coordsys,z,c,c,geographic ara.nc Tue Sep 26 10:29:39 2023: ncatted -a units,z,c,c,Omega ara.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,z,m,c,NOAA Coral Reef Watch Experimental Ocean Acidification Product Suite - Saturation State (argonite) 25km ara.nc Tue Sep 26 10:29:39 2023: ncatted -a units,y,c,c,degrees_north ara.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,y,m,c,latitude ara.nc Tue Sep 26 10:29:39 2023: ncatted -a units,x,c,c,degrees_east ara.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,x,m,c,longitude ara.nc grdmath landmask_25km.grd kk.grd MUL = ara.nc Tue Sep 26 10:29:40 2023: Appended file ph.nc had following "history" attribute: Tue Sep 26 10:29:39 2023: ncrename -v z,pH ph.nc Tue Sep 26 10:29:39 2023: ncatted -a palette_info_variable,z,c,c,Uncertain ph.nc Tue Sep 26 10:29:39 2023: ncatted -a variable_info,z,c,c,The values in this variable array and in the valid_range attribute are in the units of x100Total Scale. The data values should be multiplied by the value (=0.01) contained in thescale_factor attribute to obtain the actual values in the units of .Total Scale. The values are modeled according to Gledhill et al., 2008. Fields of TA & fCO2sw were were coupled to solve for the carbonic acid system using the CO2SYS program (Lewis & Wallace, 1998). Constants: K1,K2 from Mehrbach et al, 1973 refit by Dickson & Millero, 1987; fCO2 (versus pCO2); KSO4 from Dickson; pH = total scale ph.nc Tue Sep 26 10:29:39 2023: ncatted -a coordsys,z,c,c,geographic ph.nc Tue Sep 26 10:29:39 2023: ncatted -a units,z,c,c,Total Scale ph.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,z,m,c,NOAA Coral Reef Watch Experimental Ocean Acidification Product Suite - pH 25km ph.nc Tue Sep 26 10:29:39 2023: ncatted -a units,y,c,c,degrees_north ph.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,y,m,c,latitude ph.nc Tue Sep 26 10:29:39 2023: ncatted -a units,x,c,c,degrees_east ph.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,x,m,c,longitude ph.nc grdmath landmask_25km.grd kk.grd MUL = ph.nc Tue Sep 26 10:29:40 2023: Appended file dic.nc had following "history" attribute: Tue Sep 26 10:29:39 2023: ncrename -v z,TC dic.nc Tue Sep 26 10:29:39 2023: ncatted -a palette_info_variable,z,c,c,Uncertain dic.nc Tue Sep 26 10:29:39 2023: ncatted -a variable_info,z,c,c,The values in this variable array and in the valid_range attribute are in the units of x10umol/kg . The data values should be multiplied by the value (=0.1) contained in thescale_factor attribute to obtain the actual values in the units of .umol/kg . The values are modeled according to Gledhill et al., 2008. Fields of TA & fCO2sw were were coupled to solve for the carbonic acid system using the CO2SYS program (Lewis & Wallace, 1998). Constants: K1,K2 from Mehrbach et al, 1973 refit by Dickson & Millero, 1987; fCO2 (versus pCO2); KSO4 from Dickson; pH = total scale dic.nc Tue Sep 26 10:29:39 2023: ncatted -a coordsys,z,c,c,geographic dic.nc Tue Sep 26 10:29:39 2023: ncatted -a units,z,c,c,umol/kg dic.nc Tue Sep 26 10:29:39 2023: ncatted -a long_name,z,m,c,NOAA Coral Reef Watch Experimental Ocean Acidification Product Suite - Total Inorganic Carbon (TC)

Description

This repository holds global spatial estimates of the Net Ecosystem Productivity of forests (NEP), circa 2010, for a grid spacing of 0.5° by 0.5º pixel size. Three different approaches were used to create the maps.

1. Model M1 (Regional Age–NEP Relationships Per Biome): This model scales site-level NEP observations to a global gridded field using biome-specific NEP-age curves and site-level anomalies. The random forest model (RF1) is trained on forest age, GPP, temperature, and nitrogen deposition, producing NEP anomalies that reflect site-specific deviations from biome-wide trends. Gridded predictor fields of forest age, GPP, temperature (MAT), and nitrogen deposition are used to create 0.5° by 0.5° NEP grids, with uncertainties estimated using an ensemble of 180 members. The data from Model M1 can be investigated from the ForestAge_EC_NEP_M1_v1.0.nc file.

2. Model M2 (Global Age–NEP Relationship): This model uses a random forest algorithm (RF2) to upscale NEP observations but applies a global NEP-age relationship across all sites. It uses the same gridded predictor fields as M1—forest age, GPP, MAT, and nitrogen deposition—but the age–NEP relationship is determined globally. Uncertainty is calculated similarly to M1, using ensembles of model parameters and predictor fields. The data from Model M2 can be investigated from the ForestAge_EC_NEP_M2_v1.0.nc file.

3. Model M3 (Without Age Consideration): This model predicts NEP solely based on GPP, MAT, and nitrogen deposition without accounting for forest age. It follows a similar approach to RF3 models from previous work and uses the same gridded predictors and uncertainty estimation methods as M1 and M2. The data from Model M3 can be investigated from the ForestAge_EC_NEP_M3_v1.0.nc file.

The variation across each model's members can assess the uncertainty in each model, which represents uncertainty caused by input variables and the k-fold cross-validation approach.

More details about the methodologies behind the three approaches can be found in Ciais, P., Yao, Y. Besnard, S. et al. (2024) (see reference below).

Data structure

The datasets are stored in NetCDF format with a structure consistent across the different models (M1, M2, M3). Each file contains multiple variables representing components of the Net Ecosystem Production (NEP) estimates, such as the mean NEP and its quantiles. The primary variables are:

• NEP_MX_mean: The mean estimate of NEP for each model (M1, M2, M3), with units of grams of carbon per square meter per year (gC m⁻² year⁻¹).
• NEP_MX_quantiles: Estimates of NEP at different quantiles, providing uncertainty ranges. The quantiles represented in the data are: [0.25, 0.3, 0.35, 0.4, 0.45, 0.5, 0.55, 0.6, 0.65, 0.7, 0.75]
  
• Members dimension: Each model includes a members dimension, representing several NEP estimates generated using different ensemble members. These members capture uncertainty from input variables such as GPP, temperature, nitrogen deposition, and forest age. The members dimension provides users with multiple realizations of NEP estimates, reflecting the variability these factors introduce.

Coordinates include latitude and longitude with CRS information (EPSG:4326). Missing data values are represented by -9999.

Citation

When using the maps, please cite the dataset, including the version number and the following paper: Ciais, P., Yao, Y. Besnard, S. et al. (2024) The global carbon balance of forests based on flux towers and forest age data, submitted.

Version History

• 1.0 - Initial version, covering 2010

The TIGER/Line shapefiles and related database files (.dbf) are an extract of selected geographic and cartographic information from the U.S. Census Bureau's Master Address File / Topologically Integrated Geographic Encoding and Referencing (MAF/TIGER) Database (MTDB). The MTDB represents a seamless national file with no overlaps or gaps between parts, however, each TIGER/Line shapefile is designed to stand alone as an independent data set, or they can be combined to cover the entire nation. The Feature Names Relationship File (FEATNAMES.dbf) contains a record for each feature name and any attributes associated with it. Each feature name can be linked to the corresponding edges that make up that feature in the All Lines Shapefile (EDGES.shp), where applicable to the corresponding address range or ranges in the Address Ranges Relationship File (ADDR.dbf), or to both files. Although this file includes feature names for all linear features, not just road features, the primary purpose of this relationship file is to identify all street names associated with each address range. An edge can have several feature names; an address range located on an edge can be associated with one or any combination of the available feature names (an address range can be linked to multiple feature names). The address range is identified by the address range identifier (ARID) attribute, which can be used to link to the Address Ranges Relationship File (ADDR.dbf). The linear feature is identified by the linear feature identifier (LINEARID) attribute, which can be used to relate the address range back to the name attributes of the feature in the Feature Names Relationship File or to the feature record in the Primary Roads, Primary and Secondary Roads, or All Roads Shapefiles. The edge to which a feature name applies can be determined by linking the feature name record to the All Lines Shapefile (EDGES.shp) using the permanent edge identifier (TLID) attribute. The address range identifier(s) (ARID) for a specific linear feature can be found by using the linear feature identifier (LINEARID) from the Feature Names Relationship File (FEATNAMES.dbf) through the Address Range / Feature Name Relationship File (ADDRFN.dbf).