The emerging global trend of satellite operators producing analysis ready data combined with open source tools for managing and exploiting this data are leading to more and more countries using Earth observation data to drive progress against key national and international development agendas. This paper provides examples from Australia, Mexico, Switzerland and Tanzania on how the Open Data Cube technology has been combined with analysis ready data to provide new insights and support better policy making across issues as diverse as water resource management through to urbanization and environmental-economic accounting.

The emerging global trend of satellite operators producing analysis ready data combined with open source tools for managing and exploiting this data are leading to more and more countries using …Show full descriptionThe emerging global trend of satellite operators producing analysis ready data combined with open source tools for managing and exploiting this data are leading to more and more countries using Earth observation data to drive progress against key national and international development agendas. This paper provides examples from Australia, Mexico, Switzerland and Tanzania on how the Open Data Cube technology has been combined with analysis ready data to provide new insights and support better policy making across issues as diverse as water resource management through to urbanization and environmental-economic accounting.

AgsSAT Multiannual (2017-2021) Sentinel-2 Built-Up Indices Composites

The 2,815 images available for the state of Aguascalientes, Mexico for the years 2017 to 2021 were processed using the Open Data Cube (ODC) platform [Lewis et al. (2017), Gavin et al. (2018), https://www.opendatacube.org/]. These images correspond to multiple coverages of the region of interest. The images were then used to generate cloud-free annual composites by applying geometric median (geomedian) algorithm, as defined in [Roberts et al. (2017)].

Geomedian algorithm produces a pixel-level summary for every pixel, in this case this means that each summary corresponds to a 10m x 10m region in the territory and its observations throughout a calendar year.

All these summary pixels form a 12-band (coastal aerosol, blue, green, red, vegetation red edge 5, vegetation red edge 6, vegetation red edge 7, near-infrared, narrow nir, water vapor, swir1 and swir2) composite of the state of Aguascalientes.

Another product called GeoMad was generated, which calculates the robust dispersion statistic called MAD, as defined in [Roberts, D., Dunn, B., & Mueller, N. (2018)]. In the resulting image composite, each of the three-pixel bands represents the variation over three distances: Spectral Distance (smad), Euclidean Distance (emad) and the Bray-Curtis Distance (bcmad).

More bands were generated to represent different environmental conditions during the study years (2017-2021), these conditions can be captured by analyzing various combinations of bands, these combinations are also called spectral indices, which allow detecting vegetation, presence of water, urbanization, etc., Finally, 28 indices divided into 4 categories were calculated:

Vegetation Indices

(Atmospherically Resistant Vegetation Index, Kaufman 1972)

(Enhanced Vegetation Index, Huete 2002):

(Modified Soil Adjusted Vegetation Index, Qi Et Al. 1994)

(Normalized Difference Chlorophyll Index, Mishra & Mishra, 2012)

(Normalised Difference Moisture Index, Gao 1996)

(Normalized Difference Vegetation Index, Rouse 1973)

(Optimized Soil Adjusted Vegetation Index, Rondeaux. 1996)

(Simple Ratio Vegetation Index Jordan, C.F.1 969)

(Soil Adjusted Vegetation Index, Huete 1988)

(Visible Atmospherically Resistant Index, Gittleson 2002)

Built-up Indexes

(Band Ration For Built-up Area, Waqar 2012)

(Built-up Area Extraction Index, Bouzekri 2015)

(Built-up Index, He Et Al. 2010)

(Index-based Built-up Index, Xu 2008)

(New Built-up Index, Jieli Et Al. 2010)

(Normalized Difference Built-up Index, Zha 2003)

(Normalized Built-up Area Index, Waqar 2012)

(Urban Index, Kawamura 1996)

Water Indices

(Modified Normalized Difference Water Index, Xu 1996)

(Normalized Difference Water Index, Mcfeeters 1996)

(Water Index, Fisher 2016)

Other Indices

(Bare Soil Index, Rikimaru Et Al. 2002)

(Bare Soil Index, Wanhui 2004)

(Burn Area Index, Martin 1998)

(Clay Minerals Ratio, Drury 1987)

(Ferrous Minerals Ratio, Segal 1982)

(Iron Oxide Ratio, Segal 1982)

(Normalized Burn Ratio, Lopez Garcia 1991)

(Normalised Difference Snow Index, Hall 1995).

Benchmark set at 77.1% O.A at: https://doi.org/10.1117/1.JRS.14.048503

The dataset consists of 60,000 images, corresponding to Landsat patches of 33x33 pixels with 102 bands. Randomly selected from Mexico (country). Each patch is labeled with one of 12 Land Use and Vegetation classes according to the classification described at https://doi.org/10.3390/rs6053923.

The zip file contains 12 folders numbered 1-12 and each contains 5,000 .npy python files (can be loaded with the NumPy library).

The labeled classes correspond to the following identifier.

1, Temperate Coniferous forest 2, Temperate Decidius Forest 3, Temperate Mixed Forest 4, Tropical Evergreen Forest 5, Tropical Deciduous Forest 6, Scrubland 7, Wetland Vegetation 8, Agriculture 9, Grassland 10, Water body 11, Barren Land 12, Urban Area

To build that dataset, we take the information of the National Continuum of Land Use and Vegetation series number 5 generated by the National Institute of Statistics and Geography from Mexico (INEGI) from The National Commission for the Knowledge and Use of Biodiversity (CONABIO) web page (http://geoportal.conabio.gob.mx/metadatos/doc/html/usv250s5ugw.html).

The file used for this dataset construction is the shape format file with geographic coordinates located in http://www.conabio.gob.mx/informacion/gis/maps/geo/usv250s5ugw.zip. Later, a transformation to Albers equal-area conic projection was done with the followings parameters:

Fake east: 2500000.0 Fake North: 0.0 Origin longitude: -102.0º Origin latitude: 12.0º First standard parallel: 17.5º Second standard parallel: 29.5º Linear unit: Meter (1.0) Reference ellipsoid: GRS80

Once the data was projected, using the classes identified in the National Continuum of Land Use and Vegetation, correspondence was applied to the classes identified in https://doi.org/10.3390/rs6053923, these classes being: Agriculture, Barren land, Grassland, Scrubland, Temperate coniferous forest, Temperate deciduous forest, Temperate mixed forest, Tropical deciduous forest, Tropical evergreen forest, Urban area, Waterbody and Wetland vegetation.

Once the information layer was generated with the 12 classes indicated above, the reference layer was rasterized. Thus, a national grid of 1,975,940 regions of 1 x 1 kilometers was generated and the percentage of pixels of the dominant class in each corresponding 1 km region was associated.

A total of cells with 70% or more pixels from one dominant class corresponds to 1,640,827 which represents a total of 83% of the Mexican territory. That means, only 17% of cells have less than 70% of their pixels from one dominant class. Then, 5000 regions were randomly selected from each land cover class at the national level. For this random selection only were selected the regions in which cells have 70% or more of their pixels from one dominant class. The above, for looking to have consistent and reliable data for the automatic classification task. This random selection generates a total of 60,000 regions selected.

Image patches were extracted from the selected regions in the sample.

The image used is the result of the application of multiple time series analysis algorithms on a cube of image data with mainly Tier 1 (T1) quality and a few Tier 2 (T2) as described in https: // www. usgs.gov/land-resources/nli/landsat/landsat-collection-1. An Open Data Cube (ODC, https://www.opendatacube.org/) was constructed from 3,515 Landsat 5 and 7 images corresponding to the year 2011, which is the same reference year of the National Continuum of Land Use and Vegetation Series 5.

From the analysis of the ODC images, the Geomedian (https://doi.org/10.1109/TGRS.2017.2723896) was calculated, which generated a national cloud-free mosaic from 2011, pixels at 30 meters resolution and 6 spectral bands (blue, green, red, nir, swir 1, swir 2). Finally, 15 spectral indices were calculated for each pixel in the image. This resulted in 15 national mosaics from the analysis of the time series of each pixel available for the year 2011 using all the combinations of normalized difference indices, which were possible with the 6 bands that were incorporated into the data cube, with which resulted in 102 information channels. Since Landsat images have a resolution of 30 meters, we have images of 33 pixels x 33 pixels for each region of 1 km x 1 km.

The 102 channels in the patches correspond to:

Geomedian Bands (6): blue, green, red, nir, swir 1, swir 2 Geomedian Based Indexes (15): evi, bu, sr, arvi, ui, ndbi, ibi, ndvi, ndwi, mndwi, nbi, brba, nbai, baei, bi Geomedian Based Tasseled cap transformation (6): brightness, greenness, wetness, fourth, fifth, sixth

2011 Landsat Time Analysis Series by Pixel

(red-swir 1)/(red+swir 1); (5): min, mean, max, std, median (red-nir)/( red+nir); (5): min, mean, max, std, median (swir 1-swir 2)/( swir 1+swir 2); (5): min, mean, max, std, median (nir-swir 2)/(nir+swir 2); (5): min, mean, max, std, median (nir-swir 1)/( nir+swir 1); (5): min, mean, max, std, median (red-swir 2)/( red+swir 2); (5): min, mean, max, std, median (green-swir 2)/(green+swir 2); (5): min, mean, max, std, median (green-swir 1)/(green+swir 1); (5): min, mean, max, std, median (green-red)/(green+red); (5): min, mean, max, std, median (green-nir)/(green+nir); (5): min, mean, max, std, median (blue-swir 2)/(blue+swir 2); (5): min, mean, max, std, median (blue-swir 1)/(blue+swir 1); (5): min, mean, max, std, median (blue-red)/(blue+red); (5): min, mean, max, std, median (blue-nir)/(blue+nir); (5): min, mean, max, std, median (blue-green)/( blue+green); (5): min, mean, max, std, median

This dataset shows how the Eurostat data cube in the orginal publicatin is modelled in QB4OLAP.

This data is based on statistical data about asylum applications to the European Union, provided by Eurostat on

http://ec.europa.eu/eurostat/web/products-datasets/-/migr_asyappctzm

Further data has been integrated from: https://github.com/lorenae/qb4olap/tree/master/examples

Digital Earth Australia manages a cloud based service that makes use of open source software and open standards to deliver satellite imagery to its clients.

In conjunction with Frontier SI and Commonwealth Scientific and Industrial Research Organisation, Geoscience Australia’s Digital Earth Australia project has developed a cloud architecture that utilizes the Open Data Cube (ODC) to deliver Earth Observation (EO) data through Open Geospatial Consortium (OGC) API standards, interactive Jupyter notebooks and direct file access.​

This infrastructure enables EO data to be used to make decisions by industry and government partners, and reduces the time required to deliver new EO data products. ​

To store the data, DEA utilises Amazon Web Services (AWS) Object store: Simple Storage Service (S3) to hold an archive of Cloud Optimised GeoTIFFs (COGs). ​

This data is indexed by Open Data Cube (ODC) an open source python library. DEA deploy processing, visualisation and analysis applications that make use of the indexed data. This method reduces the duplication of code and effort and creates an extensible framework for delivering data.

This joint NASA/USGS program provides the longest continuous space-based record of Earth’s land in existence. Every day, Landsat satellites provide essential information to help land managers and policy makers make wise decisions about our resources and our environment. Data is provided for Landsats 1, 2, 3, 4, 5, 7, 8, and 9 (excludes Landsat 6).As of June 28, 2023 (announcement), the previous single SNS topic arn:aws:sns:us-west-2:673253540267:public-c2-notify was replaced with three new SNS topics for different types of scenes.

The High Resolution Digital Elevation Model Mosaic provides a unique and continuous representation of the high resolution elevation data available across the country. The High Resolution Digital Elevation Model (HRDEM) product used is derived from airborne LiDAR data (mainly in the south) and satellite images in the north. The mosaic is available for both the Digital Terrain Model (DTM) and the Digital Surface Model (DSM) from web mapping services. It is part of the CanElevation Series created to support the National Elevation Data Strategy implemented by NRCan. This strategy aims to increase Canada's coverage of high-resolution elevation data and increase the accessibility of the products. Unlike the HRDEM product in the same series, which is distributed by acquisition project without integration between projects, the mosaic is created to provide a single, continuous representation of strategy data. The most recent datasets for a given territory are used to generate the mosaic. This mosaic is disseminated through the Data Cube Platform, implemented by NRCan using geospatial big data management technologies. These technologies enable the rapid and efficient visualization of high-resolution geospatial data and allow for the rapid generation of dynamically derived products. The mosaic is available from Web Map Services (WMS), Web Coverage Services (WCS) and SpatioTemporal Asset Catalog (STAC) collections. Accessible data includes the Digital Terrain Model (DTM), the Digital Surface Model (DSM) and derived products such as shaded relief and slope. The mosaic is referenced to the Canadian Height Reference System 2013 (CGVD2013) which is the reference standard for orthometric heights across Canada. Source data for HRDEM datasets used to create the mosaic is acquired through multiple projects with different partners. Collaboration is a key factor to the success of the National Elevation Strategy. Refer to the “Supporting Document” section to access the list of the different partners including links to their respective data.