Training Classifiers, Supervised Classification and Error Assessment

• How to add raster and vector data from the catalog in Google Earth Engine;

• Train a classifier;

• Perform the error assessment;

• Download the results.

Clipping

• How to clip a raster image to the extent of a vector polygon in order to speed up processing times as well to display only the imagery you want.

This example is about how to use Google Earth Engine API on Jupyter Notebooks. We show the example of how to get Landsat Net Primary Production (NPP) CONUS DataSet from Google Earth Engine Data Catalog.

Charts, Histograms, and Time Series

• Create a histogram graph from band values of an image collection

• Create a time series graph from band values of an image collection

After 2022-01-25, Sentinel-2 scenes with PROCESSING_BASELINE '04.00' or above have their DN (value) range shifted by 1000. The HARMONIZED collection shifts data in newer scenes to be in the same range as in older scenes. Sentinel-2 is a wide-swath, high-resolution, multi-spectral imaging mission supporting Copernicus Land Monitoring studies, including the …

Pixel Selection • Select pixels from rasters with conditional statements and boolean operators;

• Select from multiple bands and images to create a single selection;

• Create new image and transfer selected pixels to new image.

Data Management

• Create and edit fusion tables

• Upload imagery, vector, and tabular data using Fusion Tables and KMLs

• Share data with other Google Earth Engine (GEE) users as well as download imagery after manipulation in GEE.

Region of Interest (ROI) is comprised of the Belgium, the Netherlands and Luxembourg

We use the communes adminitrative division which is standardized across Europe by EUROSTAT at: https://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/administrative-units-statistical-units This is roughly equivalent to the notion municipalities in most countries.

From the link above, communes definition are taken from COMM_RG_01M_2016_4326.shp and country borders are taken from NUTS_RG_01M_2021_3035.shp.

images: Sentinel2 RGB from 2020-01-01 to 2020-31-12 filtered out pixels with clouds acoording to QA60 band following the example given in GEE dataset info page at: see https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2_SR_HARMONIZED

  see also https://github.com/rramosp/geetiles/blob/main/geetiles/defs/sentinel2rgbmedian2020.py

labels: Global Human Settlement Layers, Population Grid 2015

  labels range from 0 to 31, with the following meaning:
    label value   original value in GEE dataset
    0        0
    1        1-10
    2        11-20
    3        21-30
    ...
    31       >=291 


  see https://developers.google.com/earth-engine/datasets/catalog/JRC_GHSL_P2016_POP_GPW_GLOBE_V1


  see also https://github.com/rramosp/geetiles/blob/main/geetiles/defs/humanpop2015.py

_aschips.geojson the image chips geometries along with label proportions for easy visualization with QGIS, GeoPandas, etc.

_communes.geojson the communes geometries with their label prortions for easy visualization with QGIS, GeoPandas, etc.

splits.csv contains two splits of image chips in train, test, val - with geographical bands at 45° angles in nw-se direction - the same as above reorganized to that all chips within the same commune fall within the same split.

data/ a pickle file for each image chip containing a dict with - the 100x100 RGB sentinel 2 chip image - the 100x100 chip level lavels - the label proportions of the chip - the aggregated label proportions of the commune the chip belongs to

Top of Atmosphere (TOA) reflectance data in bands from the USGS Landsat 5 and Landsat 8 satellites were accessed via Google Earth Engine. CANUE staff used Google Earth Engine functions to create cloud free annual composites, and mask water features, then export the resulting band data. NDVI indices were calculated as (band 4 - Band 3)/(Band 4 Band 3) for Landsat 5 data, and as (band 5 - band 4)/(band 5 Band 4) for Landsat 8 data. These composites are created from all the scenes in each annual period beginning from the first day of the year and continuing to the last day of the year. No data were available for 2012, due to decommissioning of Landsat 5 in 2011 prior to the start of Landsat 8 in 2013. No cross-calibration between the sensors was performed, please be aware there may be small bias differences between NDVI values calculated using Landsat 5 and Landsat 8. Final NDVI metrics were linked to all 6-digit DMTI Spatial single link postal code locations in Canada, and for surrounding areas within 100m, 250m, 500m, and 1km.

Visualization

• How to use the knowledge of how to visualize images that you learned in previous tutorials and embed the visualization parameters inside of the GEE script so that the imagery will appear with the same visualization every time it is run.

Sentinel2GlobalLULC is a deep learning-ready dataset of RGB images from the Sentinel-2 satellites designed for global land use and land cover (LULC) mapping. Sentinel2GlobalLULC v1.1 contains 195,572 images in GeoTiff and JPEG format corresponding to 29 broad LULC classes. Each image has 224 x 224 pixels at 10 m spatial resolution and was produced by assigning the 25th percentile of all available observations in the Sentinel-2 collection between June 2015 and October 2020 in order to remove atmospheric effects (i.e., clouds, aerosols, shadows, snow, etc.). A spatial purity value was assigned to each image based on the consensus across 15 different global LULC products available in Google Earth Engine (GEE).

Our dataset is structured into 3 main zip-compressed folders, an Excel file with a dictionary for class names and descriptive statistics per LULC class, and a python script to convert RGB GeoTiff images into JPEG format. The first folder contains 29 zip-compressed subfolders where each one corresponds to a specific LULC class with hundreds to thousands of GeoTiff Sentinel-2 RGB images. The second folder contains 29 zip-compressed subfolders with a JPEG formatted version of the same images provided in the first main folder. The third folder includes 29 zip-compressed CSV files with as many rows as images and with 11 columns containing the following metadata (this same metadata is provided in the image filenames):

• Land Cover Class ID: is the identification number of each LULC class
• Land Cover Class Short Name: is the short name of each LULC class
• Pixel purity Value: is the spatial purity of each pixel for its corresponding LULC class calculated as the spatial consensus across up to 15 land-cover products
• Image ID: is the identification number of each image within its corresponding LULC class
• GHM Value: is the spatial average of the Global Human Modification index (gHM) for each image
• Latitude: is the latitude of the center point of each image
• Longitude: is the longitude of the center point of each image
• Country Code: is the Alpha-2 country code of each image as described in the ISO 3166 international standard. To understand the country codes, we recommend the user to visit the following website where they present the Alpha-2 code for each country as described in the ISO 3166 international standard:https: //www.iban.com/country-codes
• Administrative Department Level1: is the administrative level 1 name to which each image belongs
• Administrative Department Level2: is the administrative level 2 name to which each image belongs
• Locality: is the name of the locality to which each image belongs

For seven LULC classes, we could not export from GEE all images that fulfilled a spatial purity of 100% since there were millions of them. In this case, we exported a stratified random sample of 14,000 images and provided an additional CSV file with the images actually contained in our dataset. That is, for these seven LULC classes, we provide these 2 CSV files:

• A CSV file that contains all exported images for this class
• A CSV file that contains all images available for this class at spatial purity of 100%, both the ones exported and the ones not exported, in case the user wants to export them. These CSV filenames end with "including_non_downloaded_images".

© Sentinel2GlobalLULC Dataset by Yassir Benhammou, Domingo Alcaraz-Segura, Emilio Guirado, Rohaifa Khaldi, Boujemâa Achchab, Francisco Herrera & Siham Tabik is marked with Attribution 4.0 International (CC-BY 4.0)

An accurate global impervious surface map at a resolution of 30-m for 2015 by combining Landsat-8 OLI optical images, Sentinel-1 SAR images and VIIRS NTL images based on the Google Earth Engine (GEE) platform.

The Shuttle Radar Topography Mission (SRTM) digital elevation dataset was originally produced to provide consistent, high-quality elevation data at near global scope. This version of the SRTM digital elevation data has been processed to fill data voids, and to facilitate its ease of use.

This project aims to use remote sensing data from the Landsata database from Google Earth Engine to evaluate the spatial extent changes in the Bear Lake located between the US states of Utah and Idaho. This work is part of a term project submitted to Dr Alfonso Torres-Rua as a requirment to pass the Remote Sensing of Land Surfaces class (CEE6003). More information about the course is provided below. This project uses the geemap Python package (https://github.com/giswqs/geemap) for dealing with the google earth engine datasets. The content of this notebook can be used to:

learn how to retrive the Landsat 8 remote sensed data. The same functions and methodology can also be used to get the data of other Landsat satallites and other satallites such as Sentinel-2, Sentinel-3 and many others. However, slight changes might be required when dealing with other satallites then Landsat. Learn how to create time lapse images that visulaize changes in some parameters over time. Learn how to use supervised classification to track the changes in the spatial extent of water bodies such as Bear Lake that is located between the US states of Utah and Idaho. Learn how to use different functions and tools that are part of the geemap Python package. More information about the geemap Pyhton package can be found at https://github.com/giswqs/geemap and https://github.com/diviningwater/RS_of_Land_Surfaces_laboratory Course information:

Name: Remote Sensing of Land Surfaces class (CEE6003) Instructor: Alfonso Torres-Rua (alfonso.torres@usu.edu) School: Utah State University Semester: Spring semester 2023

Fast flood extent monitoring with SAR change detection using Google Earth Engine This dataset develops a tool for near real-time flood monitoring through a novel combining of multi-temporal and multi-source remote sensing data. We use a SAR change detection and thresholding method, and apply sensitivity analytics and thresholding calibration, using SAR-based and optical-based indices in a format that is streamlined, reproducible, and geographically agile. We leverage the massive repository of satellite imagery and planetary-scale geospatial analysis tools of GEE to devise a flood inundation extent model that is both scalable and replicable. The flood extents from the 2021 Hurricane Ida and the 2017 Hurricane Harvey were selected to test the approach. The methodology provides a fast, automatable, and geographically reliable tool for assisting decision-makers and emergency planners using near real-time multi-temporal satellite SAR data sets. GEE code was developed by Ebrahim Hamidi and reviewed by Brad G. Peter; Figures were created by Brad G. Peter. This tool accompanies a publication Hamidi et al., 2023: E. Hamidi, B. G. Peter, D. F. Muñoz, H. Moftakhari and H. Moradkhani, "Fast Flood Extent Monitoring with SAR Change Detection Using Google Earth Engine," in IEEE Transactions on Geoscience and Remote Sensing, doi: 10.1109/TGRS.2023.3240097. GEE input datasets: Methodology flowchart: Sensitivity Analysis: GEE code (muti-source and multi-temporal flood monitoring): https://code.earthengine.google.com/7f4942ab0c73503e88287ad7e9187150 The threshold sensitivity analysis is automated in the below GEE code: https://code.earthengine.google.com/a3fbfe338c69232a75cbcd0eb6bc0c8e The above scripts can be run independently. The threshold automation code identifies the optimal threshold values for use in the flood monitoring procedure. GEE code for Hurricane Harvey, east of Houston Java script: // Study Area Boundaries var bounds = /* color: #d63000 */ee.Geometry.Polygon( [[[-94.5214452285728, 30.165244882083663], [-94.5214452285728, 29.56024879238989], [-93.36650748443218, 29.56024879238989], [-93.36650748443218, 30.165244882083663]]], null, false); // [before_start,before_end,after_start,after_end,k_ndfi,k_ri,k_diff,mndwi_threshold] var params = ['2017-06-01','2017-06-15','2017-08-01','2017-09-10',1.0,0.25,0.8,0.4] // SAR Input Data var before_start = params[0] var before_end = params[1] var after_start = params[2] var after_end = params[3] var polarization = "VH" var pass_direction = "ASCENDING" // k Coeficient Values for NDFI, RI and DII SAR Indices (Flooded Pixel Thresholding; Equation 4) var k_ndfi = params[4] var k_ri = params[5] var k_diff = params[6] // MNDWI flooded pixels Threshold Criteria var mndwi_threshold = params[7] // Datasets ----------------------------------- var dem = ee.Image("USGS/3DEP/10m").select('elevation') var slope = ee.Terrain.slope(dem) var swater = ee.Image('JRC/GSW1_0/GlobalSurfaceWater').select('seasonality') var collection = ee.ImageCollection('COPERNICUS/S1_GRD') .filter(ee.Filter.eq('instrumentMode', 'IW')) .filter(ee.Filter.listContains('transmitterReceiverPolarisation', polarization)) .filter(ee.Filter.eq('orbitProperties_pass', pass_direction)) .filter(ee.Filter.eq('resolution_meters', 10)) .filterBounds(bounds) .select(polarization) var before = collection.filterDate(before_start, before_end) var after = collection.filterDate(after_start, after_end) print("before", before) print("after", after) // Generating Reference and Flood Multi-temporal SAR Data ------------------------ // Mean Before and Min After ------------------------ var mean_before = before.mean().clip(bounds) var min_after = after.min().clip(bounds) var max_after = after.max().clip(bounds) var mean_after = after.mean().clip(bounds) Map.addLayer(mean_before, {min: -29.264204107025904, max: -8.938093778644141, palette: []}, "mean_before",0) Map.addLayer(min_after, {min: -29.29334290990966, max: -11.928313976797138, palette: []}, "min_after",1) // Flood identification ------------------------ // NDFI ------------------------ var ndfi = mean_before.abs().subtract(min_after.abs()) .divide(mean_before.abs().add(min_after.abs())) var ndfi_filtered = ndfi.focal_mean({radius: 50, kernelType: 'circle', units: 'meters'}) // NDFI Normalization ----------------------- var ndfi_min = ndfi_filtered.reduceRegion({ reducer: ee.Reducer.min(), geometry: bounds, scale: 10, maxPixels: 1e13 }) var ndfi_max = ndfi_filtered.reduceRegion({ reducer: ee.Reducer.max(), geometry: bounds, scale: 10, maxPixels: 1e13 }) var ndfi_rang = ee.Number(ndfi_max.get('VH')).subtract(ee.Number(ndfi_min.get('VH'))) var ndfi_subtctMin = ndfi_filtered.subtract(ee.Number(ndfi_min.get('VH'))) var ndfi_norm = ndfi_subtctMin.divide(ndfi_rang) Map.addLayer(ndfi_norm, {min: 0.3862747346632676, max: 0.7632898395906615}, "ndfi_norm",0) var histogram = ui.Chart.image.histogram({ image: ndfi_norm, region: bounds, scale: 10, maxPixels: 1e13 })...

The Lithology dataset provides classes of the general types of parent material of soil on the surface. It is not derived from any DEM. The Conservation Science Partners (CSP) Ecologically Relevant Geomorphology (ERGo) Datasets, Landforms and Physiography contain detailed, multi-scale data on landforms and physiographic (aka land facet) patterns. Although there are many potential uses of these data, the original purpose for these data was to develop an ecologically relevant classification and map of landforms and physiographic classes that are suitable for climate adaptation planning. Because there is large uncertainty associated with future climate conditions and even more uncertainty around ecological responses, providing information about what is unlikely to change offers a strong foundation for managers to build robust climate adaptation plans. The quantification of these features of the landscape is sensitive to the resolution, so we provide the highest resolution possible given the extent and characteristics of a given index.

CANUE staff developed annual estimates of maximum mean warm-season land surface temperature (LST) recorded by LandSat 8 at 30m resolution. To reduce the effect of missing data/cloud cover/shadows, the highest mean warm-season value reported over three years was retained - for example, the data for 2021 represent the maximum of the mean land surface temperature at a pixel location between April 1st and September 30th in 2019, 2020 and 2021. Land surface temperature was calculated in Google Earth Engine, using a public algorithm (see supplementary documentation). In general, annual mean LST may not reflect ambient air temperatures experienced by individuals at any given time, but does identify areas that are hotter during the day and therefore more likely to radiate excess heat at night - both factors that contribute to heat islands within urban areas.

Classified maps exported from Google Earth Engine for 2019 and 2020. Change in coal mining areas between 2019 and 2020

Geospatial raster data and vector data created in the frame of the study "Mapping Arctic Lake Ice Backscatter Anomalies using Sentinel-1 Time Series on Google Earth Engine" submitted to the journal "Remote Sensing" and Python code to reproduce the results.

In addition to the full repository (Supplement_to_RS_Arctic_Lake_Ice_Backscatter_Anomalies.zip), two reduced alternatives of this repository are available due to large file size of the full repository:

Supplement_to_RS_Arctic_Lake_Ice_Backscatter_Anomalies_without_IW_result_data.zip contains the same data and Python scripts as the full repository, but results based on IW data and tiled EW delta sigma0 images directly exported from Google Earth Engine have been removed. The merged data (from tiled EW delta sigma0 images) and all other results deduced thereof are included.

Supplement_to_RS_Arctic_Lake_Ice_Backscatter_Anomalies_scripts_and_reference_data_only.zip contains only the Python scripts and reference data. The directory structure was retained for better reproducibility.

Please see the associated README-files for details.