This dataset is true colour cloud-free composite satellite imagery optimised for mapping shallow marine habitats in northern Australia, based on 10-meter resolution Sentinel 2 data collected from 2015 to 2024. It contains composite imagery for 333 Sentinel 2 tiles of northern Australia and the Great Barrier Reef. This dataset offers improved visual clarity of shallow water features as compared to existing satellite imagery, allowing deeper marine features to be observed. These composites were specifically designed to address challenges such as sun glint, clouds and turbidity that typically hinder marine environment analyses. No tides were considered in the selection of the imagery and so this imagery corresponds to an 'All tide' image, approximating mean sea level.

This dataset is an updated version (Version 2), published in July 2024, which succeeds the initial draft version (Version 1, published in March 2024). The current version spans imagery from 2015–2024, an extension of the earlier timeframe that covered 2018–2022. This longer temporal range allowed the imagery to be cleaner with lower image noise allowing deeper marine features to be visible. The deprecated draft version was removed from online download to save on storage space and is now only available on request.

While the final imagery corresponds to true colour based primarily Sentinel 2 bands B2 (blue), B3 (green), and B4 (red), the near infrared (B8) band was used as part of sun glint correction and automated selection of low noise imagery.

Contrast enhancement was applied to the imagery to compress the original 12 bit per channel Sentinel 2 imagery into the final 8-bit per channel GeoTiffs. Black and white point correction was used to enhance the contrast as much as possible without too much clipping of the darkest and lightest marine features. Gamma correction of 2 (red), 2 (green) and 2.3 (blue) was applied allow a wider dynamic range to be represented in the 8-bit data, helping to ensure that little precision was lost in representing darker marine features. As a result, the image brightness is not linearly scaled. Further details of the corrections applied is available from https://github.com/eatlas/AU_NESP-MaC-3-17_AIMS_S2-comp/blob/main/src/processors/s2processor.py.


Methods:

The satellite image composites were created by combining multiple Sentinel 2 images using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile filter the "COPERNICUS/S2_HARMONIZED" image collection by
- tile ID
- maximum cloud cover 20%
- date between '2015-06-27' and '2024-05-31'
- asset_size > 100000000 (remove small fragments of tiles)
Note: A maximum cloud cover of 20% was used to improve the processing times. In most cases this filtering does not have an effect on the final composite as images with higher cloud coverage mostly result in higher noise levels and are not used in the final composite.
2. Split images by "SENSING_ORBIT_NUMBER" (see "Using SENSING_ORBIT_NUMBER for a more balanced composite" for more information).
3. For each SENSING_ORBIT_NUMBER collection filter out all noise-adding images:
3.1 Calculate image noise level for each image in the collection (see "Image noise level calculation for more information") and sort collection by noise level.
3.2 Remove all images with a very high noise index (>15).
3.3 Calculate a baseline noise level using a minimum number of images (min_images_in_collection=30). This minimum number of images is needed to ensure a smoth composite where cloud "holes" in one image are covered by other images.
3.4 Iterate over remaining images (images not used in base noise level calculation) and check if adding image to the composite adds to or reduces the noise. If it reduces the noise add it to the composite. If it increases the noise stop iterating over images.
4. Combine SENSING_ORBIT_NUMBER collections into one image collection.
5. Remove sun-glint (true colour only) and apply atmospheric correction on each image (see "Sun-glint removal and atmospheric correction" for more information).
6. Duplicate image collection to first create a composite image without cloud masking and using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
7. Apply cloud masking to all images in the original image collection (see "Cloud Masking" for more information) and create a composite by using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
8. Combine the two composite images (no cloud mask composite and cloud mask composite). This solves the problem of some coral cays and islands being misinterpreted as clouds and therefore creating holes in the composite image. These holes are "plugged" with the underlying composite without cloud masking. (Lawrey et al. 2022)
9. The final composite was exported as cloud optimized 8 bit GeoTIFF

Note: The following tiles were generated with no "maximum cloud cover" as they did not have enough images to create a composite with the standard settings: 46LGM, 46LGN, 46LHM, 50KKD, 50KPG, 53LMH, 53LMJ, 53LNH, 53LPH, 53LPJ, 54LVP, 57JVH, 59JKJ.

Compositing Process:

The dataset was created using a multi-step compositing process. A percentile-based image compositing technique was employed, with the 15th percentile chosen as the optimal value for most regions. This percentile was identified as the most effective in minimizing noise and enhancing key features such as coral reefs, islands, and other shallow water habitats. The 15th percentile was chosen as a trade off between the desire to select darker pixels that typically correspond to clearer water, and very dark values (often occurring at the 10th percentile) corresponding to cloud shadows.

The cloud masking predictor would often misinterpret very pale areas, such as cays and beaches as clouds. To overcome this limitation a dual-image compositing method was used. A primary composite was generated with cloud masks applied, and a secondary, composite with no cloud masking was layered beneath to fill in potential gaps (or “holes”) caused by the cloud masking mistakes

Image noise level calculation:

The noise level for each image in this dataset is calculated to ensure high-quality composites by minimizing the inclusion of noisy images. This process begins by creating a water mask using the Normalized Difference Water Index (NDWI) derived from the NIR and Green bands. High reflectance areas in the NIR and SWIR bands, indicative of sun-glint, are identified and masked by the water mask to focus on water areas affected by sun-glint. The proportion of high sun-glint pixels within these water areas is calculated and amplified to compute a noise index. If no water pixels are detected, a high noise index value is assigned.

In any set of satellite images, some will be taken under favourable conditions (low wind, low sun-glint, and minimal cloud cover), while others will be affected by high sun-glint or cloud. Combining multiple images into a composite reduces noise by averaging out these fluctuations.

When all images have the same noise level, increasing the number of images in the composite reduces the overall noise. However, in practice, there is a mix of high and low noise images. The optimal composite is created by including as many low-noise images as possible while excluding high-noise ones. The challenge lies in the determining the acceptable noise threshold for a given scene as some areas are more cloudy and sun glint affected than others.

To address this, we rank the available Sentinel 2 images for each scene by their noise index, from lowest to highest. The goal is to determine the ideal number of images (N) to include in the composite to minimize overall noise. For each N, we use the lowest noise images and estimate the final composite noise based on the noise index. This is repeated for all values of N up to a maximum of 200 images, and we select the N that results in the lowest noise.

This approach has some limitations. It estimates noise based on sun glint and residual clouds (after cloud masking) using NIR bands, without accounting for image turbidity. The final composite noise is not directly measured as this would be computationally expensive. It is instead estimated by dividing the average noise of the selected images by the square root of the number of images. We found this method tends to underestimate the ideal image count, so we adjusted the noise estimates, scaling them by the inverse of their ranking, to favor larger sets of images. The algorithm is not fully optimized, and further refinement is needed to improve accuracy.

Full details of the algorithm can be found in https://github.com/eatlas/AU_NESP-MaC-3-17_AIMS_S2-comp/blob/main/src/utilities/noise_predictor.py

Sun glint removal and atmospheric correction:

Sun glint was removed from the images using the infrared B8 band to estimate the reflection off the water from the sun glint. B8 penetrates water less than 0.5 m and so in water areas it only detects reflections off the surface of the water. The sun glint detected by B8 correlates very highly with the sun glint experienced by the visible channels (B2, B3 and B4) and so the sun glint in these channels can be removed by subtracting B8 from these channels.

Eric Lawrey developed this algorithm by fine tuning the value of the scaling between the B8 channel and each individual visible channel (B2, B3 and B4) so that the maximum level of sun glint would be removed. This work was based on a representative set of images, trying to determine a set of values that represent a good compromise across different water surface

This dataset contains cloud free, low tide composite satellite images for the tropical Australia region based on 10 m resolution Sentinel 2 imagery from 2018 – 2023. This image collection was created as part of the NESP MaC 3.17 project and is intended to allow mapping of the reef features in tropical Australia.

This collection contains composite imagery for 200 Sentinel 2 tiles around the tropical Australian coast. This dataset uses two styles:
1. a true colour contrast and colour enhancement style (TrueColour) using the bands B2 (blue), B3 (green), and B4 (red)
2. a near infrared false colour style (Shallow) using the bands B5 (red edge), B8 (near infrared), and B12 (short wave infrared).
These styles are useful for identifying shallow features along the coastline.

The Shallow false colour styling is optimised for viewing the first 3 m of the water column, providing an indication of water depth. This is because the different far red and near infrared bands used in this styling have limited penetration of the water column. In clear waters the maximum penetrations of each of the bands is 3-5 m for B5, 0.5 - 1 m for B8 and < 0.05 m for B12. As a result, the image changes in colour with the depth of the water with the following colours indicating the following different depths:
- White, brown, bright green, red, light blue: dry land
- Grey brown: damp intertidal sediment
- Turquoise: 0.05 - 0.5 m of water
- Blue: 0.5 - 3 m of water
- Black: Deeper than 3 m
In very turbid areas the visible limit will be slightly reduced.


Change log:

This dataset will be progressively improved and made available for download. These additions will be noted in this change log.
2024-07-24 - Add tiles for the Great Barrier Reef
2024-05-22 - Initial release for low-tide composites using 30th percentile (Git tag: "low_tide_composites_v1")


Methods:

The satellite image composites were created by combining multiple Sentinel 2 images using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile filter the "COPERNICUS/S2_HARMONIZED" image collection by
- tile ID
- maximum cloud cover 0.1%
- date between '2018-01-01' and '2023-12-31'
- asset_size > 100000000 (remove small fragments of tiles)
2. Remove high sun-glint images (see "High sun-glint image detection" for more information).
3. Split images by "SENSING_ORBIT_NUMBER" (see "Using SENSING_ORBIT_NUMBER for a more balanced composite" for more information).
4. Iterate over all images in the split collections to predict the tide elevation for each image from the image timestamp (see "Tide prediction" for more information).
5. Remove images where tide elevation is above mean sea level to make sure no high tide images are included.
6. Select the 10 images with the lowest tide elevation.
7. Combine SENSING_ORBIT_NUMBER collections into one image collection.
8. Remove sun-glint (true colour only) and apply atmospheric correction on each image (see "Sun-glint removal and atmospheric correction" for more information).
9. Duplicate image collection to first create a composite image without cloud masking and using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
10. Apply cloud masking to all images in the original image collection (see "Cloud Masking" for more information) and create a composite by using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
11. Combine the two composite images (no cloud mask composite and cloud mask composite). This solves the problem of some coral cays and islands being misinterpreted as clouds and therefore creating holes in the composite image. These holes are "plugged" with the underlying composite without cloud masking. (Lawrey et al. 2022)
12. The final composite was exported as cloud optimized 8 bit GeoTIFF

Note: The following tiles were generated with different settings as they did not have enough images to create a composite with the standard settings:
- 51KWA: no high sun-glint filter
- 54LXP: maximum cloud cover set to 1%
- 54LXP: maximum cloud cover set to 1%
- 54LYK: maximum cloud cover set to 2%
- 54LYM: maximum cloud cover set to 5%
- 54LYN: maximum cloud cover set to 1%
- 54LYQ: maximum cloud cover set to 5%
- 54LYP: maximum cloud cover set to 1%
- 54LZL: maximum cloud cover set to 1%
- 54LZM: maximum cloud cover set to 1%
- 54LZN: maximum cloud cover set to 1%
- 54LZQ: maximum cloud cover set to 5%
- 54LZP: maximum cloud cover set to 1%
- 55LBD: maximum cloud cover set to 2%
- 55LBE: maximum cloud cover set to 1%
- 55LCC: maximum cloud cover set to 5%
- 55LCD: maximum cloud cover set to 1%


High sun-glint image detection:

Images with high sun-glint can lead to lower quality composite images. To determine high sun-glint images, a mask is created for all pixels above a high reflectance threshold for the near-infrared and short-wave infrared bands. Then the proportion of this is calculated and compared against a sun-glint threshold. If the image exceeds this threshold, it is filtered out of the image collection. As we are only interested in the sun-glint on water pixels, a water mask is created using NDWI before creating the sun-glint mask.


Sun-glint removal and atmospheric correction:

Sun-glint was removed from the images using the infrared B8 band to estimate the reflection off the water from the sun-glint. B8 penetrates water less than 0.5 m and so in water areas it only detects reflections off the surface of the water. The sun-glint detected by B8 correlates very highly with the sun-glint experienced by the visible channels (B2, B3 and B4) and so the sun-glint in these channels can be removed by subtracting B8 from these channels.

Eric Lawrey developed this algorithm by fine tuning the value of the scaling between the B8 channel and each individual visible channel (B2, B3 and B4) so that the maximum level of sun-glint would be removed. This work was based on a representative set of images, trying to determine a set of values that represent a good compromise across different water surface conditions.

This algorithm is an adjustment of the algorithm already used in Lawrey et al. 2022


Tide prediction:

To determine the tide elevation in a specific satellite image, we used a tide prediction model to predict the tide elevation for the image timestamp. After investigating and comparing a number of models, it was decided to use the empirical ocean tide model EOT20 (Hart-Davis et al., 2021). The model data can be freely accessed at https://doi.org/10.17882/79489 and works with the Python library pyTMD (https://github.com/tsutterley/pyTMD). In our comparison we found this model was able to predict accurately the tide elevation across multiple points along the study coastline when compared to historic Bureau of Meteorolgy and AusTide data. To determine the tide elevation of the satellite images we manually created a point dataset where we placed a central point on the water for each Sentinel tile in the study area . We used these points as centroids in the ocean models and calculated the tide elevation from the image timestamp.


Using "SENSING_ORBIT_NUMBER" for a more balanced composite:

Some of the Sentinel 2 tiles are made up of different sections depending on the "SENSING_ORBIT_NUMBER". For example, a tile could have a small triangle on the left side and a bigger section on the right side. If we filter an image collection and use a subset to create a composite, we could end up with a high number of images for one section (e.g. the left side triangle) and only few images for the other section(s). This would result in a composite image with a balanced section and other sections with a very low input. To avoid this issue, the initial unfiltered image collection is divided into multiple image collections by using the image property "SENSING_ORBIT_NUMBER". The filtering and limiting (max number of images in collection) is then performed on each "SENSING_ORBIT_NUMBER" image collection and finally, they are combined back into one image collection to generate the final composite.


Cloud Masking:

Each image was processed to mask out clouds and their shadows before creating the composite image.
The cloud masking uses the COPERNICUS/S2_CLOUD_PROBABILITY dataset developed by SentinelHub (Google, n.d.; Zupanc, 2017). The mask includes the cloud areas, plus a mask to remove cloud shadows. The cloud shadows were estimated by projecting the cloud mask in the direction opposite the angle to the sun. The shadow distance was estimated in two parts.

A low cloud mask was created based on the assumption that small clouds have a small shadow distance. These were detected using a 35% cloud probability threshold. These were projected over 400 m, followed by a 150 m buffer to expand the final mask.

A high cloud mask was created to cover longer shadows created by taller, larger clouds. These clouds were detected based on an 80% cloud probability threshold, followed by an erosion and dilation of 300 m to remove small clouds. These were then projected over a 1.5 km distance followed by a 300 m buffer.

The parameters for the cloud masking (probability threshold, projection distance and buffer radius) were determined through trial and error on a small number of scenes. As such there are probably significant potential improvements that could be made to this algorithm.

Erosion, dilation and buffer operations were performed at a lower image resolution than the native satellite image resolution

This dataset contains both large (A0) printable maps of the Torres Strait broken into six overlapping regions, based on a clear sky, clear water composite Sentinel 2 composite imagery and the imagery used to create these maps. These maps show satellite imagery of the region, overlaid with reef and island boundaries and names. Not all features are named, just the more prominent features. This also includes a vector map of Ashmore Reef and Boot Reef in Coral Sea as these were used in the same discussions that these maps were developed for. The map of Ashmore Reef includes the atoll platform, reef boundaries and depth polygons for 5 m and 10 m.

This dataset contains all working files used in the development of these maps. This includes all a copy of all the source datasets and all derived satellite image tiles and QGIS files used to create the maps. This includes cloud free Sentinel 2 composite imagery of the Torres Strait region with alpha blended edges to allow the creation of a smooth high resolution basemap of the region.

The base imagery is similar to the older base imagery dataset: Torres Strait clear sky, clear water Landsat 5 satellite composite (NERP TE 13.1 eAtlas, AIMS, source: NASA).

Most of the imagery in the composite imagery from 2017 - 2021.


Method:
The Sentinel 2 basemap was produced by processing imagery from the World_AIMS_Marine-satellite-imagery dataset (01-data/World_AIMS_Marine-satellite-imagery in the data download) for the Torres Strait region. The TrueColour imagery for the scenes covering the mapped area were downloaded. Both the reference 1 imagery (R1) and reference 2 imagery (R2) was copied for processing. R1 imagery contains the lowest noise, most cloud free imagery, while R2 contains the next best set of imagery. Both R1 and R2 are typically composite images from multiple dates.

The R2 images were selectively blended using manually created masks with the R1 images. This was done to get the best combination of both images and typically resulted in a reduction in some of the cloud artefacts in the R1 images. The mask creation and previewing of the blending was performed in Photoshop. The created masks were saved in 01-data/R2-R1-masks. To help with the blending of neighbouring images a feathered alpha channel was added to the imagery. The processing of the merging (using the masks) and the creation of the feathered borders on the images was performed using a Python script (src/local/03-merge-R2-R1-images.py) using the Pillow library and GDAL. The neighbouring image blending mask was created by applying a blurring of the original hard image mask. This allowed neighbouring image tiles to merge together.

The imagery and reference datasets (reef boundaries, EEZ) were loaded into QGIS for the creation of the printable maps.

To optimise the matching of the resulting map slight brightness adjustments were applied to each scene tile to match its neighbours. This was done in the setup of each image in QGIS. This adjustment was imperfect as each tile was made from a different combinations of days (to remove clouds) resulting in each scene having a different tonal gradients across the scene then its neighbours. Additionally Sentinel 2 has slight stripes (at 13 degrees off the vertical) due to the swath of each sensor having a slight sensitivity difference. This effect was uncorrected in this imagery.


Single merged composite GeoTiff:
The image tiles with alpha blended edges work well in QGIS, but not in ArcGIS Pro. To allow this imagery to be used across tools that don't support the alpha blending we merged and flattened the tiles into a single large GeoTiff with no alpha channel. This was done by rendering the map created in QGIS into a single large image. This was done in multiple steps to make the process manageable.

The rendered map was cut into twenty 1 x 1 degree georeferenced PNG images using the Atlas feature of QGIS. This process baked in the alpha blending across neighbouring Sentinel 2 scenes. The PNG images were then merged back into a large GeoTiff image using GDAL (via QGIS), removing the alpha channel. The brightness of the image was adjusted so that the darkest pixels in the image were 1, saving the value 0 for nodata masking and the boundary was clipped, using a polygon boundary, to trim off the outer feathering. The image was then optimised for performance by using internal tiling and adding overviews. A full breakdown of these steps is provided in the README.md in the 'Browse and download all data files' link.

The merged final image is available in export\TS_AIMS_Torres Strait-Sentinel-2_Composite.tif.


Source datasets:
Complete Great Barrier Reef (GBR) Island and Reef Feature boundaries including Torres Strait Version 1b (NESP TWQ 3.13, AIMS, TSRA, GBRMPA), https://eatlas.org.au/data/uuid/d2396b2c-68d4-4f4b-aab0-52f7bc4a81f5

Geoscience Australia (2014b), Seas and Submerged Lands Act 1973 - Australian Maritime Boundaries 2014a - Geodatabase [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, https://dx.doi.org/10.4225/25/5539DFE87D895

Basemap/AU_GA_AMB_2014a/Exclusive_Economic_Zone_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014a). The Geodatabase was loaded in ArcMap. The Exclusive_Economic_Zone_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

Geoscience Australia (2014a), Treaties - Australian Maritime Boundaries (AMB) 2014a [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, http://dx.doi.org/10.4225/25/5539E01878302
Basemap/AU_GA_Treaties-AMB_2014a/Papua_New_Guinea_TSPZ_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014b). The Geodatabase was loaded in ArcMap. The Papua_New_Guinea_TSPZ_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

AIMS Coral Sea Features (2022) - DRAFT
This is a draft version of this dataset. The region for Ashmore and Boot reef was checked. The attributes in these datasets haven't been cleaned up. Note these files should not be considered finalised and are only suitable for maps around Ashmore Reef. Please source an updated version of this dataset for any other purpose.
CS_AIMS_Coral-Sea-Features/CS_Names/Names.shp
CS_AIMS_Coral-Sea-Features/CS_Platform_adj/CS_Platform.shp
CS_AIMS_Coral-Sea-Features/CS_Reef_Boundaries_adj/CS_Reef_Boundaries.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth5m_Coral-Sea.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth10m_Coral-Sea.shp

Murray Island 20 Sept 2011 15cm SISP aerial imagery, Queensland Spatial Imagery Services Program, Department of Resources, Queensland
This is the high resolution imagery used to create the map of Mer.

World_AIMS_Marine-satellite-imagery
The base image composites used in this dataset were based on an early version of Lawrey, E., Hammerton, M. (2024). Marine satellite imagery test collections (AIMS) [Data set]. eAtlas. https://doi.org/10.26274/zq26-a956. A snapshot of the code at the time this dataset was developed is made available in the 01-data/World_AIMS_Marine-satellite-imagery folder of the download of this dataset.


Data Location:
This dataset is filed in the eAtlas enduring data repository at: data\custodian\2020-2029-AIMS\TS_AIMS_Torres-Strait-Sentinel-2-regional-maps. On the eAtlas server it is stored at eAtlas GeoServer\data\2020-2029-AIMS.


Change Log:
2025-05-12: Eric Lawrey
Added Torres-Strait-Region-Map-Masig-Ugar-Erub-45k-A0 and Torres-Strait-Eastern-Region-Map-Landscape-A0. These maps have a brighten satellite imagery to allow easier reading of writing on the maps. They also include markers for geo-referencing the maps for digitisation.

2025-02-04: Eric Lawrey
Fixed up the reference to the World_AIMS_Marine-satellite-imagery dataset, clarifying where the source that was used in this dataset. Added ORCID and RORs to the record.

2023-11-22: Eric Lawrey
Added the data and maps for close up of Mer.
- 01-data/TS_DNRM_Mer-aerial-imagery/
- preview/Torres-Strait-Mer-Map-Landscape-A0.jpeg
- exports/Torres-Strait-Mer-Map-Landscape-A0.pdf
Updated 02-Torres-Strait-regional-maps.qgz to include the layout for the new map.

2023-03-02: Eric Lawrey
Created a merged version of the satellite imagery, with no alpha blending so that it can be used in ArcGIS Pro. It is now a single large GeoTiff image. The Google Earth Engine source code for the World_AIMS_Marine-satellite-imagery was included to improve the reproducibility and provenance of the dataset, along with a calculation of the distribution of image dates that went into the final composite image. A WMS service for the imagery was also setup and linked to from the metadata. A cross reference to the older Torres Strait clear sky clear water Landsat composite imagery was also added to the record.

Background When deriving information from satellite images it may be beneficial to see how an area usually looks over a given year rather than how it was viewed at a single point in time. Other useful interpretations include how dynamic or variable the Australian landscape is over any given year, especially to help with understanding how the landscape changes, or to discriminateparts of the landscape that stay the same throughout the year (like bare rock), from those areas that go through big changes in cover (such as cropping areas).

By understanding general conditions (which can be thought of as "average" for the year) and the different patterns of variation, we can characterise various types of land cover and land use, and detect changes of significance in the landscape.

The Geometric Median provides information on the general conditions of the landscape for a given year.

The Median Absolute Deviation provides information on how the landscape is changing in the same year.

The DEA GeoMAD product combines the Geometric Median and the Median Absolute Deviation in a single, annually produced package.

What this product offers This product provides statistical tools to exploit the time series of Earth observation data available in Digital Earth Australia, providing annual images of general conditions and how much an area changes for a given year.

Thegeomedian part of the product provides an "average" cloud-free image over the given year. The geomedian image is calculated with a multi-dimensional median, using all the spectral measurements from the satellite imagery at the same time in order to maintain the relationships among the measurements.

The median absolute deviation part of the product uses three measures of variance, each of whichprovides a "second order" high dimensional statistical compositefor the given year. The three variance measures show how much an area varies from the "average" in terms of "distance" based on factors such as brightness and spectra:

Euclidean distance (EMAD)
Cosine (spectral) distance (SMAD)
Bray Curtis dissimilarity (BCMAD)

Together, they provide information on variance in the landscape over the given year and are useful for change detection applications.

This dataset features a collection of historical orthorectified aerial photographed images of the Brisbane City Council local government area captured by piloted aircraft during 1946.Prior to satellite imagery, extensive use was made of aerial photography to capture land information. The 1946 imagery service uses the Geocentric Datum of Australia 1994 (GDA94) datum and is projected in Zone 56 of the Map Grid of Australia (MGA56).This dataset is a tile layer, to view the images or to access the data, use the ArcGIS Hub, HTML and API links in the Data and resources section below.

This dataset contains Sentinel 2 and Landsat 8 cloud free composite satellite images of the Coral Sea reef areas and some parts of the Great Barrier Reef. It also contains raw depth contours derived from the satellite imagery. This dataset was developed as the base information for mapping the boundaries of reefs and coral cays in the Coral Sea. It is likely that the satellite imagery is useful for numerous other applications. The full source code is available and can be used to apply these techniques to other locations.

This dataset contains two sets of raw satellite derived bathymetry polygons for 5 m, 10 m and 20 m depths based on both the Landsat 8 and Sentinel 2 imagery. These are intended to be post-processed using clipping and manual clean up to provide an estimate of the top structure of reefs. This dataset also contains select scenes on the Great Barrier Reef and Shark bay in Western Australia that were used to calibrate the depth contours. Areas in the GBR were compared with the GA GBR30 2020 (Beaman, 2017) bathymetry dataset and the imagery in Shark bay was used to tune and verify the Satellite Derived Bathymetry algorithm in the handling of dark substrates such as by seagrass meadows. This dataset also contains a couple of small Sentinel 3 images that were used to check the presence of reefs in the Coral Sea outside the bounds of the Sentinel 2 and Landsat 8 imagery.

The Sentinel 2 and Landsat 8 imagery was prepared using the Google Earth Engine, followed by post processing in Python and GDAL. The processing code is available on GitHub (https://github.com/eatlas/CS_AIMS_Coral-Sea-Features_Img).

This collection contains composite imagery for Sentinel 2 tiles (59 in Coral Sea, 8 in GBR) and Landsat 8 tiles (12 in Coral Sea, 4 in GBR and 1 in WA). For each Sentinel tile there are 3 different colour and contrast enhancement styles intended to highlight different features. These include:
- TrueColour - Bands: B2 (blue), B3 (green), B4 (red): True colour imagery. This is useful to identifying shallow features are and in mapping the vegetation on cays.
- DeepFalse - Bands: B1 (ultraviolet), B2 (blue), B3 (green): False colour image that shows deep marine features to 50 - 60 m depth. This imagery exploits the clear waters of the Coral Sea to allow the ultraviolet band to provide a much deeper view of coral reefs than is typically achievable with true colour imagery. This imagery has a high level of contrast enhancement applied to the imagery and so it appears more noisy (in particular showing artefact from clouds) than the TrueColour styling.
- Shallow - Bands: B5 (red edge), B8 (Near Infrared) , B11 (Short Wave infrared): This false colour imagery focuses on identifying very shallow and dry regions in the imagery. It exploits the property that the longer wavelength bands progressively penetrate the water less. B5 penetrates the water approximately 3 - 5 m, B8 approximately 0.5 m and B11 < 0.1 m. Features less than a couple of metres appear dark blue, dry areas are white. This imagery is intended to help identify coral cay boundaries.

For Landsat 8 imagery only the TrueColour and DeepFalse stylings were rendered.

All Sentinel 2 and Landsat 8 imagery has Satellite Derived Bathymetry (SDB) depth contours.
- Depth5m - This corresponds to an estimate of the area above 5 m depth (Mean Sea Level).
- Depth10m - This corresponds to an estimate of the area above 10 m depth (Mean Sea Level).
- Depth20m - This corresponds to an estimate of the area above 20 m depth (Mean Sea Level).

For most Sentinel and some Landsat tiles there are two versions of the DeepFalse imagery based on different collections (dates). The R1 imagery are composites made up from the best available imagery while the R2 imagery uses the next best set of imagery. This splitting of the imagery is to allow two composites to be created from the pool of available imagery. This allows any mapped features to be checked against two images. Typically the R2 imagery will have more artefacts from clouds. In one Sentinel 2 tile a third image was created to help with mapping the reef platform boundary.

The satellite imagery was processed in tiles (approximately 100 x 100 km for Sentinel 2 and 200 x 200 km for Landsat 8) to keep each final image small enough to manage. These tiles were not merged into a single mosaic as it allowed better individual image contrast enhancement when mapping deep features. The dataset only covers the portion of the Coral Sea where there are shallow coral reefs and where their might have been potential new reef platforms indicated by existing bathymetry datasets and the AHO Marine Charts. The extent of the imagery was limited by those available through the Google Earth Engine.

# Methods:

The Sentinel 2 imagery was created using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile, images from 2015 – 2021 were reviewed manually after first filtering to remove cloudy scenes. The allowable cloud cover was adjusted so that at least the 50 least cloud free images were reviewed. The typical cloud cover threshold was 1%. Where very few images were available the cloud cover filter threshold was raised to 100% and all images were reviewed. The Google Earth Engine image IDs of the best images were recorded, along with notes to help sort the images based on those with the clearest water, lowest waves, lowest cloud, and lowest sun glint. Images where there were no or few clouds over the known coral reefs were preferred. No consideration of tides was used in the image selection process. The collection of usable images were grouped into two sets that would be combined together into composite images. The best were added to the R1 composite, and the next best images into the R2 composite. Consideration was made as to whether each image would improve the resultant composite or make it worse. Adding clear images to the collection reduces the visual noise in the image allowing deeper features to be observed. Adding images with clouds introduces small artefacts to the images, which are magnified due to the high contrast stretching applied to the imagery. Where there were few images all available imagery was typically used.
2. Sunglint was removed from the imagery using estimates of the sunglint using two of the infrared bands (described in detail in the section on Sun glint removal and atmospheric correction).
3. A composite image was created from the best images by taking the statistical median of the stack of images selected in the previous stage, after masking out clouds and their shadows (described in detail later).
4. The brightness of the composite image was normalised so that all tiles would have a similar average brightness for deep water areas. This correction was applied to allow more consistent contrast enhancement. Note: this brightness adjustment was applied as a single offset across all pixels in the tile and so this does not correct for finer spatial brightness variations.
5. The contrast of the images was enhanced to create a series of products for different uses. The TrueColour colour image retained the full range of tones visible, so that bright sand cays still retain detail. The DeepFalse style was optimised to see features at depth and the Shallow style provides access to far red and infrared bands for assessing shallow features, such as cays and island.
6. The various contrast enhanced composite images were exported from Google Earth Engine and optimised using Python and GDAL. This optimisation added internal tiling and overviews to the imagery. The depth polygons from each tile were merged into shapefiles covering the whole for each depth.

## Cloud Masking

Prior to combining the best images each image was processed to mask out clouds and their shadows.

The cloud masking uses the COPERNICUS/S2_CLOUD_PROBABILITY dataset developed by SentinelHub (Google, n.d.; Zupanc, 2017). The mask includes the cloud areas, plus a mask to remove cloud shadows. The cloud shadows were estimated by projecting the cloud mask in the direction opposite the angle to the sun. The shadow distance was estimated in two parts.

A low cloud mask was created based on the assumption that small clouds have a small shadow distance. These were detected using a 40% cloud probability threshold. These were projected over 400 m, followed by a 150 m buffer to expand the final mask.

A high cloud mask was created to cover longer shadows created by taller, larger clouds. These clouds were detected based on an 80% cloud probability threshold, followed by an erosion and dilation of 300 m to remove small clouds. These were then projected over a 1.5 km distance followed by a 300 m buffer.

The buffering was applied as the cloud masking would often miss significant portions of the edges of clouds and their shadows. The buffering allowed a higher percentage of the cloud to be excluded, whilst retaining as much of the original imagery as possible.

The parameters for the cloud masking (probability threshold, projection distance and buffer radius) were determined through trial and error on a small number of scenes. The algorithm used is significantly better than the default Sentinel 2 cloud masking and slightly better than the COPERNICUS/S2_CLOUD_PROBABILITY cloud mask because it masks out shadows, however there is potentially significant improvements that could be made to the method in the future.

Erosion, dilation and buffer operations were performed at a lower image resolution than the native satellite image resolution to improve the computational speed. The resolution of these operations were adjusted so that they were performed with approximately a 4 pixel resolution during these operations. This made the cloud mask

VegMachine is an online tool that uses satellite imagery to summarise decades of change in Australia’s landscape. It’s simple to operate, easy to understand, and free to use.

With VegMachine you can: view satellite image land cover products; measure land cover change and fire scars; generate comprehensive ground cover monitoring reports and better understand the links between management, climate and vegetation cover.

This dataset features a collection of historical orthorectified aerial photographed images of the Brisbane City Council local government area captured by piloted aircraft during 1946.Prior to satellite imagery, extensive use was made of aerial photography to capture land information. The 1946 imagery service uses the Geocentric Datum of Australia 1994 (GDA94) and is projected in Zone 56 of the Map Grid of Australia (MGA56).This dataset utilises Brisbane City Council's Open Spatial Data website to provide additional features for viewing and downloading the data.

Geoscience Australia (GA) generated a series of gravity and magnetic grids and enhancements covering Northern Australia. Several derivative gravity datasets have been generated 1) for the North-West Shield Western Australia region (approximately between latitudes 7‒26⁰ S and longitudes 110‒130⁰ E), 2) for the Northern Territory (approximately between latitudes 7‒26⁰ S and longitudes 125.5‒141⁰ E) and for Queensland (approximately between latitudes 7‒30⁰ S and longitudes 135‒160⁰ E). The magnetic dataset has been generated only for the North-West Shield Western Australia region (approximately between latitudes 7‒26⁰ S and longitudes 110‒130⁰ E). The magnetic and gravity data were downloaded from the Geophysical Archive Data Delivery System (GADDS), website (http://www.geoscience.gov.au/cgi-bin/mapserv?map=/nas/web/ops/prod/apps/mapserver/gadds/wms_map/gadds.map&mode=browse). Satellite Free-air (FA) gravity v27.1 (released March 11, 2019) and Satellite Topography v19.1 (released January 14, 2019) data were sourced from Sandwell et al. (2014) and downloaded from the Scripps Institution of Oceanography (SIO), National Oceanic and Atmospheric Administration (NOAA), U.S. Navy and National Geospatial-Intelligence Agency (NGA) (SIO Satellite Geodesy, website, http://topex.ucsd.edu/WWW_html/mar_grav.html). The Satellite Bouguer gravity grid with onshore correction density of 2.67 gcm-3 and offshore correction density of 2.20 gcm-3 was derived from the Free-air gravity v27.1 and Topography data V19.1. This Bouguer gravity grid was used for filling areas of data gaps in the offshore region.

Data evaluation and processing of gravity and magnetic data available in the area of interest resulted in the production of stitched onshore-offshore Bouguer gravity grid derived from offshore satellite Bouguer gravity grid and GA’s onshore ground and airborne gravity survey data and a stitched Total Magnetic Intensity (TMI) grid derived from airborne and shipborne surveys (Tables 1 and 5). A Reduction to the Pole (RTP) grid was derived from the stitched TMI grid. The TMI, RTP, FA and terrain corrected Bouguer gravity anomalies are standard datasets for geological analysis. The free-air gravity anomaly provides the raw and basic gravity information. Images of free-air gravity are useful for first-pass interpretation and the data is used for gravity modelling. Magnetic anomalies provide information on numerous magnetic sources, including deep sources as arising from the structure and composition of magnetic basement and shallow sources such as intra-sedimentary magnetic units (e.g. volcanics, intrusions, and magnetic sedimentary layers). A standard TMI image will contain information from all these sources. Geosoft Oasis montaj software was used throughout the data processing and enhancement procedure and the montaj GridKnit module was used to generate the stitched gravity and magnetic grids.

Enhancement techniques have been applied to the final processed Bouguer gravity and RTP magnetic grids to highlight subtle features from various sources and to separate anomalies from different source depths. These enhancement techniques are described in the next section.

Enhancement processing techniques and results

A summary of image processing techniques used to achieve various outcomes is described in Table 1.

Data type Filter applied Enhancement/outcome

Gravity/Magnetic First vertical derivative (1VD) Near surface features (e.g. intrabasinal)

Gravity/Magnetic Upward continuation Noise reduction in data

Gravity/Magnetic Low pass filter, or large distance upward continuation Enhancement of deep features (e.g. basement)

Gravity/Magnetic High pass filter Enhancement of shallow features (e.g. surface anomalies)

Gravity/Magnetic Tilt filter and 1VD Enhancement of structure (e.g. in basement)

Gravity/Magnetic ZS-Edgezone and ZS-Edge filters Enhancement of edges

Gravity/Magnetic horizontal modulus / horizontal gradient Enhancement of boundaries

Magnetic RTP (reduction to the pole), Compound Anomaly, and Analytic Signal filter Accurate location of sources

Gravity Anomaly Greyscale Image of the Australian Region - 2010 An image made from the grid that represents gravity anomalies of the Australian region. The grid combines accurate onshore gravity measurements, with satellite data over the offshore region. The cell values represent simple Bouguer anomalies at a density of 2.67 tonnes per cubic metre onshore and free-air anomalies offshore. The grid cell size is 0.5 minutes of arc, which is equivalent to about 800 metres. Gravity Anomaly Greyscale Image of the Australian Region - 2010 An image made from the grid that represents gravity anomalies of the Australian region. The grid combines accurate onshore gravity measurements, with satellite data over the offshore region. The cell values represent simple Bouguer anomalies at a density of 2.67 tonnes per cubic metre onshore and free-air anomalies offshore. The grid cell size is 0.5 minutes of arc, which is equivalent to about 800 metres.

This 4th version of the Tanami 3D model covers a 200 km x 200 km area of the Tanami Region and Arunta Inlier (Aileron Province). This model consists of a series of 3D surfaces of major geological elements ranging from relatively shallow cover sequences, down to the Moho. The 3D surfaces were constructed using 3D GeoModeller and combines the interpreted seismic data from the 2005 Tanami Seismic Collaborative Research Project, with geological mapping, interpreted solid geology maps, and gravity and magnetic data. Elements of the model have been tested by gravity forward modelling.

Geographic extent

Tanami Region, Northern Territory-Western Australia.

Contents

Digital elevation model; images of magnetics, gravity, radiometrics, satellite imagery, mapped geology and regolith; roads; images of solid geology interpretation maps; modelled cross-sections; 3D fault surfaces; gravity and magnetic inversion surfaces; gravity and magnetics strings; mineral occurrences; SHRIMP geochronology; seismic images; and depth to basement points, surfaces and contours.

Software required

Geoscience Australia's X3D and older VRML models require the free plugin BS Contact and work best with the web browser Internet Explorer version 6 or higher.

Size

Approximately 27MB Startup download is 850KB - the remaining datasets download when selected.