This dataset contains Landsat 5 imagery for selected areas of Queensland, currently Torres Strait and around Lizard Island and Cape Tribulation. This collection was made as a result of the …Show full descriptionThis dataset contains Landsat 5 imagery for selected areas of Queensland, currently Torres Strait and around Lizard Island and Cape Tribulation. This collection was made as a result of the development of the Torres Strait Features dataset. It includes a number (typically 4 - 8) of selected Landsat images for each scene from the entire Landsat 5 archive. These images were selected for having low cloud cover and clear water. The aim of this collection was to allow investigation of the marine features. The complete catalogue of Landsat 5 for scenes 96_70, 96_71, 97_67, 97_68, 98_66, 98_67, 98_68_99_66, 99_67 and 99_68 were downloaded from the Google Earth Engine site ( https://console.developers.google.com/storage/earthengine-public/landsat/ ). The images were then processed into low resolution true colour using GDAL. They were then reviewed for picture clarity and the best ones were selected and processed at full resolution to be part of this collection. The true colour conversion was achieved by applying level adjustment to each channel to ensure that the tonal scaling of each channel was adjusted to give a good overall colour balance. This effectively set the black point of the channel and the gain. This adjustment was applied consistently to all images. Red: Channel B3, Black level 8, White level 58 Green: Channel B2, Black level 10, White level 55 Blue: Channel B1, Black level 32, White level 121 Note: A constant level adjustment was made to the images regardless of the time of the year that the images were taken. As a result images in the summer tend to be brighter than those in the winter. After level adjustment the three channels were merged into a single colour image using gdal_merge. The black surround on the image was then made transparent using the GDAL nearblack command. This collection consists of 59 images saved as 4 channel (Red, Green, Blue, Alpha) GeoTiff images with LZW compression (lossless) and internal overviews with a WGS 84 UTM 54N projection. Each of the individual images can be downloaded from the eAtlas map client (Overlay layers: eAtlas/Imagery Base Maps Earth Cover/Landsat 5) or as a collection of all images for each scene. Data Location: This dataset is filed in the eAtlas enduring data repository at: data\NERP-TE\13.1_eAtlas\QLD_NERP-TE-13-1_eAtlas_Landsat-5_1988-2011

This dataset consists of collections of satellite image composites (Sentinel 2 and Landsat 8) that are created from manually curated image dates for a range of projects. These images are typically prepared for subsequent analysis or testing of analysis algorithms as part of other projects. This dataset acts as a repository of reproducible test sets of images processed from Google Earth Engine using a standardised workflow.

Details of the algorithms used to produce the imagery are described in the GEE code and code repository available on GitHub (https://github.com/eatlas/World_AIMS_Marine-satellite-imagery).


Project test image sets:

As new projects are added to this dataset, their details will be described here:

- NESP MaC 2.3 Benthic reflection estimation (projects/CS_NESP-MaC-2-3_AIMS_Benth-reflect):
This collection consists of six Sentinel 2 image composites in the Coral Sea and GBR for the purpose of testing a method of determining benthic reflectance of deep lagoonal areas of coral atolls. These image composites are in GeoTiff format, using 16-bit encoding and LZW compression. These images do not have internal image pyramids to save on space.
[Status: final and available for download]

- NESP MaC 2.3 Oceanic Vegetation (projects/CS_NESP-MaC-2-3_AIMS_Oceanic-veg):
This project is focused on mapping vegetation on the bottom of coral atolls in the Coral Sea. This collection consists of additional images of Ashmore Reef. The lagoonal area of Ashmore has low visibility due to coloured dissolved organic matter, making it very hard to distinguish areas that are covered in vegetation. These images were manually curated to best show the vegetation. While these are the best images in the Sentinel 2 series up to 2023, they are still not very good. Probably 80 - 90% of the lagoonal benthos is not visible.
[Status: final and available for download]

- NESP MaC 3.17 Australian reef mapping (projects/AU_NESP-MaC-3-17_AIMS_Reef-mapping):
This collection of test images was prepared to determine if creating a composite from manually curated image dates (corresponding to images with the clearest water) would produce a better composite than a fully automated composite based on cloud filtering. The automated composites are described in https://doi.org/10.26274/HD2Z-KM55. This test set also includes composites from low tide imagery. The images in this collection are not yet available for download as the collection of images that will be used in the analysis has not been finalised.
[Status: under development, code is available, but not rendered images]

- Capricorn Regional Map (projects/CapBunk_AIMS_Regional-map): This collection was developed for making a set of maps for the region to facilitate participatory mapping and reef restoration field work planning.
[Status: final and available for download]

- Default (project/default): This collection of manual selected scenes are those that were prepared for the Coral Sea and global areas to test the algorithms used in the developing of the original Google Earth Engine workflow. This can be a good starting point for new test sets. Note that the images described in the default project are not rendered and made available for download to save on storage space.
[Status: for reference, code is available, but not rendered images]


Filename conventions:

The images in this dataset are all named using a naming convention. An example file name is Wld_AIMS_Marine-sat-img_S2_NoSGC_Raw-B1-B4_54LZP.tif. The name is made up of:
- Dataset name (Wld_AIMS_Marine-sat-img), short for World, Australian Institute of Marine Science, Marine Satellite Imagery.
- Satellite source: L8 for Landsat 8 or S2 for Sentinel 2.
- Additional information or purpose: NoSGC - No sun glint correction, R1 best reference imagery set or R2 second reference imagery.
- Colour and contrast enhancement applied (DeepFalse, TrueColour,Shallow,Depth5m,Depth10m,Depth20m,Raw-B1-B4),
- Image tile (example: Sentinel 2 54LZP, Landsat 8 091086)


Limitations:

Only simple atmospheric correction is applied to land areas and as a result the imagery only approximates the bottom of atmosphere reflectance.

For the sentinel 2 imagery the sun glint correction algorithm transitions between different correction levels from deep water (B8) to shallow water (B11) and a fixed atmospheric correction for land (bright B8 areas). Slight errors in the tuning of these transitions can result in unnatural tonal steps in the transitions between these areas, particularly in very shallow areas.

For the Landsat 8 image processing land areas appear as black from the sun glint correction, which doesn't separately mask out the land. The code for the Landsat 8 imagery is less developed than for the Sentinel 2 imagery.

The depth contours are estimated using satellite derived bathymetry that is subject to errors caused by cloud artefacts, substrate darkness, water clarity, calibration issues and uncorrected tides. They were tuned in the clear waters of the Coral Sea. The depth contours in this dataset are RAW and contain many false positives due to clouds. They should not be used without additional dataset cleanup.



Change log:

As changes are made to the dataset, or additional image collections are added to the dataset then those changes will be recorded here.

2nd Edition, 2024-06-22: CapBunk_AIMS_Regional-map
1st Edition, 2024-03-18: Initial publication of the dataset, with CS_NESP-MaC-2-3_AIMS_Benth-reflect, CS_NESP-MaC-2-3_AIMS_Oceanic-veg and code for AU_NESP-MaC-3-17_AIMS_Reef-mapping and Default projects.


Data Format:

GeoTiff images with LZW compression. Most images do not have internal image pyramids to save on storage space. This makes rendering these images very slow in a desktop GIS. Pyramids should be added to improve performance.

Data Location:

This dataset is filed in the eAtlas enduring data repository at: data\custodian\2020-2029-AIMS\Wld-AIMS-Marine-sat-img

This dataset contains both large (A0) printable maps of the Torres Strait broken into six overlapping regions, based on a clear sky, clear water composite Sentinel 2 composite imagery and the imagery used to create these maps. These maps show satellite imagery of the region, overlaid with reef and island boundaries and names. Not all features are named, just the more prominent features. This also includes a vector map of Ashmore Reef and Boot Reef in Coral Sea as these were used in the same discussions that these maps were developed for. The map of Ashmore Reef includes the atoll platform, reef boundaries and depth polygons for 5 m and 10 m.

This dataset contains all working files used in the development of these maps. This includes all a copy of all the source datasets and all derived satellite image tiles and QGIS files used to create the maps. This includes cloud free Sentinel 2 composite imagery of the Torres Strait region with alpha blended edges to allow the creation of a smooth high resolution basemap of the region.

The base imagery is similar to the older base imagery dataset: Torres Strait clear sky, clear water Landsat 5 satellite composite (NERP TE 13.1 eAtlas, AIMS, source: NASA).

Most of the imagery in the composite imagery from 2017 - 2021.


Method:
The Sentinel 2 basemap was produced by processing imagery from the World_AIMS_Marine-satellite-imagery dataset (01-data/World_AIMS_Marine-satellite-imagery in the data download) for the Torres Strait region. The TrueColour imagery for the scenes covering the mapped area were downloaded. Both the reference 1 imagery (R1) and reference 2 imagery (R2) was copied for processing. R1 imagery contains the lowest noise, most cloud free imagery, while R2 contains the next best set of imagery. Both R1 and R2 are typically composite images from multiple dates.

The R2 images were selectively blended using manually created masks with the R1 images. This was done to get the best combination of both images and typically resulted in a reduction in some of the cloud artefacts in the R1 images. The mask creation and previewing of the blending was performed in Photoshop. The created masks were saved in 01-data/R2-R1-masks. To help with the blending of neighbouring images a feathered alpha channel was added to the imagery. The processing of the merging (using the masks) and the creation of the feathered borders on the images was performed using a Python script (src/local/03-merge-R2-R1-images.py) using the Pillow library and GDAL. The neighbouring image blending mask was created by applying a blurring of the original hard image mask. This allowed neighbouring image tiles to merge together.

The imagery and reference datasets (reef boundaries, EEZ) were loaded into QGIS for the creation of the printable maps.

To optimise the matching of the resulting map slight brightness adjustments were applied to each scene tile to match its neighbours. This was done in the setup of each image in QGIS. This adjustment was imperfect as each tile was made from a different combinations of days (to remove clouds) resulting in each scene having a different tonal gradients across the scene then its neighbours. Additionally Sentinel 2 has slight stripes (at 13 degrees off the vertical) due to the swath of each sensor having a slight sensitivity difference. This effect was uncorrected in this imagery.


Single merged composite GeoTiff:
The image tiles with alpha blended edges work well in QGIS, but not in ArcGIS Pro. To allow this imagery to be used across tools that don't support the alpha blending we merged and flattened the tiles into a single large GeoTiff with no alpha channel. This was done by rendering the map created in QGIS into a single large image. This was done in multiple steps to make the process manageable.

The rendered map was cut into twenty 1 x 1 degree georeferenced PNG images using the Atlas feature of QGIS. This process baked in the alpha blending across neighbouring Sentinel 2 scenes. The PNG images were then merged back into a large GeoTiff image using GDAL (via QGIS), removing the alpha channel. The brightness of the image was adjusted so that the darkest pixels in the image were 1, saving the value 0 for nodata masking and the boundary was clipped, using a polygon boundary, to trim off the outer feathering. The image was then optimised for performance by using internal tiling and adding overviews. A full breakdown of these steps is provided in the README.md in the 'Browse and download all data files' link.

The merged final image is available in export\TS_AIMS_Torres Strait-Sentinel-2_Composite.tif.


Source datasets:
Complete Great Barrier Reef (GBR) Island and Reef Feature boundaries including Torres Strait Version 1b (NESP TWQ 3.13, AIMS, TSRA, GBRMPA), https://eatlas.org.au/data/uuid/d2396b2c-68d4-4f4b-aab0-52f7bc4a81f5

Geoscience Australia (2014b), Seas and Submerged Lands Act 1973 - Australian Maritime Boundaries 2014a - Geodatabase [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, https://dx.doi.org/10.4225/25/5539DFE87D895

Basemap/AU_GA_AMB_2014a/Exclusive_Economic_Zone_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014a). The Geodatabase was loaded in ArcMap. The Exclusive_Economic_Zone_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

Geoscience Australia (2014a), Treaties - Australian Maritime Boundaries (AMB) 2014a [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, http://dx.doi.org/10.4225/25/5539E01878302
Basemap/AU_GA_Treaties-AMB_2014a/Papua_New_Guinea_TSPZ_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014b). The Geodatabase was loaded in ArcMap. The Papua_New_Guinea_TSPZ_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

AIMS Coral Sea Features (2022) - DRAFT
This is a draft version of this dataset. The region for Ashmore and Boot reef was checked. The attributes in these datasets haven't been cleaned up. Note these files should not be considered finalised and are only suitable for maps around Ashmore Reef. Please source an updated version of this dataset for any other purpose.
CS_AIMS_Coral-Sea-Features/CS_Names/Names.shp
CS_AIMS_Coral-Sea-Features/CS_Platform_adj/CS_Platform.shp
CS_AIMS_Coral-Sea-Features/CS_Reef_Boundaries_adj/CS_Reef_Boundaries.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth5m_Coral-Sea.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth10m_Coral-Sea.shp

Murray Island 20 Sept 2011 15cm SISP aerial imagery, Queensland Spatial Imagery Services Program, Department of Resources, Queensland
This is the high resolution imagery used to create the map of Mer.

World_AIMS_Marine-satellite-imagery
The base image composites used in this dataset were based on an early version of Lawrey, E., Hammerton, M. (2024). Marine satellite imagery test collections (AIMS) [Data set]. eAtlas. https://doi.org/10.26274/zq26-a956. A snapshot of the code at the time this dataset was developed is made available in the 01-data/World_AIMS_Marine-satellite-imagery folder of the download of this dataset.


Data Location:
This dataset is filed in the eAtlas enduring data repository at: data\custodian\2020-2029-AIMS\TS_AIMS_Torres-Strait-Sentinel-2-regional-maps. On the eAtlas server it is stored at eAtlas GeoServer\data\2020-2029-AIMS.


Change Log:
2025-05-12: Eric Lawrey
Added Torres-Strait-Region-Map-Masig-Ugar-Erub-45k-A0 and Torres-Strait-Eastern-Region-Map-Landscape-A0. These maps have a brighten satellite imagery to allow easier reading of writing on the maps. They also include markers for geo-referencing the maps for digitisation.

2025-02-04: Eric Lawrey
Fixed up the reference to the World_AIMS_Marine-satellite-imagery dataset, clarifying where the source that was used in this dataset. Added ORCID and RORs to the record.

2023-11-22: Eric Lawrey
Added the data and maps for close up of Mer.
- 01-data/TS_DNRM_Mer-aerial-imagery/
- preview/Torres-Strait-Mer-Map-Landscape-A0.jpeg
- exports/Torres-Strait-Mer-Map-Landscape-A0.pdf
Updated 02-Torres-Strait-regional-maps.qgz to include the layout for the new map.

2023-03-02: Eric Lawrey
Created a merged version of the satellite imagery, with no alpha blending so that it can be used in ArcGIS Pro. It is now a single large GeoTiff image. The Google Earth Engine source code for the World_AIMS_Marine-satellite-imagery was included to improve the reproducibility and provenance of the dataset, along with a calculation of the distribution of image dates that went into the final composite image. A WMS service for the imagery was also setup and linked to from the metadata. A cross reference to the older Torres Strait clear sky clear water Landsat composite imagery was also added to the record.

World Imagery provides one meter or better satellite and aerial imagery for most of the world’s landmass and lower resolution satellite imagery worldwide. The map is currently comprised of the following sources: Worldwide 15-m resolution TerraColor imagery at small and medium map scales.Maxar imagery basemap products around the world: Vivid Premium at 15-cm HD resolution for select metropolitan areas, Vivid Advanced 30-cm HD for more than 1,000 metropolitan areas, and Vivid Standard from 1.2-m to 0.6-cm resolution for the most of the world, with 30-cm HD across the United States and parts of Western Europe. More information on the Maxar products is included below. High-resolution aerial photography contributed by the GIS User Community. This imagery ranges from 30-cm to 3-cm resolution. You can contribute your imagery to this map and have it served by Esri via the Community Maps Program. Maxar Basemap ProductsVivid PremiumProvides committed image currency in a high-resolution, high-quality image layer over defined metropolitan and high-interest areas across the globe. The product provides 15-cm HD resolution imagery.Vivid AdvancedProvides committed image currency in a high-resolution, high-quality image layer over defined metropolitan and high-interest areas across the globe. The product includes a mix of native 30-cm and 30-cm HD resolution imagery.Vivid StandardProvides a visually consistent and continuous image layer over large areas through advanced image mosaicking techniques, including tonal balancing and seamline blending across thousands of image strips. Available from 1.2-m down to 30-cm HD. More on Maxar HD. Imagery UpdatesYou can use the Updates Mode in the World Imagery Wayback app to learn more about recent and pending updates. Accessing this information requires a user login with an ArcGIS organizational account. CitationsThis layer includes imagery provider, collection date, resolution, accuracy, and source of the imagery. With the Identify tool in ArcGIS Desktop or the ArcGIS Online Map Viewer you can see imagery citations. Citations returned apply only to the available imagery at that location and scale. You may need to zoom in to view the best available imagery. Citations can also be accessed in the World Imagery with Metadata web map.UseYou can add this layer to the ArcGIS Online Map Viewer, ArcGIS Desktop, or ArcGIS Pro. To view this layer with a useful reference overlay, open the Imagery Hybrid web map.FeedbackHave you ever seen a problem in the Esri World Imagery Map that you wanted to report? You can use the Imagery Map Feedback web map to provide comments on issues. The feedback will be reviewed by the ArcGIS Online team and considered for one of our updates.

A basemap image service that displays the best openly available (latest and highest spatial resolution) authoritative general reference imagery datasets captured by remotely piloted aircraft systems (drones), piloted aircraft, and satellite space craft over areas of Queensland. This basemap service is designed to provide access to the latest publicly available State Remotely Sensed Image Library collection of aerial imagery capture under the Spatial Imagery Services Program (SISP). Aerial imagery that is three years or older captured under SISP is made available for public use openly by the Department of Resources, Queensland. Satellite imagery (Planet Q3 2017 mosaic) is visible in areas over Queensland where aerial photography is unavailable. Basemap services comprise a single layer of static imagery optimised for display purposes. This service has a tile cache built down to a scale of 1:1129. The tile cache can be turned off in client software for viewing at lower scales and printing purposes, and can be used in dynamic mode to filter and display individual project areas. The projects range from 2cm to 240cm resolution. Accuracy is dependent on the individual projects. Periodical updates will be made to the service as new projects are captured. Projects are visible at scales 1:50,000, 1:250,000 and 1:25000000. The images comprised in each of the projects are orthorectified which removes the effects of image perspective (tilt) and relief (terrain) effects for the purpose of creating a planimetrically correct image. The resultant orthorectified images have a constant scale wherein features are represented in their 'true' positions. If you would like to receive updates for new projects, functionality or planned downtime please subscribe here: http://ems.gs/3qln0iYbIYQ.

World Imagery provides one meter or better satellite and aerial imagery in many parts of the world and lower resolution satellite imagery worldwide. The map includes 15m TerraColor imagery at small and mid-scales (~1:591M down to ~1:72k) and 2.5m SPOT Imagery (~1:288k to ~1:72k) for the world. The map features 0.5m resolution imagery in the continental United States and parts of Western Europe from Maxar. Additional Maxar sub-meter imagery is featured in many parts of the world. In the United States, 1 meter or better resolution NAIP imagery is available in some areas. In other parts of the world, imagery at different resolutions has been contributed by the GIS User Community. In select communities, very high resolution imagery (down to 0.03m) is available down to ~1:280 scale. You can contribute your imagery to this map and have it served by Esri via the Community Maps Program. View the list of Contributors for the World Imagery Map.See World Imagery for more information on this map.Metadata: Point and click on the map to see the resolution, collection date, and source of the imagery. Values of "99999" mean that metadata is not available for that field. The metadata applies only to the best available imagery at that location. You may need to zoom in to view the best available imagery.Feedback: Have you ever seen a problem in the Esri World Imagery Map that you wanted to see fixed? You can use the Imagery Map Feedback web map to provide feedback on issues or errors that you see. The feedback will be reviewed by the ArcGIS Online team and considered for one of our updates.Need Newer Imagery?: If you need to access more recent or higher resolution imagery, you can find and order that in the Content Store for ArcGIS app.Learn MoreGet AccessOpen App

This dataset is true colour cloud-free composite satellite imagery optimised for mapping shallow marine habitats in northern Australia, based on 10-meter resolution Sentinel 2 data collected from 2015 to 2024. It contains composite imagery for 333 Sentinel 2 tiles of northern Australia and the Great Barrier Reef. This dataset offers improved visual clarity of shallow water features as compared to existing satellite imagery, allowing deeper marine features to be observed. These composites were specifically designed to address challenges such as sun glint, clouds and turbidity that typically hinder marine environment analyses. No tides were considered in the selection of the imagery and so this imagery corresponds to an 'All tide' image, approximating mean sea level.

This dataset is an updated version (Version 2), published in July 2024, which succeeds the initial draft version (Version 1, published in March 2024). The current version spans imagery from 2015–2024, an extension of the earlier timeframe that covered 2018–2022. This longer temporal range allowed the imagery to be cleaner with lower image noise allowing deeper marine features to be visible. The deprecated draft version was removed from online download to save on storage space and is now only available on request.

While the final imagery corresponds to true colour based primarily Sentinel 2 bands B2 (blue), B3 (green), and B4 (red), the near infrared (B8) band was used as part of sun glint correction and automated selection of low noise imagery.

Contrast enhancement was applied to the imagery to compress the original 12 bit per channel Sentinel 2 imagery into the final 8-bit per channel GeoTiffs. Black and white point correction was used to enhance the contrast as much as possible without too much clipping of the darkest and lightest marine features. Gamma correction of 2 (red), 2 (green) and 2.3 (blue) was applied allow a wider dynamic range to be represented in the 8-bit data, helping to ensure that little precision was lost in representing darker marine features. As a result, the image brightness is not linearly scaled. Further details of the corrections applied is available from https://github.com/eatlas/AU_NESP-MaC-3-17_AIMS_S2-comp/blob/main/src/processors/s2processor.py.


Methods:

The satellite image composites were created by combining multiple Sentinel 2 images using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile filter the "COPERNICUS/S2_HARMONIZED" image collection by
- tile ID
- maximum cloud cover 20%
- date between '2015-06-27' and '2024-05-31'
- asset_size > 100000000 (remove small fragments of tiles)
Note: A maximum cloud cover of 20% was used to improve the processing times. In most cases this filtering does not have an effect on the final composite as images with higher cloud coverage mostly result in higher noise levels and are not used in the final composite.
2. Split images by "SENSING_ORBIT_NUMBER" (see "Using SENSING_ORBIT_NUMBER for a more balanced composite" for more information).
3. For each SENSING_ORBIT_NUMBER collection filter out all noise-adding images:
3.1 Calculate image noise level for each image in the collection (see "Image noise level calculation for more information") and sort collection by noise level.
3.2 Remove all images with a very high noise index (>15).
3.3 Calculate a baseline noise level using a minimum number of images (min_images_in_collection=30). This minimum number of images is needed to ensure a smoth composite where cloud "holes" in one image are covered by other images.
3.4 Iterate over remaining images (images not used in base noise level calculation) and check if adding image to the composite adds to or reduces the noise. If it reduces the noise add it to the composite. If it increases the noise stop iterating over images.
4. Combine SENSING_ORBIT_NUMBER collections into one image collection.
5. Remove sun-glint (true colour only) and apply atmospheric correction on each image (see "Sun-glint removal and atmospheric correction" for more information).
6. Duplicate image collection to first create a composite image without cloud masking and using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
7. Apply cloud masking to all images in the original image collection (see "Cloud Masking" for more information) and create a composite by using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
8. Combine the two composite images (no cloud mask composite and cloud mask composite). This solves the problem of some coral cays and islands being misinterpreted as clouds and therefore creating holes in the composite image. These holes are "plugged" with the underlying composite without cloud masking. (Lawrey et al. 2022)
9. The final composite was exported as cloud optimized 8 bit GeoTIFF

Note: The following tiles were generated with no "maximum cloud cover" as they did not have enough images to create a composite with the standard settings: 46LGM, 46LGN, 46LHM, 50KKD, 50KPG, 53LMH, 53LMJ, 53LNH, 53LPH, 53LPJ, 54LVP, 57JVH, 59JKJ.

Compositing Process:

The dataset was created using a multi-step compositing process. A percentile-based image compositing technique was employed, with the 15th percentile chosen as the optimal value for most regions. This percentile was identified as the most effective in minimizing noise and enhancing key features such as coral reefs, islands, and other shallow water habitats. The 15th percentile was chosen as a trade off between the desire to select darker pixels that typically correspond to clearer water, and very dark values (often occurring at the 10th percentile) corresponding to cloud shadows.

The cloud masking predictor would often misinterpret very pale areas, such as cays and beaches as clouds. To overcome this limitation a dual-image compositing method was used. A primary composite was generated with cloud masks applied, and a secondary, composite with no cloud masking was layered beneath to fill in potential gaps (or “holes”) caused by the cloud masking mistakes

Image noise level calculation:

The noise level for each image in this dataset is calculated to ensure high-quality composites by minimizing the inclusion of noisy images. This process begins by creating a water mask using the Normalized Difference Water Index (NDWI) derived from the NIR and Green bands. High reflectance areas in the NIR and SWIR bands, indicative of sun-glint, are identified and masked by the water mask to focus on water areas affected by sun-glint. The proportion of high sun-glint pixels within these water areas is calculated and amplified to compute a noise index. If no water pixels are detected, a high noise index value is assigned.

In any set of satellite images, some will be taken under favourable conditions (low wind, low sun-glint, and minimal cloud cover), while others will be affected by high sun-glint or cloud. Combining multiple images into a composite reduces noise by averaging out these fluctuations.

When all images have the same noise level, increasing the number of images in the composite reduces the overall noise. However, in practice, there is a mix of high and low noise images. The optimal composite is created by including as many low-noise images as possible while excluding high-noise ones. The challenge lies in the determining the acceptable noise threshold for a given scene as some areas are more cloudy and sun glint affected than others.

To address this, we rank the available Sentinel 2 images for each scene by their noise index, from lowest to highest. The goal is to determine the ideal number of images (N) to include in the composite to minimize overall noise. For each N, we use the lowest noise images and estimate the final composite noise based on the noise index. This is repeated for all values of N up to a maximum of 200 images, and we select the N that results in the lowest noise.

This approach has some limitations. It estimates noise based on sun glint and residual clouds (after cloud masking) using NIR bands, without accounting for image turbidity. The final composite noise is not directly measured as this would be computationally expensive. It is instead estimated by dividing the average noise of the selected images by the square root of the number of images. We found this method tends to underestimate the ideal image count, so we adjusted the noise estimates, scaling them by the inverse of their ranking, to favor larger sets of images. The algorithm is not fully optimized, and further refinement is needed to improve accuracy.

Full details of the algorithm can be found in https://github.com/eatlas/AU_NESP-MaC-3-17_AIMS_S2-comp/blob/main/src/utilities/noise_predictor.py

Sun glint removal and atmospheric correction:

Sun glint was removed from the images using the infrared B8 band to estimate the reflection off the water from the sun glint. B8 penetrates water less than 0.5 m and so in water areas it only detects reflections off the surface of the water. The sun glint detected by B8 correlates very highly with the sun glint experienced by the visible channels (B2, B3 and B4) and so the sun glint in these channels can be removed by subtracting B8 from these channels.

Eric Lawrey developed this algorithm by fine tuning the value of the scaling between the B8 channel and each individual visible channel (B2, B3 and B4) so that the maximum level of sun glint would be removed. This work was based on a representative set of images, trying to determine a set of values that represent a good compromise across different water surface

Abstract:This dataset contains bathymetry (depth) products from the compilation of all available source bathymetry data within the Great Barrier Reef into a 30 m-resolution Digital Elevation Model (DEM).The Great Barrier Reef (GBR) is the largest coral reef ecosystem on Earth and stretches over 2500 km along the north-eastern Australia margin.Bathymetry mapping of this extensive reef system is vital for the protection of the GBR allowing for the safe navigation of shipping and improved environmental management.Over the past ten years, deep-water multibeam surveys have revealed the highly complex shelf-edge drowned reefs and continental slope canyons.Airborne LiDAR bathymetry acquired by the Australian Hydrographic Service cover most of the GBR reefs, with coverage gaps supplemented by satellite derived bathymetry.The Geoscience Australia-developed Intertidal Elevation Model DEM improves the source data gap along Australia’s vast intertidal zone.All source bathymetry data were extensively edited as point clouds to remove noise, given a consistent WGS84 horizontal datum, and where possible, an approximate MSL vertical datum.The High-resolution depth model for the Great Barrier Reef - 30 m (Version 10 Nov 2020) can be downloaded as four separate but overlapping grids, with the area coverage:Great Barrier Reef A 2020 30m 10-17°S 143-147°EGreat Barrier Reef B 2020 30m 16-23°S 144-149°EGreat Barrier Reef C 2020 30m 18-24°S 148-154°EGreat Barrier Reef D 2020 30m 23-29°S 150-156°EDownloads and Links:Web Serviceshttps://warehouse.ausseabed.gov.au/geoserver/ows?REQUEST=GetCapabilities&SERVICE=WMS&VERSION=1.3.0Downloads available from the expanded catalogue link, belowMetadata URL:https://pid.geoscience.gov.au/dataset/ga/115066

This dataset collection contains A0 maps of the Keppel Island region based on satellite imagery and fine-scale habitat mapping of the islands and marine environment. This collection provides the source satellite imagery used to produce these maps and the habitat mapping data.

The imagery used to produce these maps was developed by blending high-resolution imagery (1 m) from ArcGIS Online with a clear-sky composite derived from Sentinel 2 imagery (10 m). The Sentinel 2 imagery was used to achieve full coverage of the entire region, while the high-resolution was used to provide detail around island areas.

The blended imagery is a derivative product of the Sentinel 2 imagery and ArcGIS Online imagery, using Photoshop to to manually blend the best portions of each imagery into the final product. The imagery is provided for the sole purpose of reproducing the A0 maps.

Methods:

The high resolution satellite composite composite was developed by manual masking and blending of a Sentinel 2 composite image and high resolution imagery from ArcGIS Online World Imagery (2019).

The Sentinel 2 composite was produced by statistically combining the clearest 10 images from 2016 - 2019. These images were manually chosen based on their very low cloud cover, lack of sun glint and clear water conditions. These images were then combined together to remove clouds and reduce the noise in the image.

The processing of the images was performed using a script in Google Earth Engine. The script combines the manually chosen imagery to estimate the clearest imagery. The dates of the images were chosen using the EOBrowser (https://www.sentinel-hub.com/explore/eobrowser) to preview all the Sentinel 2 imagery from 2015-2019. The images that were mostly free of clouds, with little or no sun glint, were recorded. Each of these dates was then viewed in Google Earth Engine with high contrast settings to identify images that had high water surface noise due to algal blooms, waves, or re-suspension. These were excluded from the list. All the images were then combined by applying a histogram analysis of each pixel, with the final image using the 40th percentile of the time series of the brightness of each pixel. This approach helps exclude effects from clouds.

The contrast of the image was stretched to highlight the marine features, whilst retaining detail in the land features. This was done by choosing a black point for each channel that would provide a dark setting for deep clear water. Gamma correction was then used to lighten up the dark water features, whilst not ove- exposing the brighter shallow areas.

Both the high resolution satellite imagery and Sentinel 2 imagery was combined at 1 m pixel resolution. The resolution of the Sentinel 2 tiles was up sampled to match the resolution of the high-resolution imagery. These two sets of imagery were then layered in Photoshop. The brightness of the high-resolution satellite imagery was then adjusting to match the Sentinel 2 imagery. A mask was then used to retain and blend the imagery that showed the best detail of each area. The blended tiles were then merged with the overall area imagery by performing a GDAL merge, resulting in an upscaling of the Sentinel 2 imagery to 1 m resolution.


Habitat Mapping:

A 5 m resolution habitat mapping was developed based on the satellite imagery, aerial imagery available, and monitoring site information. This habitat mapping was developed to help with monitoring site selection and for the mapping workshop with the Woppaburra TOs on North Keppel Island in Dec 2019.

The habitat maps should be considered as draft as they don't consider all available in water observations. They are primarily based on aerial and satellite images.

The habitat mapping includes: Asphalt, Buildings, Mangrove, Cabbage-tree palm, Sheoak, Other vegetation, Grass, Salt Flat, Rock, Beach Rock, Gravel, Coral, Sparse coral, Unknown not rock (macroalgae on rubble), Marine feature (rock).

This assumed layers allowed the digitisation of these features to be sped up, so for example, if there was coral growing over a marine feature then the boundary of the marine feature would need to be digitised, then the coral feature, but not the boundary between the marine feature and the coral. We knew that the coral was going to cut out from the marine feature because the coral is on top of the marine feature, saving us time in digitising this boundary. Digitisation was performed on an iPad using Procreate software and an Apple pencil to draw the features as layers in a drawing. Due to memory limitations of the iPad the region was digitised using 6000x6000 pixel tiles. The raster images were converted back to polygons and the tiles merged together.

A python script was then used to clip the layer sandwich so that there is no overlap between feature types.

Habitat Validation:

Only limited validation was performed on the habitat map. To assist in the development of the habitat mapping, nearly every YouTube video available, at the time of development (2019), on the Keppel Islands was reviewed and, where possible, georeferenced to provide a better understanding of the local habitats at the scale of the mapping, prior to the mapping being conducted. Several validation points were observed during the workshop. The map should be considered as largely unvalidated.

data/coastline/Keppels_AIMS_Coastline_2017.shp:
The coastline dataset was produced by starting with the Queensland coastline dataset by DNRME (Downloaded from http://qldspatial.information.qld.gov.au/catalogue/custom/detail.page?fid={369DF13C-1BF3-45EA-9B2B-0FA785397B34} on 31 Aug 2019). This was then edited to work at a scale of 1:5000, using the aerial imagery from Queensland Globe as a reference and a high-tide satellite image from 22 Feb 2015 from Google Earth Pro. The perimeter of each island was redrawn. This line feature was then converted to a polygon using the "Lines to Polygon" QGIS tool. The Keppel island features were then saved to a shapefile by exporting with a limited extent.

data/labels/Keppel-Is-Map-Labels.shp:
This contains 70 named places in the Keppel island region. These names were sourced from literature and existing maps. Unfortunately, no provenance of the names was recorded. These names are not official. This includes the following attributes:
- Name: Name of the location. Examples Bald, Bluff
- NameSuffix: End of the name which is often a description of the feature type: Examples: Rock, Point
- TradName: Traditional name of the location
- Scale: Map scale where the label should be displayed.

data/lat/Keppel-Is-Sentinel2-2016-19_B4-LAT_Poly3m_V3.shp:
This corresponds to a rough estimate of the LAT contours around the Keppel Islands. LAT was estimated from tidal differences in Sentinel-2 imagery and light penetration in the red channel. Note this is not very calibrated and should be used as a rough guide. Only one rough in-situ validation was performed at low tide on Ko-no-mie at the edge of the reef near the education centre. This indicated that the LAT estimate was within a depth error range of about +-0.5 m.

data/habitat/Keppels_AIMS_Habitat-mapping_2019.shp:
This shapefile contains the mapped land and marine habitats. The classification type is recorded in the Type attribute.

Format:

GeoTiff (Internal JPEG format - 538 MB)
PDF (A0 regional maps - ~30MB each)
Shapefile (Habitat map, Coastline, Labels, LAT estimate)

Data Location:

This dataset is filed in the eAtlas enduring data repository at: data\custodian\2020-2029-AIMS\Keppels_AIMS_Regional-maps

PLEASE NOTE:

_ GEEBAM is an interim product and there is no ground truthing or assessment of accuracy. Fire Extent and Severity Mapping (FESM) data should be used for accurate information on fire severity and loss of biomass in relation to bushfires._

The intention of this dataset was to provide a rapid assessment of fire impact.

In collaboration with the University of NSW, the NSW Department of Planning Infrastructure and Environment (DPIE) Remote Sensing and Landscape Science team has developed a rapid mapping approach to find out where wildfires in NSW have affected vegetation. We call it the Google Earth Engine Burnt Area Map (GEEBAM) and it relies on Sentinel 2 satellite imagery. The product output is a TIFF image with a resolution of 15m. Burnt Area Classes:

1. Little change observed between pre and post fire

2. Canopy unburnt - A green canopy within the fire ground that may act as refugia for native fauna, may be affected by fire

3. Canopy partially affected - A mix of burnt and unburnt canopy vegetation

4. Canopy fully affected -The canopy and understorey are most likely burnt

Using GEEBAM at a local scale requires visual interpretation with reference to satellite imagery. This will ensure the best results for each fire or vegetation class.

Important Note: GEEBAM is an interim product and there is no ground truthing or assessment of accuracy. It is updated fortnightly.

Please see Google Earth Engine Burnt Area Factsheet

This dataset consists of a shapefile of the reefs, islands, sand banks, cays and rocks of the whole Great Barrier Reef (GBR) including Torres Strait. This dataset is an extension of the mapping in the GBR Marine Park to include Torres Strait. The Torres Strait region was mapped at a scale of 1:50,000 (Lawrey, E. P., Stewart M., 2016) and these new features are referred to as the "Torres Strait Reef and Island Features" dataset.

The Complete GBR Reef and Island Features dataset integrates the "Torres Strait Reef and Island Features" dataset with the existing "GBR Features" (Great Barrier Reef Marine Park Authority, 2007) to create a single composite dataset of the whole Great Barrier Reef. This dataset includes 9600 features overall with 5685 from the "GBR Features" dataset and 3927 from the "Torres Strait Reef and Island Features" dataset.

These two datasets can be easily separated if necessary based on the "DATASET" attribute.

All new mapped features in Torres Strait were allocated permanent IDs (such as 10-479 for Thursday Island and 09-246 for Mabuiag Reef). These IDs are for easy unambiguous communication of features, especially for unnamed features.

The reference imagery used for the mapping of the reefs is available on request as it is large (~45 GB). These files are saved in the eAtlas enduring repository.

Methods:

This project mapped Torres Strait using a combination of existing island datasets as well as a semi-automated and manual digitising of marine features (reefs and sand banks) from the latest aerial and satellite imagery. No features were added to the dataset without confirmed evidence of their existence and position from at least two satellite image sources. The Torres Strait Reef and Island Feature mapping was integrated with the existing "GBR Features" dataset by GBRMPA to ensure that there were no duplicate feature ID allocations and to create a single dataset of the whole GBR. The overall dataset development was as follows: 1. Dataset collation and image preparation: - Collation of existing maps and datasets. - Download and preparation of the Landsat 5, 7, and 8 satellite image archive for Torres Strait. - Spatial position correction of Landsat imagery against a known reference image. 2. Sand Bank features: - Manual digitisation of sand banks from Landsat 5 imagery. - Conversion to a polygon shapefile for integration with the reef features. 3. Reef features: - Semi-automated digitisation of the marine features from Landsat 5 imagery. - Manual trimming, cleaning and checking of marine features against available aerial and satellite imagery. 4. Island features: - Compilation of island features from existing datasets (DNRM 1:25k Queensland Coastline, and Geoscience Australia Geodata Coast 100k 2004) - Correction of the island features from available aerial and Landsat imagery. 5. Merging: of marine and island features into one dataset. 6. Classification: of mapped features, including splitting fringing reefs based on changes in classification. 7. ID allocation: - Clustering to make groups of related features (i.e. an island, plus its fringing reefs and related sand banks; a reef plus its neighbouring patch reefs, etc.).
- Merging with the GBR Features dataset. This was to ensure that there were no duplicate allocations of feature IDs. This involved removing any overlapping features above the Great Barrier Reef Marine Park from the GBR Feature dataset. - Allocation of group IDs (i.e. 10-362) following the scheme used in the GBR Features dataset. Using R scripting. - Allocation of subgroup IDs (10-362b) to each feature in the dataset. Using R scripting. 8. Allocation of names: - Names of features were copied from some existing maps (Nautical Charts, 250k, 100k Topographic maps, CSIRO Torres Strait Atlas). For more information about the methods used in the development of this dataset see the associated technical report (Lawrey, E. P., Stewart M., 2016)

Limitations:

This dataset has mapped features from remote sensing and thus in some parts of Torres Strait where it is very turbid this may result in an underestimate of boundary of features. It also means that some features may be missing from the dataset.

This dataset is NOT SUITABLE FOR NAVIGATION.

The classification of features in this dataset was determined from remote sensing and not in-situ surveys. Each feature has a confidence rating associated with this classification. Features with a 'Low' confidence should be considered only as guidance.

This project only digitised reefs in Torres Strait, no modifications were made to the features from the integrated GBR Features dataset.

Format:

This dataset is available as a shapefile, a set of associated A1 preview maps of the Torres Strait region, ArcMap MXD file with map styling and ArcMap map layer file. The shapefile is also available in KMZ format suitable for viewing in Google Earth. TS_AIMS_NESP_Torres_Strait_Features_V1b_with_GBR_Features.shp (26 MB), TS_AIMS_NESP_Torres_Strait_Features_V1b_with_GBR_Features.kmz: Torres Strait features (3927 polygon features) integrated with the (GBRMPA) GBR Features dataset (5685 polygon features). This dataset covers the entire GBR.

Data Dictionary:

• DATASET: (TS Features, GBR Features) Which dataset this feature belongs to. This attribute is used when the Torres Strait Reef and Island Features dataset is merged with the GBRMPA GBR Features dataset.
• LOC_NAME_S: (e.g. Tobin (Zagarsum) Island (10-147a)) Location Name: Name of the feature and its ID
• GBR_NAME: (e.g. Tobin (Zagarsum) Island) Name of the features with no ID
• CHART_NAME: (e.g. Tobin Island) Name of the feature on the Australian Nautical Charts
• TRAD_NAME: (Zagarsum) Traditional name. From various sources.
• UN_FEATURE: (TRUE, FALSE) Unnamed Feature: If TRUE then the feature is unnamed. Useful for limiting labels in maps to features with names.
• LABEL_ID: (10-147a) ID of the feature
• SORT_GBR_I: (10147) ID of each feature cluster made up from the Latitude ID and Group ID. Used for sorting the features.
• FEAT_NAME: (Island, Rock, Reef, Cay, Mainland, Bank, Terrestrial Reef, Other ) Classification of the feature that is used in the GBR Features dataset. See 3.6 Classification scheme for more information.
• LEVEL_1, LEVEL_2, LEVEL_3: Hierarchical classification of the features. See Appendix 3: Feature Classification Descriptions.
• Checked: (TRUE, FALSE) Flag to record if the feature was reviewed in detail (at a scale of approximately 1:5000) after the initial digitisation. Unchecked features were only reviewed at a coarser scale (1:25000) to spot significant problems.
• IMG_SOURCE: (Aerial, AGRI, Landsat, ESRI) Imagery type used for the final digitisation checking and correction. (AGRI - AGRI PRISM by GA, Landsat is Landsat 8 or Landsat 5, ESRI - ArcMap satellite basemap)
• CLASS_SRC: (Aerial, AGRI, Landsat, Google, Marine Chart) Imagery type used to determine the classification of the feature. Often the classification will be an aggregation of information from multiple image sources. This field will record the highest resolution source used. For some small features the classification was obtained from the Marine Chart, generally for Rocky Reefs.
• CLASS_CONF: (High, Medium, Low) Confidence of the classification applied to the feature. The confidence is dependent on the clarity and range of the imagery available for classification. High - Clear high resolution imagery available (Aerial, Google) with good water visibility. Key characteristics of the classification clear visible. Feature classification fits the context for the neighbouring region. For unconsolidated features (such as sand banks) a High confidence classification would be applied if the shape, colour and context fit and in particular if movement is visible over time-lapse Landsat imagery. Medium - Moderate imagery available (Landsat 8 pan sharpened, some high resolution imagery) that shows key characteristics of the feature and the classification fits the context for the neighbouring region. Low - Only Landsat 5 imagery is available, the feature is small and its origin is unclear from the neighbouring context. This is the default confidence rating for any features that were not individually checked.
• POLY_ORIG: (QLD_DNRM_Coastline_25k, New, GBR_Features, AU_GA_Coast100k_2004) Original source of the polygon prior to any modifications. New features correspond to all the mapped marine features. Most features from the other source would have been modified as part of the checking and trimming of the dataset.
• SUB_NO: (100, 101, ¿) Subgroup number. Numeric count, starting at 100 of each feature in a group. Matches the subgroup ID i.e. 100 -> blank, 101 -> a, 102 -> b, etc.
• CODE: (e.g. 10-147-102-101) Unique code made from the various IDs. This is a GBR Feature attribute.
• UNIQUE_ID: (10147102101) Same as the CODE but without the hyphens, This is a GBR Feature attribute. Note: Version 1b, this attribution is currently out of date.
• FEATURE_C: (100 - 110) Code applied to each of the FEAT_NAMEs.
• QLD_NAME: (Tobin Island) Same as the GBR_NAME
• X_COORD: Longitude in decimal degrees east, in GDA94.
• Y_COORD: Latitude in decimal degrees north, in GDA94.
• SHAPE_AREA: Shape Area in km2
• SHAPE_LEN: Shape perimeter length in km
• CHECKED: (TRUE, FALSE) Whether the features was carefully checked (at a scale of better than ~1:5000) and manually corrected to this level of precision. If FALSE then the feature was only checked to approximately a1:25000 scale.
• PriorityLn: (TRUE, FALSE) Priority Label - If TRUE then this feature's label should be included in a map. Usually correspond to features with names. Use to reduce near duplicate labels of the islands and their surrounding fringing reefs.
• COUNTRY: (Australia, Papua-New Guinea) Sovereignty of the feature. This is based on a spatial join with the Australian Maritime Boundaries 2014a. The Territorial Sea and the Exclusive Economic

This dataset contains cloud free, low tide composite satellite images for the tropical Australia region based on 10 m resolution Sentinel 2 imagery from 2018 – 2023. This image collection was created as part of the NESP MaC 3.17 project and is intended to allow mapping of the reef features in tropical Australia.

This collection contains composite imagery for 200 Sentinel 2 tiles around the tropical Australian coast. This dataset uses two styles:
1. a true colour contrast and colour enhancement style (TrueColour) using the bands B2 (blue), B3 (green), and B4 (red)
2. a near infrared false colour style (Shallow) using the bands B5 (red edge), B8 (near infrared), and B12 (short wave infrared).
These styles are useful for identifying shallow features along the coastline.

The Shallow false colour styling is optimised for viewing the first 3 m of the water column, providing an indication of water depth. This is because the different far red and near infrared bands used in this styling have limited penetration of the water column. In clear waters the maximum penetrations of each of the bands is 3-5 m for B5, 0.5 - 1 m for B8 and < 0.05 m for B12. As a result, the image changes in colour with the depth of the water with the following colours indicating the following different depths:
- White, brown, bright green, red, light blue: dry land
- Grey brown: damp intertidal sediment
- Turquoise: 0.05 - 0.5 m of water
- Blue: 0.5 - 3 m of water
- Black: Deeper than 3 m
In very turbid areas the visible limit will be slightly reduced.


Change log:

This dataset will be progressively improved and made available for download. These additions will be noted in this change log.
2024-07-24 - Add tiles for the Great Barrier Reef
2024-05-22 - Initial release for low-tide composites using 30th percentile (Git tag: "low_tide_composites_v1")


Methods:

The satellite image composites were created by combining multiple Sentinel 2 images using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile filter the "COPERNICUS/S2_HARMONIZED" image collection by
- tile ID
- maximum cloud cover 0.1%
- date between '2018-01-01' and '2023-12-31'
- asset_size > 100000000 (remove small fragments of tiles)
2. Remove high sun-glint images (see "High sun-glint image detection" for more information).
3. Split images by "SENSING_ORBIT_NUMBER" (see "Using SENSING_ORBIT_NUMBER for a more balanced composite" for more information).
4. Iterate over all images in the split collections to predict the tide elevation for each image from the image timestamp (see "Tide prediction" for more information).
5. Remove images where tide elevation is above mean sea level to make sure no high tide images are included.
6. Select the 10 images with the lowest tide elevation.
7. Combine SENSING_ORBIT_NUMBER collections into one image collection.
8. Remove sun-glint (true colour only) and apply atmospheric correction on each image (see "Sun-glint removal and atmospheric correction" for more information).
9. Duplicate image collection to first create a composite image without cloud masking and using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
10. Apply cloud masking to all images in the original image collection (see "Cloud Masking" for more information) and create a composite by using the 30th percentile of the images in the collection (i.e. for each pixel the 30th percentile value of all images is used).
11. Combine the two composite images (no cloud mask composite and cloud mask composite). This solves the problem of some coral cays and islands being misinterpreted as clouds and therefore creating holes in the composite image. These holes are "plugged" with the underlying composite without cloud masking. (Lawrey et al. 2022)
12. The final composite was exported as cloud optimized 8 bit GeoTIFF

Note: The following tiles were generated with different settings as they did not have enough images to create a composite with the standard settings:
- 51KWA: no high sun-glint filter
- 54LXP: maximum cloud cover set to 1%
- 54LXP: maximum cloud cover set to 1%
- 54LYK: maximum cloud cover set to 2%
- 54LYM: maximum cloud cover set to 5%
- 54LYN: maximum cloud cover set to 1%
- 54LYQ: maximum cloud cover set to 5%
- 54LYP: maximum cloud cover set to 1%
- 54LZL: maximum cloud cover set to 1%
- 54LZM: maximum cloud cover set to 1%
- 54LZN: maximum cloud cover set to 1%
- 54LZQ: maximum cloud cover set to 5%
- 54LZP: maximum cloud cover set to 1%
- 55LBD: maximum cloud cover set to 2%
- 55LBE: maximum cloud cover set to 1%
- 55LCC: maximum cloud cover set to 5%
- 55LCD: maximum cloud cover set to 1%


High sun-glint image detection:

Images with high sun-glint can lead to lower quality composite images. To determine high sun-glint images, a mask is created for all pixels above a high reflectance threshold for the near-infrared and short-wave infrared bands. Then the proportion of this is calculated and compared against a sun-glint threshold. If the image exceeds this threshold, it is filtered out of the image collection. As we are only interested in the sun-glint on water pixels, a water mask is created using NDWI before creating the sun-glint mask.


Sun-glint removal and atmospheric correction:

Sun-glint was removed from the images using the infrared B8 band to estimate the reflection off the water from the sun-glint. B8 penetrates water less than 0.5 m and so in water areas it only detects reflections off the surface of the water. The sun-glint detected by B8 correlates very highly with the sun-glint experienced by the visible channels (B2, B3 and B4) and so the sun-glint in these channels can be removed by subtracting B8 from these channels.

Eric Lawrey developed this algorithm by fine tuning the value of the scaling between the B8 channel and each individual visible channel (B2, B3 and B4) so that the maximum level of sun-glint would be removed. This work was based on a representative set of images, trying to determine a set of values that represent a good compromise across different water surface conditions.

This algorithm is an adjustment of the algorithm already used in Lawrey et al. 2022


Tide prediction:

To determine the tide elevation in a specific satellite image, we used a tide prediction model to predict the tide elevation for the image timestamp. After investigating and comparing a number of models, it was decided to use the empirical ocean tide model EOT20 (Hart-Davis et al., 2021). The model data can be freely accessed at https://doi.org/10.17882/79489 and works with the Python library pyTMD (https://github.com/tsutterley/pyTMD). In our comparison we found this model was able to predict accurately the tide elevation across multiple points along the study coastline when compared to historic Bureau of Meteorolgy and AusTide data. To determine the tide elevation of the satellite images we manually created a point dataset where we placed a central point on the water for each Sentinel tile in the study area . We used these points as centroids in the ocean models and calculated the tide elevation from the image timestamp.


Using "SENSING_ORBIT_NUMBER" for a more balanced composite:

Some of the Sentinel 2 tiles are made up of different sections depending on the "SENSING_ORBIT_NUMBER". For example, a tile could have a small triangle on the left side and a bigger section on the right side. If we filter an image collection and use a subset to create a composite, we could end up with a high number of images for one section (e.g. the left side triangle) and only few images for the other section(s). This would result in a composite image with a balanced section and other sections with a very low input. To avoid this issue, the initial unfiltered image collection is divided into multiple image collections by using the image property "SENSING_ORBIT_NUMBER". The filtering and limiting (max number of images in collection) is then performed on each "SENSING_ORBIT_NUMBER" image collection and finally, they are combined back into one image collection to generate the final composite.


Cloud Masking:

Each image was processed to mask out clouds and their shadows before creating the composite image.
The cloud masking uses the COPERNICUS/S2_CLOUD_PROBABILITY dataset developed by SentinelHub (Google, n.d.; Zupanc, 2017). The mask includes the cloud areas, plus a mask to remove cloud shadows. The cloud shadows were estimated by projecting the cloud mask in the direction opposite the angle to the sun. The shadow distance was estimated in two parts.

A low cloud mask was created based on the assumption that small clouds have a small shadow distance. These were detected using a 35% cloud probability threshold. These were projected over 400 m, followed by a 150 m buffer to expand the final mask.

A high cloud mask was created to cover longer shadows created by taller, larger clouds. These clouds were detected based on an 80% cloud probability threshold, followed by an erosion and dilation of 300 m to remove small clouds. These were then projected over a 1.5 km distance followed by a 300 m buffer.

The parameters for the cloud masking (probability threshold, projection distance and buffer radius) were determined through trial and error on a small number of scenes. As such there are probably significant potential improvements that could be made to this algorithm.

Erosion, dilation and buffer operations were performed at a lower image resolution than the native satellite image resolution

LandCoverNet is a global annual land cover classification training dataset with labels for the multi-spectral satellite imagery from Sentinel-1, Sentinel-2 and Landsat-8 missions in 2018. LandCoverNet Australia contains data across Australia, which accounts for ~7% of the global dataset. Each pixel is identified as one of the seven land cover classes based on its annual time series. These classes are water, natural bare ground, artificial bare ground, woody vegetation, cultivated vegetation, (semi) natural vegetation, and permanent snow/ice.
There are a total of 600 image chips of 256 x 256 pixels in LandCoverNet Australia V1.0 spanning 20 tiles. Each image chip contains temporal observations from the following satellite products with an annual class label, all stored in raster format (GeoTIFF files):
* Sentinel-1 ground range distance (GRD) with radiometric calibration and orthorectification at 10m spatial resolution
* Sentinel-2 surface reflectance product (L2A) at 10m spatial resolution
* Landsat-8 surface reflectance product from Collection 2 Level-2

Radiant Earth Foundation designed and generated this dataset with a grant from Schmidt Futures with additional support from NASA ACCESS, Microsoft AI for Earth and in kind technology support from Sinergise.

The size of the ANZ Satellite Imagery Services market was valued at USD XXX Million in 2023 and is projected to reach USD XXX Million by 2032, with an expected CAGR of 11.61% during the forecast period.ANZ satellite imagery services are the use of space technology to capture images over high resolution of the Earth's surface, mainly for the Australia and New Zealand region. Such imagery offers information in sectors such as agriculture, mining, urban planning, environment monitoring, and disaster management.Crop health monitoring, mining site evaluation, urban development observation, detection of deforestation and pollution, among others, can now be made right by satellite imagery analysis. For instance, farmers would consider satellite data to assess best irrigation and application of fertilizers practices; companies that engage in mining can assess the mine site and look out for possible negative environmental impacts. Urban growth patterns and infrastructure demands may be identified and designed through the use of satellite imagery by urban planners, and these agencies might observe the changes in ecosystems and early signs of climate change. Under disaster response, satellite imagery plays a very important role in making fast appraisals of damage and the allocation of assets and supplies to affected areas.Much satellite imagery use opportunity is offered by the diverse land shapes of the ANZ region. The demand for an ANZ satellite imagery service will advance in a perspective of continually innovating and developing from all sectors through the development of this technology with more resolution and images. Recent developments include: May 2023: Arlula announced the successful completion of its AUD 2.2 million (USD 1.5 million) initial investment round. This significant investment will enable the firm to increase access to Earth Observation (EO) data and imagery, helping people, small companies, and big corporations entirely use space-based data. This significant support demonstrates an interest in and acceptance of Arlula's aim to transform access to EO data and change businesses utilizing this priceless resource., May 2023: SouthPAN partners Geoscience Australia and Toit Te Whenua Land Information New Zealand have signed a contract with Inmarsat Australia for the new service on one of Inmarsat's three new I-8 satellites, bringing the southern hemisphere one step closer to first-rate satellite positioning. It can assist in preventing fatalities by offering precise safety tracking, increasing farm productivity through automated device tracking, or even supporting upcoming transport management systems.. Key drivers for this market are: Government Initiatives and Investments to Support the Market Growth, Advancements in satellite technology, including High-resolution imagery, multispectral data, SAR, etc.. Potential restraints include: Affordability and Accessibility might restrain the Market Growth, Competition from Alternative Technologies such as Aerial Drones, LiDAR, and UAVs. Notable trends are: Government Initiatives and Investments to Support the Market Growth.

This dataset consists of daily estimates of photic depth on the Great Barrier Reef from MODIS satellite imagery (from 2002 - 2015) using a quasi-analytical algorithm. This algorithm is based on a …Show full descriptionThis dataset consists of daily estimates of photic depth on the Great Barrier Reef from MODIS satellite imagery (from 2002 - 2015) using a quasi-analytical algorithm. This algorithm is based on a Type II linear regression of log-transformed satellite and in situ data (2002- 2012). This algorithm was developed as part of data delivery for several NERP projects and was implemented into the NASA SeaDAS tool for processing MODIS imagery. This algorithm and its data products are now routinely run by the Bureau of Meteorology as part of the eReefs Water Quality Dashboard. The data produced from this algorithm were key input datasets for the analysis of NERP TE project 4.1 and integrated as part of the NERP TE 2.3 GBR/TS environmental conditions reports. Method: The satellite imagery was first broken down into its estimated Inherent Optical Properties (IOP) using a quasi-analytical algorithm, outlined in [1]. This process converts the multi-spectral satellite images into an estimate of the various optical properties of the water such as backscattering and absorption of the water. The IOPs were then used to estimate the depth where 10% of the surface light (PAR) level was still available (Z10%). A regression of the in situ ZSD (secchi depth) values against the matching satellite estimates of Z10% was used to adjust the satellite-derived Z10% to ZSD. A Type II linear regression (RMA) of log-transformed satellite and in situ data was used to estimate ZSD for the GBR according to: ZSD = 10^[{log10 (Z10%) - a0}/a1] where a0 and a1 are 0.529 and 0.816 for MODIS-Aqua (N = 71; r2 = 0.83; RMSE = 0.096). The regional tuning parameters a0 and a1 were determined by regression between satellite and in-situ secchi depth measurements from AIMS and QDPI. More details about the methods used to create this dataset can be found in [2]. Format: This data is available in NetCDF raster format from the BOM Marine Water Quality THREDDS server. This server also makes the data available various formats from the following services: OpenDAP, WMS and WCS. http://ereeftds.bom.gov.au/ereefs/tds/catalogs/ereefs_data.html The data for this dataset is available in the mwq P1D Aggregation, mwq P1W Aggregation, mwq P1M Aggregation, mwq P6M Aggregation, mwq P1A Aggregation service end points. These correspond to daily and weekly, monthly, 6 monthly and annual aggregates respectively. The secchi depth estimates correspond to the SD_MIM_* data layers in the service end points. References: Lee, Z.; Carder, K.L.; Arnone, R.A. Deriving inherent optical properties from water color: A multiband quasi-analytical algorithm for optically deep waters. Appl. Opt. 2002, 41, 5755¿5772. Weeks, S.; Werdell, P.J.; Schaffelke, B.; Canto, M.; Lee, Z.; Wilding, J.G.; Feldman, G.C. Satellite-Derived Photic Depth on the Great Barrier Reef: Spatio-Temporal Patterns of Water Clarity. Remote Sens. 2012, 4, 3781-3795.

Abstract The Catchment Scale Land Use of Australia – Update December 2023 dataset is the national compilation of catchment scale land use data available for Australia (CLUM), as of December 2023. It replaces the Catchment Scale Land Use of Australia – Update December 2020. It is a seamless raster dataset that combines land use data for all state and territory jurisdictions, compiled at a resolution of 50 metres by 50 metres. The CLUM data shows a single dominant land use for a given area, based on the primary management objective of the land manager (as identified by state and territory agencies). Land use is classified according to the Australian Land Use and Management Classification version 8. It has been compiled from vector land use datasets collected as part of state and territory mapping programs and other authoritative sources, through the Australian Collaborative Land Use and Management Program. Catchment scale land use data was produced by combining land tenure and other types of land use information including, fine-scale satellite data, ancillary datasets, and information collected in the field. The date of mapping (2008 to 2023) and scale of mapping (1:5,000 to 1:250,000) vary, reflecting the source data, capture date and scale. Date and scale of mapping are provided in supporting datasets.

Currency Date modified: December 2023 Publication Date: June 2024 Modification frequency: As needed (approximately annual) Data Extent Coordinate reference: WGS84 / Mercator Auxiliary Sphere Spatial Extent North: -9.995 South: -44.005 East: 154.004 West: 112.505 Source information Data, Metadata, Maps and Interactive views are available from Catchment Scale Land Use of Australia - Update 2023 Catchment Scale Land Use of Australia - Update 2023 – Descriptive metadata The data was obtained from Department of Agriculture, Fisheries and Forestry - Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES). ABARES is providing this data to the public under a Creative Commons Attribution 4.0 license. Lineage statement This catchment scale land use dataset provides the latest compilation of land use mapping information for Australia’s regions as at December 2023. It is used by the Department of Agriculture, Fisheries and Forestry, state agencies and regional natural resource management groups to address issues such as agricultural productivity and sustainability, biodiversity conservation, biosecurity, land use planning, natural disaster management and natural resource monitoring and investment. The data vary in date of mapping (2008 to 2023) and scale (1:5,000 to 1:250,000). 2023 updates include more current data and/or reclassification of existing data. The following areas have updated data since the December 2020 version:

New South Wales (2017 v1.5 from v1.2). Northern Territory (2022 from 2020). Tasmania (2021 from 2019). Victoria (2021 from 2017). Data were also added from the Great Barrier Reef Natural Resource Management (NRM) regions in Queensland (2021 from a variety of dates 2009 to 2017). the Australian Tree Crops. Australian Protected Cropping Structures and Queensland Soybean Crops maps as downloaded on 30 November 2023. The capital city of Adelaide was updated using 2021 mesh block information from the Australian Bureau of Statistics. Minor reclassifications were made for Western Australia and mining area within mining tenements more accurately delineated in South Australia.

Links to land use mapping datasets and metadata are available at the ACLUMP data download page at agriculture.gov.au. State and territory vector catchment scale land use data were produced by combining land tenure and other types of land use information, fine-scale satellite data and information collected in the field, as outlined in 'Guidelines for land use mapping in Australia: principles, procedures and definitions, 4th edition' (ABARES 2011). The Northern Territory, Queensland, South Australia, Tasmania, Victoria and Western Australia were mapped to version 8 of the ALUM classification (‘The Australian Land Use and Management Classification Version 8’, ABARES 2016). The Australian Capital Territory was mapped to version 7 of the ALUM classification and converted to version 8 using a look-up table based on Appendix 1 of ABARES (2016). Purpose for which the material was obtained: This catchment scale land use dataset provides the latest compilation of land use mapping information for Australia’s regions as at December 2023. It is used by the Department of Agriculture, Fisheries and Forestry, state agencies and regional natural resource management groups to address issues such as agricultural productivity and sustainability, biodiversity conservation, biosecurity, land use planning, natural disaster management and natural resource monitoring and investment. The data vary in date of mapping (2008 to 2023) and scale (1:5,000 to 1:250,000). Do not use this data to:

Derive national statistics. The Land use of Australia data series should be used for this purpose. Calculate land use change. The Land use of Australia data series should be used for this purpose.

It is not possible to calculate land use change statistics between annual CLUM national compilations as not all regions are updated each year; land use mapping methodologies, precision, accuracy and source data and satellite imagery have improved over the years; and the land use classification has changed over time. It is only possible to calculate change when earlier land use datasets have been revised and corrected to ensure that changes detected are real change and not an artefact of the mapping process. Note: The Digital Atlas of Australia downloaded and created a copy of the source data in October 2024 that was suitable to be hosted through ArcGIS Image Server & Image Dedicated. A copy of the raster was created with RGB fields as a colour map with Geoprocessing tools in ArcPro. Note: The Digital Atlas of Australia downloaded and created a copy of the source data in February 2025 that was suitable to be hosted through ArcGIS Image Server & Image Dedicated. A copy of the raster dataset was created with RGB fields as a colour map with Geoprocessing tools in ArcPro, and the raster dataset was re-projected from 1994 Australia Albers to WGS 1984 Web Mercator (Auxiliary Sphere). Data dictionary

Field name DField description Code values

OID Internal feature number that uniquely identifies each row Integer

Service Pixel value (Date) The year for which land use was mapped in the vector data provided by state and territory agencies or others, Date Range: 2008 to 2023 Integer

Count Count of the number of raster cells in each class of VALUE Integer

Label Reflecting the Date of the source data ranges from 2008 to 2023 Text

Contact Department of Agriculture, Fisheries and Forestry (ABARES), info.ABARES@aff.gov.au

Abstract

This dataset and its metadata statement were supplied to the Bioregional Assessment Programme by a third party and are presented here as originally supplied.

Water Observations from Space (WOfS) is a gridded dataset indicating areas where surface water has been observed using the Geoscience Australia (GA) Earth observation satellite data holdings. The WOfS product Version 1.0 includes observations taken between 1998 and 2012 (inclusive) from the Landsat 5 and 7 satellites. WOfS version 1.5 includes observations from 1987 to March 2014. Future versions of the product will extend the temporal range and diversify the data sources. WOfS covers all of mainland Australia and Tasmania but excludes off-shore Territories.

Dataset History

Water Observations from Space (WOfS) is derived from Landsat-5 and Landsat-7 satellite imagery acquired over Australia between 1987 and 2014. The Landsat data underpinning WOfS is ARG25 standard data located in the Australian Geoscience Data Cube (AGDC) at the National Computational Infrastructure (NCI) in the Australian National University (ANU), Canberra. The WOfS product is calculated from all acceptable Landsat scenes in the Geoscience Australia archive for the time period. The detection process is based on spectral analysis of each pixel in each Landsat scene. The water detecton algorithm used to detect water from each observed pixel is based on a statistical regression tree analysis of a set of normalised difference indices and corrected band values. The regression is based on a set of water and non-water samples created by visual interpretation of 20 Landsat scenes from across Australia. The sample locations, ensure that the logistic regression is based on the full geographic range of conditions experienced in Australia. The regression analysis determined a set of best indices and bands for the analysis and the associated thresholds in each component to derive a final classification tree, producing a water/non-water classification for every pixel in the Data Cube. The final water classification for each pixel is modified by Pixel Quality (see associated RG25 - PQ product information) and terrain. Once the water algorithm has completed its process, the water detection for a pixel through time is combined to produce a total number of water observations for each pixel. This is compared to a total number of clear observations for the same pixel, derived from the PQ analysis. The ratio is expressed as a percentage water recurrence. A separate analysis produces a confidence dataset, providing an assessment on whether a pixel depicted as having had water detected at some time is likely. The layer is computed by combining a set of confidence factors using a weighted sum approach, with the weightings derived by logistic regression. The confidence factors are: 1. MrVBF, a multi-resolution valley bottom flatness product (Gallant et al., 2012) derived from SRTM as part of the Terrestrial Ecosystems Research Network. Surface water pixels identified in valley bottoms were more likely to be positively detected. 2. Slope derived from SRTM Digital Surface Models. Water pixels on a slope were considered less plausible than those on a flat surface. 3. MODIS Open Water Likelihood (OWL) (Ticehurst et al, 2010) provides a plausibility based an independent water detection algorithm employing the MODIS sensor. If both detection algorithms agree on the presence of a surface water pixel, there is a greater plausibility that the detection is correct. 4. Australian Hydrological Geospatial Fabric (Geofabric) is a GIS of hydrological features derived from manually interpreted topographic map grids. If known hydrologic features (pixels) from GeoFabric coincide with detected water pixels, the plausibility of detection is greater. 5. P, the number of observations of water as a fraction of the number of clear observations of the target pixel. P is high for more permanent water bodies. 6. Built-Up areas indicating areas of dense urban development. In such areas the water detection algorithm struggles to cope with the deep shadows cast by multi-story buildings and the generally noisy spectral response created by structures. The Built-Up layer is derived from the Ausralian Bureau of Statistics ASGS 2011 dataset, for urban centres of populations of 100 000 and over. The product creation workflow is as follows: 1. Landsat raw data capture and storage 2. Data pre-processing (ARG25 and PQ products) 3. Water detection 4. Pixel Quality filtering 5. Data product storage and delivery 6. Time series data preparation 7. Summary and extent data preparation 8. Application of Confidence information 9. WMS/WCS service delivery

Dataset Citation

Geoscience Australia (2015) Australian Water Observations from Space (WOfS) - Water Summary, Filtered. Bioregional Assessment Source Dataset. Viewed 05 July 2017, http://data.bioregionalassessments.gov.au/dataset/719a5433-2af0-4601-8036-a03f77199442.

Abstract Digital Earth Australia Coastlines is a continental dataset that includes annual shorelines and rates of coastal change along the entire Australian coastline from 1988 to the present. The product combines satellite data from Geoscience Australia's Digital Earth Australia program with tidal modelling to map the most representative location of the shoreline at mean sea level for each year. The product enables trends of coastal retreat and growth to be examined annually at both a local and continental scale, and for patterns of coastal change to be mapped historically and updated regularly as data continues to be acquired. This allows current rates of coastal change to be compared with that observed in previous years or decades. The ability to map shoreline positions for each year provides valuable insights into whether changes to our coastline are the result of particular events or actions, or a process of more gradual change over time. This information can enable scientists, managers and policy makers to assess impacts from the range of drivers impacting our coastlines and potentially assist planning and forecasting for future scenarios. The DEA Coastlines product contains five layers:

Annual shorelines Rates of change points Coastal change hotspots (1 km) Coastal change hotspots (5 km) Coastal change hotspots (10 km)

Annual shorelines Annual shoreline vectors that represent the median or ‘most representative’ position of the shoreline at approximately 0 m Above Mean Sea Level for each year since 1988. Dashed shorelines have low certainty. Rates of change points A point dataset providing robust rates of coastal change for every 30 m along Australia’s non-rocky coastlines. The most recent annual shoreline is used as a baseline for measuring rates of change. Points are shown for locations with statistically significant rates of change (p-value <= 0.01; see sig_time below) and good quality data (certainty = "good"; see certainty below) only. Each point shows annual rates of change (in metres per year; see rate_time below), and an estimate of uncertainty in brackets (95% confidence interval; see se_time). For example, there is a 95% chance that a point with a label -10.0 m (±1.0 m) is retreating at a rate of between -9.0 and -11.0 metres per year. Coastal change hotspots (1 km, 5 km, 10 km) Three points layers summarising coastal change within moving 1 km, 5 km and 10km windows along the coastline. These layers are useful for visualising regional or continental-scale patterns of coastal change. Currency Date modified: August 2023 Modification frequency: Annually Data extent Spatial extent North: -9° South: -44° East: 154° West: 112° Temporal extent From 1988 to Present Source information

Product description and metadata Digital Earth Australia Coastlines catalog entry Data download Interactive Map

Lineage statement The DEA Coastlines product is under active development. A full and current product description is best sourced from the DEA Coastlines website. For a full summary of changes made in previous versions, refer to Github. Data dictionary Layer attribute columns Annual shorelines

Attribute name Description

OBJECTID Automatically generated system ID

year The year of each annual shoreline

certainty A column providing important data quality flags for each annual shoreline (see the Quality assurance section of the product description and metadata page for more detail about each data quality flag)

tide_datum The tide datum of each annual shoreline (e.g. "0 m AMSL")

id_primary The name of the annual shoreline's Primary sediment compartment from the Australian Coastal Sediment Compartments framework

Rates of change points and Coastal change hotspots

Attribute name Description

OBJECTID Automatically generated system ID

uid A unique geohash identifier for each point

rate_time Annual rates of change (in metres per year) calculated by linearly regressing annual shoreline distances against time (excluding outliers). Negative values indicate retreat and positive values indicate growth

sig_time Significance (p-value) of the linear relationship between annual shoreline distances and time. Small values (e.g. p-value < 0.01 or 0.05) may indicate a coastline is undergoing consistent coastal change through time

se-time Standard error (in metres) of the linear relationship between annual shoreline distances and time. This can be used to generate confidence intervals around the rate of change given by rate_time (e.g. 95% confidence interval = se_time * 1.96).

outl_time Individual annual shoreline are noisy estimators of coastline position that can be influenced by environmental conditions (e.g. clouds, breaking waves, sea spray) or modelling issues (e.g. poor tidal modelling results or limited clear satellite observations). To obtain reliable rates of change, outlier shorelines are excluded using a robust Median Absolute Deviation outlier detection algorithm, and recorded in this column

dist_1990, dist_1991, etc Annual shoreline distances (in metres) relative to the most recent baseline shoreline. Negative values indicate that an annual shoreline was located inland of the baseline shoreline. By definition, the most recent baseline column will always have a distance of 0 m

angle_mean, angle_std The mean angle and standard deviation between the baseline point to all annual shorelines. This data is used to calculate how well shorelines fall along a consistent line; high angular standard deviation indicates that derived rates of change are unlikely to be correct

valid_obs, valid_span The total number of valid (i.e. non-outliers, non-missing) annual shoreline observations, and the maximum number of years between the first and last valid annual shoreline

sce Shoreline Change Envelope (SCE). A measure of the maximum change or variability across all annual shorelines, calculated by computing the maximum distance between any two annual shorelines (excluding outliers). This statistic excludes sub-annual shoreline variability like tides, storms and seasonal effects

nsm Net Shoreline Movement (NSM). The distance between the oldest (1988) and most recent annual shoreline (excluding outliers). Negative values indicate the coastline retreated between the oldest and most recent shoreline; positive values indicate growth. This statistic does not reflect sub-annual shoreline variability, so will underestimate the full extent of variability at any given location

max_year, min_year The year that annual shorelines were at their maximum (i.e. located furthest towards the ocean) and their minimum (i.e. located furthest inland) respectively (excluding outliers). This statistic excludes sub-annual shoreline variability

certainty A column providing important data quality flags for each annual shoreline (see the Quality assurance section of the product description and metadata page for more detail about each data quality flag)

id_primary The name of the point's Primary sediment compartment from the Australian Coastal Sediment Compartments framework

Contact Geoscience Australia, clientservices@ga.gov.au

Abstract The Catchment Scale Land Use of Australia – Update December 2023 dataset is the national compilation of catchment scale land use data available for Australia (CLUM), as of December 2023. It replaces the Catchment Scale Land Use of Australia – Update December 2020. It is a seamless raster dataset that combines land use data for all state and territory jurisdictions, compiled at a resolution of 50 metres by 50 metres. The CLUM data shows a single dominant land use for a given area, based on the primary management objective of the land manager (as identified by state and territory agencies). Land use is classified according to the Australian Land Use and Management Classification version 8. It has been compiled from vector land use datasets collected as part of state and territory mapping programs and other authoritative sources, through the Australian Collaborative Land Use and Management Program. Catchment scale land use data was produced by combining land tenure and other types of land use information including, fine-scale satellite data, ancillary datasets, and information collected in the field. The date of mapping (2008 to 2023) and scale of mapping (1:5,000 to 1:250,000) vary, reflecting the source data, capture date and scale. Date and scale of mapping are provided in supporting datasets.

Currency Date modified: December 2023 Date Published: June 2024 Modification frequency: As needed (approximately annual) Data Extent Coordinate reference: WGS84 / Mercator Auxiliary Sphere Spatial Extent North: -9.995 South: -44.005 East: 154.004 West: 112.505 Source information Data, Metadata, Maps and Interactive views are available from Catchment Scale Land Use of Australia - Update 2023 Catchment Scale Land Use of Australia - Update 2023 – Descriptive metadata The data was obtained from Department of Agriculture, Fisheries and Forestry - Australian Bureau of Agricultural and Resource Economics and Sciences (ABARES). ABARES is providing this data to the public under a Creative Commons Attribution 4.0 license. Lineage Statement This catchment scale land use dataset provides the latest compilation of land use mapping information for Australia’s regions as at December 2023. It is used by the Department of Agriculture, Fisheries and Forestry, state agencies and regional natural resource management groups to address issues such as agricultural productivity and sustainability, biodiversity conservation, biosecurity, land use planning, natural disaster management and natural resource monitoring and investment. The data vary in date of mapping (2008 to 2023) and scale (1:5,000 to 1:250,000). 2023 updates include more current data and/or reclassification of existing data. The following areas have updated data since the December 2020 version:

New South Wales (2017 v1.5 from v1.2). Northern Territory (2022 from 2020). Tasmania (2021 from 2019). Victoria (2021 from 2017). Data were also added from the Great Barrier Reef Natural Resource Management (NRM) regions in Queensland (2021 from a variety of dates 2009 to 2017). the Australian Tree Crops. Australian Protected Cropping Structures and Queensland Soybean Crops maps as downloaded on 30 November 2023. The capital city of Adelaide was updated using 2021 mesh block information from the Australian Bureau of Statistics. Minor reclassifications were made for Western Australia and mining area within mining tenements more accurately delineated in South Australia.

Links to land use mapping datasets and metadata are available at the ACLUMP data download page at agriculture.gov.au. State and territory vector catchment scale land use data were produced by combining land tenure and other types of land use information, fine-scale satellite data and information collected in the field, as outlined in 'Guidelines for land use mapping in Australia: principles, procedures and definitions, 4th edition' (ABARES 2011). The Northern Territory, Queensland, South Australia, Tasmania, Victoria and Western Australia were mapped to version 8 of the ALUM classification (‘The Australian Land Use and Management Classification Version 8’, ABARES 2016). The Australian Capital Territory was mapped to version 7 of the ALUM classification and converted to version 8 using a look-up table based on Appendix 1 of ABARES (2016). Purpose for which the material was obtained: This catchment scale land use dataset provides the latest compilation of land use mapping information for Australia’s regions as at December 2023. It is used by the Department of Agriculture, Fisheries and Forestry, state agencies and regional natural resource management groups to address issues such as agricultural productivity and sustainability, biodiversity conservation, biosecurity, land use planning, natural disaster management and natural resource monitoring and investment. The data vary in date of mapping (2008 to 2023) and scale (1:5,000 to 1:250,000). Do not use this data to:

Derive national statistics. The Land use of Australia data series should be used for this purpose. Calculate land use change. The Land use of Australia data series should be used for this purpose.

It is not possible to calculate land use change statistics between annual CLUM national compilations as not all regions are updated each year; land use mapping methodologies, precision, accuracy and source data and satellite imagery have improved over the years; and the land use classification has changed over time. It is only possible to calculate change when earlier land use datasets have been revised and corrected to ensure that changes detected are real change and not an artefact of the mapping process. Note: The Digital Atlas of Australia downloaded and created a copy of the source data in October 2024 that was suitable to be hosted through ArcGIS Image Server & Image Dedicated. A copy of the raster was created with RGB fields as a colour map with Geoprocessing tools in ArcPro. Note: The Digital Atlas of Australia downloaded and created a copy of the source data in February 2025 that was suitable to be hosted through ArcGIS Image Server & Image Dedicated. A copy of the raster dataset was created with RGB fields as a colour map with Geoprocessing tools in ArcPro, and the raster dataset was re-projected from 1994 Australia Albers to WGS 1984 Web Mercator (Auxiliary Sphere). Data dictionary

Attribute name Description

OID Internal feature number that uniquely identifies each row.

Service Pixel value (Scale) The scale at which land use was mapped in the vector catchment scale land use data provided by state and territory agencies or others:1:5,000, 1:10,000, 1:20,000, 1:25,000, 1:50,000, 1:100,000 or 1:250,000

Count Count of the number of raster cells in each class of VALUE.

Label Reflecting the scale of the source data ranges from 1:5,000 to 1:250,000

Contact Department of Agriculture, Fisheries and Forestry (ABARES), info.ABARES@aff.gov.au

This dataset contains Sentinel 2 and Landsat 8 cloud free composite satellite images of the Coral Sea reef areas and some parts of the Great Barrier Reef. It also contains raw depth contours derived from the satellite imagery. This dataset was developed as the base information for mapping the boundaries of reefs and coral cays in the Coral Sea. It is likely that the satellite imagery is useful for numerous other applications. The full source code is available and can be used to apply these techniques to other locations.

This dataset contains two sets of raw satellite derived bathymetry polygons for 5 m, 10 m and 20 m depths based on both the Landsat 8 and Sentinel 2 imagery. These are intended to be post-processed using clipping and manual clean up to provide an estimate of the top structure of reefs. This dataset also contains select scenes on the Great Barrier Reef and Shark bay in Western Australia that were used to calibrate the depth contours. Areas in the GBR were compared with the GA GBR30 2020 (Beaman, 2017) bathymetry dataset and the imagery in Shark bay was used to tune and verify the Satellite Derived Bathymetry algorithm in the handling of dark substrates such as by seagrass meadows. This dataset also contains a couple of small Sentinel 3 images that were used to check the presence of reefs in the Coral Sea outside the bounds of the Sentinel 2 and Landsat 8 imagery.

The Sentinel 2 and Landsat 8 imagery was prepared using the Google Earth Engine, followed by post processing in Python and GDAL. The processing code is available on GitHub (https://github.com/eatlas/CS_AIMS_Coral-Sea-Features_Img).

This collection contains composite imagery for Sentinel 2 tiles (59 in Coral Sea, 8 in GBR) and Landsat 8 tiles (12 in Coral Sea, 4 in GBR and 1 in WA). For each Sentinel tile there are 3 different colour and contrast enhancement styles intended to highlight different features. These include:
- TrueColour - Bands: B2 (blue), B3 (green), B4 (red): True colour imagery. This is useful to identifying shallow features are and in mapping the vegetation on cays.
- DeepFalse - Bands: B1 (ultraviolet), B2 (blue), B3 (green): False colour image that shows deep marine features to 50 - 60 m depth. This imagery exploits the clear waters of the Coral Sea to allow the ultraviolet band to provide a much deeper view of coral reefs than is typically achievable with true colour imagery. This imagery has a high level of contrast enhancement applied to the imagery and so it appears more noisy (in particular showing artefact from clouds) than the TrueColour styling.
- Shallow - Bands: B5 (red edge), B8 (Near Infrared) , B11 (Short Wave infrared): This false colour imagery focuses on identifying very shallow and dry regions in the imagery. It exploits the property that the longer wavelength bands progressively penetrate the water less. B5 penetrates the water approximately 3 - 5 m, B8 approximately 0.5 m and B11 < 0.1 m. Features less than a couple of metres appear dark blue, dry areas are white. This imagery is intended to help identify coral cay boundaries.

For Landsat 8 imagery only the TrueColour and DeepFalse stylings were rendered.

All Sentinel 2 and Landsat 8 imagery has Satellite Derived Bathymetry (SDB) depth contours.
- Depth5m - This corresponds to an estimate of the area above 5 m depth (Mean Sea Level).
- Depth10m - This corresponds to an estimate of the area above 10 m depth (Mean Sea Level).
- Depth20m - This corresponds to an estimate of the area above 20 m depth (Mean Sea Level).

For most Sentinel and some Landsat tiles there are two versions of the DeepFalse imagery based on different collections (dates). The R1 imagery are composites made up from the best available imagery while the R2 imagery uses the next best set of imagery. This splitting of the imagery is to allow two composites to be created from the pool of available imagery. This allows any mapped features to be checked against two images. Typically the R2 imagery will have more artefacts from clouds. In one Sentinel 2 tile a third image was created to help with mapping the reef platform boundary.

The satellite imagery was processed in tiles (approximately 100 x 100 km for Sentinel 2 and 200 x 200 km for Landsat 8) to keep each final image small enough to manage. These tiles were not merged into a single mosaic as it allowed better individual image contrast enhancement when mapping deep features. The dataset only covers the portion of the Coral Sea where there are shallow coral reefs and where their might have been potential new reef platforms indicated by existing bathymetry datasets and the AHO Marine Charts. The extent of the imagery was limited by those available through the Google Earth Engine.

# Methods:

The Sentinel 2 imagery was created using the Google Earth Engine. The core algorithm was:
1. For each Sentinel 2 tile, images from 2015 – 2021 were reviewed manually after first filtering to remove cloudy scenes. The allowable cloud cover was adjusted so that at least the 50 least cloud free images were reviewed. The typical cloud cover threshold was 1%. Where very few images were available the cloud cover filter threshold was raised to 100% and all images were reviewed. The Google Earth Engine image IDs of the best images were recorded, along with notes to help sort the images based on those with the clearest water, lowest waves, lowest cloud, and lowest sun glint. Images where there were no or few clouds over the known coral reefs were preferred. No consideration of tides was used in the image selection process. The collection of usable images were grouped into two sets that would be combined together into composite images. The best were added to the R1 composite, and the next best images into the R2 composite. Consideration was made as to whether each image would improve the resultant composite or make it worse. Adding clear images to the collection reduces the visual noise in the image allowing deeper features to be observed. Adding images with clouds introduces small artefacts to the images, which are magnified due to the high contrast stretching applied to the imagery. Where there were few images all available imagery was typically used.
2. Sunglint was removed from the imagery using estimates of the sunglint using two of the infrared bands (described in detail in the section on Sun glint removal and atmospheric correction).
3. A composite image was created from the best images by taking the statistical median of the stack of images selected in the previous stage, after masking out clouds and their shadows (described in detail later).
4. The brightness of the composite image was normalised so that all tiles would have a similar average brightness for deep water areas. This correction was applied to allow more consistent contrast enhancement. Note: this brightness adjustment was applied as a single offset across all pixels in the tile and so this does not correct for finer spatial brightness variations.
5. The contrast of the images was enhanced to create a series of products for different uses. The TrueColour colour image retained the full range of tones visible, so that bright sand cays still retain detail. The DeepFalse style was optimised to see features at depth and the Shallow style provides access to far red and infrared bands for assessing shallow features, such as cays and island.
6. The various contrast enhanced composite images were exported from Google Earth Engine and optimised using Python and GDAL. This optimisation added internal tiling and overviews to the imagery. The depth polygons from each tile were merged into shapefiles covering the whole for each depth.

## Cloud Masking

Prior to combining the best images each image was processed to mask out clouds and their shadows.

The cloud masking uses the COPERNICUS/S2_CLOUD_PROBABILITY dataset developed by SentinelHub (Google, n.d.; Zupanc, 2017). The mask includes the cloud areas, plus a mask to remove cloud shadows. The cloud shadows were estimated by projecting the cloud mask in the direction opposite the angle to the sun. The shadow distance was estimated in two parts.

A low cloud mask was created based on the assumption that small clouds have a small shadow distance. These were detected using a 40% cloud probability threshold. These were projected over 400 m, followed by a 150 m buffer to expand the final mask.

A high cloud mask was created to cover longer shadows created by taller, larger clouds. These clouds were detected based on an 80% cloud probability threshold, followed by an erosion and dilation of 300 m to remove small clouds. These were then projected over a 1.5 km distance followed by a 300 m buffer.

The buffering was applied as the cloud masking would often miss significant portions of the edges of clouds and their shadows. The buffering allowed a higher percentage of the cloud to be excluded, whilst retaining as much of the original imagery as possible.

The parameters for the cloud masking (probability threshold, projection distance and buffer radius) were determined through trial and error on a small number of scenes. The algorithm used is significantly better than the default Sentinel 2 cloud masking and slightly better than the COPERNICUS/S2_CLOUD_PROBABILITY cloud mask because it masks out shadows, however there is potentially significant improvements that could be made to the method in the future.

Erosion, dilation and buffer operations were performed at a lower image resolution than the native satellite image resolution to improve the computational speed. The resolution of these operations were adjusted so that they were performed with approximately a 4 pixel resolution during these operations. This made the cloud mask