This dataset consists of shapefiles that correspond to the model grids used in the CSIRO eReefs hydrodynamic and biogeochemical models. These models store their results in multi-dimensional NetCDF files using a curvilinear grid. This dataset corresponds to an extract from these files converting the curvilinear grid into polygons in a shapefile. This dataset only captures the structure of the grid, not the time series data generated by the model. It contains shapefiles of the 4 km model grid (GBR4) and the 1 km grid (GBR1) as well as shapefiles for the bounding polygon of all the 'wet' cells in the model. This dataset is useful for visualising the extent of the various CSIRO eReefs models.

This dataset contains shapefiles for the 1 km and 4 km eReefs grids, derived from version 2.0 of the eReefs Hydrodynamic model. It contains shapefiles of the individual grid cells and the bounds. It also includes a low resolution version of the bounds suitable for detecting whether locations are inside the eReefs model extent.

The grid shapefile contains polygons representing each of the grid cells. An attribution is associated with each polygon corresponding to the depth used in the model. This can be used to show where the model has 'wet' cells.

Methods:

1. Representative data files for the GBR1 and GBR4 hydrodynamic version model were downloaded from the public repository of eReefs model data on NCI. The two common grids GBR1 and GBR4 are used over the model time series and for the both the hydrodynamic and biogeochemical models. We therefore just chose one model NetCDF for each model resolution. These were taken from the hydrodynamic model version 2.

2. The grid was converted to shapefiles using an R script that calculated the coordinates corners of each curvilinear pixel in the grid based on the centroids of the neighbouring pixels.

3. The grid boundary shapefiles were calculated using the merge GIS operation in QGIS after selecting all the 'wet' cells, where the depth was greater than 0.

Full step-by-step instructions and scripts are available to reproduce this dataset from github (https://github.com/eatlas/GBR_AIMS_eReefs-grid-shapefiles).

Format:

Shapefile

Data Dictionary:

SP_ID: Row and column indices in the NetCDF grid joined together depth: Depth used in the eReefs model in metres. This is based on the botz variable in the original NetCDF eReefs model data file. row: Row index in the NetCDF tables for this pixel. col: Column index in the NetCDF tables for this pixel.

Change Log: 2025-10-16 - Added DOI, ROR and ORCIDs to the record.

Data Location:

This dataset is filed in the eAtlas enduring data repository at: X:\data\custodian\2018-22-eReefs\GBR_AIMS_eReefs-grid-shapefiles Source code for reproducing this dataset is available on github (https://github.com/eatlas/GBR_AIMS_eReefs-grid-shapefiles).

This dataset corresponds to land area polygons of Australian coastline and surrounding islands. It was generated from 10 m Sentinel 2 imagery from 2022 - 2024 using the Normalized Difference Water Index (NDWI) to distinguish land from water. It was estimated from composite imagery made up from images where the tide is above the mean sea level. The coastline approximately corresponds to the mean high water level. This dataset was created as part of the NESP MaC 3.17 northern Australian Reef mapping project. It was developed to allow the inshore edge of digitised fringing reef features to be neatly clipped to the land areas without requiring manual digitisation of the neighbouring coastline. This required a coastline polygon with an edge positional error of below 50 m so as to not distort the shape of small fringing reefs. We found that existing coastline datasets such as the Geodata Coast 100K 2004 and the Australian Hydrographic Office (AHO) Australian land and coastline dataset did not meet our needs. The scale of the Geodata Coast 100K 2004 was too coarse to represent small islands and the the positional error of the Australian Hydrographic Office (AHO) Australian land and coastline dataset was too high (typically 80 m) for our application as the errors would have introduced significant errors in the shape of small fringing reefs. The Digital Earth Australia Coastline (GA) dataset was sufficiently accurate and detailed however the format of the data was unsuitable for our application as the coast was expressed as disconnected line features between rivers, rather than a closed polygon of the land areas. We did however base our approach on the process developed for the DEA coastline described in Bishop-Taylor et al., 2021 (https://doi.org/10.1016/j.rse.2021.112734). Adapting it to our existing Sentinel 2 Google Earth processing pipeline. The difference between the approach used for the DEA coastline and this dataset was the DEA coastline performed the tidal calculations and filtering at the pixel level, where as in this dataset we only estimated a single tidal level for each whole Sentinel image scene. This was done for computational simplicity and to align with our existing Google Earth Engine image processing code. The images in the stack were sorted by this tidal estimate and those with a tidal high greater than the mean seal level were combined into the composite. The Sentinel 2 satellite follows a sun synchronous orbit and so does not observe the full range of tidal levels. This observed tidal range varies spatially due to the relative timing of peak tides with satellite image timing. We made no accommodation for variation in the tidal levels of the images used to calculate the coastline, other than selecting images that were above the mean tide level. This means tidal height that the dataset coastline corresponds to will vary spatially. While this approach is less precise than that used in the DEA Coastline the resulting errors were sufficiently low to meet the project goals.
This simplified approach was chosen because it integrated well with our existing Sentinel 2 processing pipeline for generating composite imagery. To verify the accuracy of this dataset we manually checked the generated coastline with high resolution imagery (ArcGIS World Imagery). We found that 90% of the coastline polygons in this dataset have a horizontal position error of less than 20 m when compared to high-resolution imagery, except for isolated failure cases. During our manual checks we identified some areas where our algorithm can lead to falsely identifying land or not identifying land. We identified specific scenarios, or 'failure modes,' where our algorithm struggled to distinguish between land and water. These are shown in the image "Potential failure modes": a) The coastline is pushed out due to breaking waves (example: western coast, S2 tile ID 49KPG). b) False land polygons are created because of very turbid water due to suspended sediment. In clear water areas the near infrared channel is almost black, starkly different to the bright land areas. In very highly turbid waters the suspended sediment appears in the near infrared channel, raising its brightness to a level where it starts to overlap with the brightness of the dimmest land features. (example: Joseph Bonaparte Gulf, S2 tile ID 52LEJ). This results in turbid rivers not being correctly mapped. In version 1-1 of the dataset the rivers across northern Australia were manually corrected for these failures. c) Very shallow, gentle sloping areas are not recognised as water and the coastline is pushed out (example: Mornington Island, S2 tile ID 54KUG). Update: A second review of this area indicated that the mapped coastline is likely to be very close to the try coastline. d) The coastline is lower than the mean high water level (example: Great Keppel (Wop-pa) Island, S2 tile ID 55KHQ). Some of these potential failure modes could probably be addressed in the future by using a higher resolution tide calculation and using adjusted NDWI thresholds per region to accommodate for regional differences. Some of these failure modes are likely due to the near infrared channel (B8) being able to penetrate the water approximately 0.5 m leading to errors in very shallow areas. Some additional failures include: - Interpreting jetties as land - Interpreting oil rigs as land - Bridges being interpreted as land, cutting off rivers Methods: The coastline polygons were created in four separate steps: 1. Create above mean sea level (AMSL) composite images. 2. Calculate the Normalized Difference Water Index (NDWI) and visualise as a grey scale image. 3. Generate vector polygons from the grey scale image using a NDWI threshold. 4. Clean up and merge polygons. To create the AMSL composite images, multiple Sentinel 2 images were combined using the Google Earth Engine. The core algorithm was: 1. For each Sentinel 2 tile filter the "COPERNICUS/S2_HARMONIZED" image collection by - tile ID - maximum cloud cover 20% - date between '2022-01-01' and '2024-06-30' - asset_size > 100000000 (remove small fragments of tiles) 2. Remove high sun-glint images (see "High sun-glint image detection" for more information). 3. Split images by "SENSING_ORBIT_NUMBER" (see "Using SENSING_ORBIT_NUMBER for a more balanced composite" for more information). 4. Iterate over all images in the split collections to predict the tide elevation for each image from the image timestamp (see "Tide prediction" for more information). 5. Remove images where tide elevation is below mean sea level. 6. Select maximum of 200 images with AMSL tide elevation. 7. Combine SENSING_ORBIT_NUMBER collections into one image collection. 8. Remove sun-glint and apply atmospheric correction on each image (see "Sun-glint removal and atmospheric correction" for more information). 9. Duplicate image collection to first create a composite image without cloud masking and using the 15th percentile of the images in the collection (i.e. for each pixel the 15th percentile value of all images is used). 10. Apply cloud masking to all images in the original image collection (see "Cloud Masking" for more information) and create a composite by using the 15th percentile of the images in the collection (i.e. for each pixel the 15th percentile value of all images is used). 11. Combine the two composite images (no cloud mask composite and cloud mask composite). This solves the problem of some coral cays and islands being misinterpreted as clouds and therefore creating holes in the composite image. These holes are "plugged" with the underlying composite without cloud masking. (Lawrey et al. 2022) Next, for each image the NDWI was calculated: 1. Calculate the normalised difference using the B3 (green) and B8 (near infrared). 2. Shift the value range from between -1 and +1 to values between 1 and 255 (0 reserved as no-data value). 3. Export image as 8 bit unsigned Integer grey scale image. During the next step, we generated vector polygons from the grey scale image using a NDWI threshold: 1. Upscale image to 5 m resolution using bilinear interpolation. This was to help smooth the coastline and reduce the error introduced by the jagged pixel edges. 2. Apply a threshold to create a binary image (see "NDWI Threshold" for more information) with the value 1 for land and 2 for water (0: no data). 3. Create polygons for land values (1) in the binary image. 4. Export as shapefile. Finally, we created a single layer from the vectorised images: 1. Merge and dissolve all vector layers in QGIS. 2. Perform smoothing (QGIS toolbox, Iterations 1, Offset 0.25, Maximum node angle to smooth 180). 3. Perform simplification (QGIS toolbox, tolerance 0.00003). 4. Remove polygon vertices on the inner circle to fill out the continental Australia. 5. Perform manual QA/QC. In this step we removed false polygons created due to sun glint and breaking waves. We also removed very small features (1 – 1.5 pixel sized features, e.g. single mangrove trees) by calculating the area of each feature (in m2) and removing features smaller than 200 m2. 15th percentile composite: The composite image was created using the 15th percentile of the pixels values in the image stack. The 15th percentile was chosen, in preference to the median, to select darker pixels in the stack as these tend to correspond to images with clearer water conditions and higher tides. High sun-glint image detection: Images with high sun-glint can lead to lower quality composite images. To determine high sun-glint images, a land mask was first applied to the image to only retain water pixels. This land mask was estimated using NDWI. The proportion of the water pixels in the near-infrared and short-wave infrared bands above a sun-glint threshold was calculated. Images with a high proportion were then filtered out of the image collection.
Sun-glint removal and atmospheric correction: The Top of Atmosphere L1

ps-places-metadata-v1.01

SUMMARY

This dataset comprises a pair of layers, (points and polys) which attempt to better locate "populated places" in NZ. Populated places are defined here as settled areas, either urban or rural where densitys of around 20 persons per hectare exist, and something is able to be seen from the air.

RATIONALE

The only liberally licensed placename dataset is currently LINZ geographic placenames, which has the following drawbacks: - coordinates are not place centers but left most label on 260 series map - the attributes are outdated

METHODOLOGY

This dataset necessarily involves cleaving the linz placenames set into two, those places that are poplulated, and those unpopulated. Work was carried out in four steps. First placenames were shortlisted according to the following criterion: - all places that rated at least POPL in the linz geographic places layer, ie POPL, METR or TOWN or USAT were adopted. - Then many additional points were added from a statnz meshblock density analysis.
- Finally remaining points were added from a check against linz residential polys, and zenbu poi clusters.

Spelling is broadly as per linz placenames, but there are differences for no particular reason. Instances of LINZ all upper case have been converted to sentance case. Some places not presently in the linz dataset are included in this set, usually new places, or those otherwise unnamed. They appear with no linz id, and are not authoritative, in some cases just wild guesses.

Density was derived from the 06 meshblock boundarys (level 2, geometry fixed), multipart conversion, merging in 06 usually resident MB population then using the formula pop/area*10000. An initial urban/rural threshold level of 0.6 persons per hectare was used.

Step two was to trace the approx extent of each populated place. The main purpose of this step was to determine the relative area of each place, and to create an intersection with meshblocks for population. Step 3 involved determining the political center of each place, broadly defined as the commercial center.

Tracing was carried out at 1:9000 for small places, and 1:18000 for large places using either bing or google satellite views. No attempt was made to relate to actual town 'boundarys'. For example large parks or raceways on the urban fringe were not generally included. Outlying industrial areas were included somewhat erratically depending on their connection to urban areas.

Step 3 involved determining the centers of each place. Points were overlaid over the following layers by way of a base reference:

a. original linz placenames b. OSM nz-locations points layer c. zenbu pois, latest set as of 5/4/11 d. zenbu AllSuburbsRegions dataset (a heavily hand modified) LINZ BDE extract derived dataset courtesy Zenbu. e. LINZ road-centerlines, sealed and highway f. LINZ residential areas, g. LINZ building-locations and building footprints h. Olivier and Co nz-urban-north and south

Therefore in practice, sources c and e, form the effective basis of the point coordinates in this dataset. Be aware that e, f and g are referenced to the LINZ topo data, while c and d are likely referenced to whatever roading dataset google possesses. As such minor discrepencys may occur when moving from one to the other.

Regardless of the above, this place centers dataset was created using the following criteria, in order of priority:

• attempts to represent the present (2011) subjective 'center' of each place as defined by its commercial/retail center ie. mainstreets where they exist, any kind of central retail cluster, even a single shop in very small places.
• the coordinate is almost always at the junction of two or more roads.
• most of the time the coordinate is at or near the centroid of the poi cluster
• failing any significant retail presence, the coordinate tends to be placed near the main road junction to the community.
• when the above criteria fail to yield a definitive answer, the final criteria involves the centroids of: . the urban polygons . the clusters of building footprints/locations.

To be clear the coordinates are manually produced by eye without any kind of computation. As such the points are placed approximately perhaps plus or minus 10m, but given that the roads layers are not that flash, no attempt was made to actually snap the coordinates to the road junctions themselves.

The final step involved merging in population from SNZ meshblocks (merge+sum by location) of popl polys). Be aware that due to the inconsistent way that meshblocks are defined this will result in inaccurate populations, particular small places will collect population from their surrounding area. In any case the population will generally always overestimate by including meshblocks that just nicked the place poly. Also there are a couple of dozen cases of overlapping meshblocks between two place polys and these will double count. Which i have so far made no attempt to fix.

Merged in also tla and regions from SNZ shapes, a few of the original linz atrributes, and lastly grading the size of urban areas according to SNZ 'urban areas" criteria. Ie: class codes:

1. Not used.
2. main urban area 30K+
3. secondary urban area 10k-30K
4. minor urban area 1k-10k
5. rural center 300-1K
6. village -300

Note that while this terminology is shared with SNZ the actual places differ owing to different decisions being made about where one area ends an another starts, and what constiutes a suburb or satellite. I expect some discussion around this issue. For example i have included tinwald and washdyke as part of ashburton and timaru, but not richmond or waikawa as part of nelson and picton. Im open to discussion on these.

No attempt has or will likely ever be made to locate the entire LOC and SBRB data subsets. We will just have to wait for NZFS to release what is thought to be an authoritative set.

PROJECTION

Shapefiles are all nztm. Orig data from SNZ and LINZ was all sourced in nztm, via koordinates, or SNZ. Satellite tracings were in spherical mercator/wgs84 and converted to nztm by Qgis. Zenbu POIS were also similarly converted.

ATTRIBUTES

Shapefile: Points id : integer unique to dataset name : name of popl place, string class : urban area size as above. integer tcode : SNZ tla code, integer rcode : SNZ region code, 1-16, integer area : area of poly place features, integer in square meters. pop : 2006 usually resident popluation, being the sum of meshblocks that intersect the place poly features. Integer lid : linz geog places id desc_code : linz geog places place type code

Shapefile: Polygons gid : integer unique to dataset, shared by points and polys name : name of popl place, string, where spelling conflicts occur points wins area : place poly area, m2 Integer

LICENSE

Clarification about the minorly derived nature of LINZ and google data needs to be sought. But pending these copyright complications, the actual points data is essentially an original work, released as public domain. I retain no copyright, nor any responsibility for data accuracy, either as is, or regardless of any changes that are subsequently made to it.

Peter Scott 16/6/2011

v1.01 minor spelling and grammar edits 17/6/11

This dataset contains both large (A0) printable maps of the Torres Strait broken into six overlapping regions, based on a clear sky, clear water composite Sentinel 2 composite imagery and the imagery used to create these maps. These maps show satellite imagery of the region, overlaid with reef and island boundaries and names. Not all features are named, just the more prominent features. This also includes a vector map of Ashmore Reef and Boot Reef in Coral Sea as these were used in the same discussions that these maps were developed for. The map of Ashmore Reef includes the atoll platform, reef boundaries and depth polygons for 5 m and 10 m.

This dataset contains all working files used in the development of these maps. This includes all a copy of all the source datasets and all derived satellite image tiles and QGIS files used to create the maps. This includes cloud free Sentinel 2 composite imagery of the Torres Strait region with alpha blended edges to allow the creation of a smooth high resolution basemap of the region.

The base imagery is similar to the older base imagery dataset: Torres Strait clear sky, clear water Landsat 5 satellite composite (NERP TE 13.1 eAtlas, AIMS, source: NASA).

Most of the imagery in the composite imagery from 2017 - 2021.


Method:
The Sentinel 2 basemap was produced by processing imagery from the World_AIMS_Marine-satellite-imagery dataset (01-data/World_AIMS_Marine-satellite-imagery in the data download) for the Torres Strait region. The TrueColour imagery for the scenes covering the mapped area were downloaded. Both the reference 1 imagery (R1) and reference 2 imagery (R2) was copied for processing. R1 imagery contains the lowest noise, most cloud free imagery, while R2 contains the next best set of imagery. Both R1 and R2 are typically composite images from multiple dates.

The R2 images were selectively blended using manually created masks with the R1 images. This was done to get the best combination of both images and typically resulted in a reduction in some of the cloud artefacts in the R1 images. The mask creation and previewing of the blending was performed in Photoshop. The created masks were saved in 01-data/R2-R1-masks. To help with the blending of neighbouring images a feathered alpha channel was added to the imagery. The processing of the merging (using the masks) and the creation of the feathered borders on the images was performed using a Python script (src/local/03-merge-R2-R1-images.py) using the Pillow library and GDAL. The neighbouring image blending mask was created by applying a blurring of the original hard image mask. This allowed neighbouring image tiles to merge together.

The imagery and reference datasets (reef boundaries, EEZ) were loaded into QGIS for the creation of the printable maps.

To optimise the matching of the resulting map slight brightness adjustments were applied to each scene tile to match its neighbours. This was done in the setup of each image in QGIS. This adjustment was imperfect as each tile was made from a different combinations of days (to remove clouds) resulting in each scene having a different tonal gradients across the scene then its neighbours. Additionally Sentinel 2 has slight stripes (at 13 degrees off the vertical) due to the swath of each sensor having a slight sensitivity difference. This effect was uncorrected in this imagery.


Single merged composite GeoTiff:
The image tiles with alpha blended edges work well in QGIS, but not in ArcGIS Pro. To allow this imagery to be used across tools that don't support the alpha blending we merged and flattened the tiles into a single large GeoTiff with no alpha channel. This was done by rendering the map created in QGIS into a single large image. This was done in multiple steps to make the process manageable.

The rendered map was cut into twenty 1 x 1 degree georeferenced PNG images using the Atlas feature of QGIS. This process baked in the alpha blending across neighbouring Sentinel 2 scenes. The PNG images were then merged back into a large GeoTiff image using GDAL (via QGIS), removing the alpha channel. The brightness of the image was adjusted so that the darkest pixels in the image were 1, saving the value 0 for nodata masking and the boundary was clipped, using a polygon boundary, to trim off the outer feathering. The image was then optimised for performance by using internal tiling and adding overviews. A full breakdown of these steps is provided in the README.md in the 'Browse and download all data files' link.

The merged final image is available in export\TS_AIMS_Torres Strait-Sentinel-2_Composite.tif.


Source datasets:
Complete Great Barrier Reef (GBR) Island and Reef Feature boundaries including Torres Strait Version 1b (NESP TWQ 3.13, AIMS, TSRA, GBRMPA), https://eatlas.org.au/data/uuid/d2396b2c-68d4-4f4b-aab0-52f7bc4a81f5

Geoscience Australia (2014b), Seas and Submerged Lands Act 1973 - Australian Maritime Boundaries 2014a - Geodatabase [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, https://dx.doi.org/10.4225/25/5539DFE87D895

Basemap/AU_GA_AMB_2014a/Exclusive_Economic_Zone_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014a). The Geodatabase was loaded in ArcMap. The Exclusive_Economic_Zone_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

Geoscience Australia (2014a), Treaties - Australian Maritime Boundaries (AMB) 2014a [Dataset]. Canberra, Australia: Author. https://creativecommons.org/licenses/by/4.0/ [license]. Sourced on 12 July 2017, http://dx.doi.org/10.4225/25/5539E01878302
Basemap/AU_GA_Treaties-AMB_2014a/Papua_New_Guinea_TSPZ_AMB2014a_Limit.shp
The original data was obtained from GA (Geoscience Australia, 2014b). The Geodatabase was loaded in ArcMap. The Papua_New_Guinea_TSPZ_AMB2014a_Limit layer was loaded and exported as a shapefile. Since this file was small no clipping was applied to the data.

AIMS Coral Sea Features (2022) - DRAFT
This is a draft version of this dataset. The region for Ashmore and Boot reef was checked. The attributes in these datasets haven't been cleaned up. Note these files should not be considered finalised and are only suitable for maps around Ashmore Reef. Please source an updated version of this dataset for any other purpose.
CS_AIMS_Coral-Sea-Features/CS_Names/Names.shp
CS_AIMS_Coral-Sea-Features/CS_Platform_adj/CS_Platform.shp
CS_AIMS_Coral-Sea-Features/CS_Reef_Boundaries_adj/CS_Reef_Boundaries.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth5m_Coral-Sea.shp
CS_AIMS_Coral-Sea-Features/CS_Depth/CS_AIMS_Coral-Sea-Features_Img_S2_R1_Depth10m_Coral-Sea.shp

Murray Island 20 Sept 2011 15cm SISP aerial imagery, Queensland Spatial Imagery Services Program, Department of Resources, Queensland
This is the high resolution imagery used to create the map of Mer.

World_AIMS_Marine-satellite-imagery
The base image composites used in this dataset were based on an early version of Lawrey, E., Hammerton, M. (2024). Marine satellite imagery test collections (AIMS) [Data set]. eAtlas. https://doi.org/10.26274/zq26-a956. A snapshot of the code at the time this dataset was developed is made available in the 01-data/World_AIMS_Marine-satellite-imagery folder of the download of this dataset.


Data Location:
This dataset is filed in the eAtlas enduring data repository at: data\custodian\2020-2029-AIMS\TS_AIMS_Torres-Strait-Sentinel-2-regional-maps. On the eAtlas server it is stored at eAtlas GeoServer\data\2020-2029-AIMS.


Change Log:
2025-05-12: Eric Lawrey
Added Torres-Strait-Region-Map-Masig-Ugar-Erub-45k-A0 and Torres-Strait-Eastern-Region-Map-Landscape-A0. These maps have a brighten satellite imagery to allow easier reading of writing on the maps. They also include markers for geo-referencing the maps for digitisation.

2025-02-04: Eric Lawrey
Fixed up the reference to the World_AIMS_Marine-satellite-imagery dataset, clarifying where the source that was used in this dataset. Added ORCID and RORs to the record.

2023-11-22: Eric Lawrey
Added the data and maps for close up of Mer.
- 01-data/TS_DNRM_Mer-aerial-imagery/
- preview/Torres-Strait-Mer-Map-Landscape-A0.jpeg
- exports/Torres-Strait-Mer-Map-Landscape-A0.pdf
Updated 02-Torres-Strait-regional-maps.qgz to include the layout for the new map.

2023-03-02: Eric Lawrey
Created a merged version of the satellite imagery, with no alpha blending so that it can be used in ArcGIS Pro. It is now a single large GeoTiff image. The Google Earth Engine source code for the World_AIMS_Marine-satellite-imagery was included to improve the reproducibility and provenance of the dataset, along with a calculation of the distribution of image dates that went into the final composite image. A WMS service for the imagery was also setup and linked to from the metadata. A cross reference to the older Torres Strait clear sky clear water Landsat composite imagery was also added to the record.