Using survey statistics, reference land cover data to compare to mapped land cover data for development of data quality and its evaluation. This dataset is associated with the following publication: Wickham, J., S. Stehman, D. Sorenson, L. Gass, and J. Dewitz. Thematic accuracy assessment of the NLCD 2019 land cover for the conterminous United States. GIScience and Remote Sensing. Taylor & Francis Group, London, UK, 60(1): 2181143, (2023).

Accuracy assessment is one of the most important components of both applied and research-oriented remote sensing projects. For mapped classes that have sharp and easily identified boundaries, a broad array of accuracy assessment methods has been developed. However, accuracy assessment is in many cases complicated by classes that have fuzzy, indeterminate, or gradational boundaries, a condition which is common in real landscapes; for example, the boundaries of wetlands, many soil map units, and tree crowns. In such circumstances, the conventional approach of treating all reference pixels as equally important, whether located on the map close to the boundary of a class, or in the class center, can lead to misleading results. We therefore propose an accuracy assessment approach that relies on center-weighting map segment area to calculate a variety of common classification metrics including overall accuracy, class user’s and producer’s accuracy, precision, recall, specificity, and the F1 score. This method offers an augmentation of traditional assessment methods, can be used for both binary and multiclass assessment, allows for the calculation of count- and area-based measures, and permits the user to define the impact of distance from map segment edges based on a distance weighting exponent and a saturation threshold distance, after which the weighting ceases to grow. The method is demonstrated using synthetic and real examples, highlighting its use when the accuracy of maps with inherently uncertain class boundaries is evaluated.

Accuracy assessment is one of the most important components of both applied and research-oriented remote sensing projects. For mapped classes that have sharp and easily identified boundaries, a broad array of accuracy assessment methods has been developed. However, accuracy assessment is in many cases complicated by classes that have fuzzy, indeterminate, or gradational boundaries, a condition which is common in real landscapes; for example, the boundaries of wetlands, many soil map units, and tree crowns. In such circumstances, the conventional approach of treating all reference pixels as equally important, whether located on the map close to the boundary of a class, or in the class center, can lead to misleading results. We therefore propose an accuracy assessment approach that relies on center-weighting map segment area to calculate a variety of common classification metrics including overall accuracy, class user’s and producer’s accuracy, precision, recall, specificity, and the F1 score. This method offers an augmentation of traditional assessment methods, can be used for both binary and multiclass assessment, allows for the calculation of count- and area-based measures, and permits the user to define the impact of distance from map segment edges based on a distance weighting exponent and a saturation threshold distance, after which the weighting ceases to grow. The method is demonstrated using synthetic and real examples, highlighting its use when the accuracy of maps with inherently uncertain class boundaries is evaluated.

In May 2021, the Grand Canyon Monitoring and Research Center (GCMRC) of the U.S. Geological Survey’s (USGS), Southwest Biological Science Center (SBSC) acquired airborne multispectral high resolution data for the Colorado River in Grand Canyon in Arizona, USA. The imagery data consist of four bands (Band 1 – red, Band 2 – green, Band 3 – blue, and Band 4 – near infrared) with a ground resolution of 20 centimeters (cm). These image data are available to the public as 16-bit GeoTIFF files, which can be read and used by most geographic information system (GIS) and image-processing software. The spatial reference of the image data are in the State Plane (SP) map projection using the central Arizona zone (FIPS 0202) and the North American Datum of 1983 (NAD83) National Adjustment of 2011 (NA2011). The airborne data acquisition was conducted under contract by Fugro Earthdata Inc (Fugro) using two fixed wing aircraft from May 29th to June 4th, 2021 at flight altitudes from approximately 2,440 to 3,350 meters above mean sea level. Fugro produced a corridor-wide mosaic using the best possible flight line images with the least amount of smear, the smallest shadow extent, and clearest, most glint-free water possible. The mosaic delivered by Fugro was then further corrected by GCMRC for smear, shadow extent and water clarity as described in the process steps of this metadata and for previous image acquisitions in Durning et al. (2016) and Davis (2012). 47 ground controls points (GCPs) were used to conduct an independent spatial accuracy assessment by GCMRC. The accuracy calculated from the GCPs is reported at 95% confidence as 0.514 m and a Root Mean Square Error (RMSE) of 0.297 m.

An assessment of the overall accuracy of subzone classification across all locations that were used as modeling plots in the most recent Gradient Nearest Neighbor (GNN) modeling.

Learn everything about why and how to do an accuracy assessment on remotely sensed data through online videos and associated exercises. This video series was developed by Dr. Russ Congalton with New HampshireView.

Biome description:1Tropical and Subtropical Moist Broadleaf Forest (Amazon basin section).2Tropical and Subtropical Moist Broadleaf Forest (Coastal lowlands section).3Tropical and Subtropical Dry Broadleaf Forest.4Tropical and Subtropical Grasslands, Savannas and Shrublands.

The zip file contains raw accuracy assessment data used and two R Markdown files used to process and analyse the data. These markdowns produce the tables and figures presented in the paper

Release of NLCD 2006 provides the first land-cover change database for the Conterminous United States (CONUS) from Landsat Thematic Mapper data. Accuracy assessment of NLCD 2006 focused on four primary products: 2001 land cover, 2006 land cover, land-cover change between 2001 and 2006, and impervious surface change between 2001 and 2006. The accuracy assessment was conducted by selecting a stratified random sample of pixels with the reference classification interpreted from multi-temporal high resolution digital imagery. The NLCD Level II (16 classes) overall accuracies for the 2001 and 2006 land cover were 79% and 78%, respectively, with Level II user's accuracies exceeding 80% for water, high density urban, all upland forest classes, shrubland, and cropland for both dates. Level I (8 classes) accuracies were 85% for NLCD 2001 and 84% for NLCD 2006. The high overall and user's accuracies for the individual dates translated into high user's accuracies for the 2001–2006 change repo ...

This project is a cooperative effort among the National Ocean Service, National Centers for Coastal Ocean Science, Center for Coastal Monitoring and Assessment; the University of Hawaii; BAE Systems Spectral Solutions; and Analytical Laboratories of Hawaii, LLC. The goal of the work was to incorporate previously developed mapping methods to produce coral reef habitat maps for American Samoa, Guam and the Commonwealth of the Northern Mariana Islands. GPS field observations were used to establish the thematic accuracy of this thematic product. 651 benthic habitat characterizations were completed for this work.

Accuracy assessment is a standard protocol of National Land Cover Database (NLCD) mapping. Here we report agreement statistics between map and reference labels for NLCD 2011, which includes land cover for 2001, 2006, and 2011. The two main objectives were assessment of agreement between map and reference labels for the three, single-date NLCD land cover products at Level II and Level I of the classification hierarchy, and agreement for 17 land cover change themes based on Level I classes (e.g., forest loss; forest gain; forest, no change) for three change periods (2001–2006, 2006–2011, and 2001–2011). The single-date overall accuracies were 82%, 83%, and 83% at Level II and 88%, 89%, and 89% at Level I for 2011, 2006, and 2001, respectively. Overall accuracies for 2006 and 2001 land cover components of NLCD 2011 were approximately 4% higher (at Level II and Level I) than the overall accuracies for the same components of NLCD 2006. User's accuracies were high for the no change repo ...

One hundred points were randomly chosen within the study area (50 cliffs and 50 non-cliff, i.e. ground truthing) against which the results of the classification of each method were compared: Google Street View (SV) and three DEM-based maps with different thresholds of slope (Smin, S25th and S50th, see text for more details). The table shows also overall accuracy, producer and user accuracy, omission and commission error rates and Cohen’s Kappa coefficients for each method.

This project is a cooperative effort between the National Ocean Service, National Centers for Coastal Ocean Science, Center for Coastal Monitoring and Assessment, the University of Hawaii, BAE Systems Spectral Solutions and Analytical Laboratories of Hawaii, LLC. The goal of the work was to incorporate previously developed mapping methods to produce coral reef habitat maps for the Main Eight Hawaiian Islands. GPS field observations were used to establish the thematic accuracy of this thematic product. 39 benthic habitat characterizations were completed in UTM Zone 5 for this work.

The purpose of this survey data was to collect information on current habitat conditions at random locations throughout the mapping area. Locations were determined by an iterative, GIS-based, stratified random sampling technique to ensure that all bottom classifications would be assessed. This information was used to evaluate the thematic accuracy of the St. John benthic habitat map.

The National Land Cover Database (NLCD) is a land cover monitoring program providing land cover information for the United States. NLCD2016 extended temporal coverage to 15 years (2001–2016). We collected land cover reference data for the 2011 and 2016 nominal dates to report land cover accuracy for the NLCD2016 database 2011 and 2016 land cover components. We measured land cover accuracy at Level II and Level I, and change accuracy at Level I. For both the 2011 and 2016 land cover components, single-date Level II overall accuracies (OA) were 72% (standard error of ±0.9%) when agreement was defined as match between the map label and primary reference label only and 86% (± 0.7%) when agreement also included the alternate reference label. The corresponding level I OA for both dates were 79% (± 0.9%) and 91% (± 1.0%). For land cover change, the 2011–2016 user’s and producer’s accuracies (UA and PA) were ~ 75% for forest loss. PA for water loss, grassland loss, and grass gain were > 70% when agreement included a match between the map label and either the primary or alternate reference label. Depending on agreement definition and level of the classification hierarchy, OA for the 2011 land cover component of the NLCD2016 database was about 4% to 7% higher than OA for the 2011 land cover component of the NLCD2011 database, suggesting that the changes in mapping methodologies initiated for production of the NLCD2016 database have led to improved product quality.

Accuracy assessment of average crop interval distance.

This project is a cooperative effort among the National Ocean Service, National Centers for Coastal Ocean Science, Center for Coastal Monitoring and Assessment; the University of Hawaii; BAE Systems Spectral Solutions; and Analytical Laboratories of Hawaii, LLC. The goal of the work was to incorporate previously developed mapping methods to produce coral reef habitat maps for American Samoa, Guam and the Commonwealth of the Northern Mariana Islands. GPS field observations were used to establish the thematic accuracy of this thematic product. 1113 benthic habitat characterizations were completed for this work.

This dataset contains the data and scripts to reproduce figures from 'Evaluating the accuracy of binary classifiers for geomorphic applications' published in Earth Surface Dynamics (Rossi, 2024).Figure 1 elevation data was downloaded from OpenTopography (2010 Channel Islands Lidar Collection, 2012; Anderson et al., 2012; Reed, 2006). GIS files for elevation data and transect locations are provided in the zipped geodatabase gis_fig1.gdb.zip.Figure 2 is based on the bedrock mapping at site P01 from Rossi et al. (2020). GIS files for 1-m slope, air photo mapping, its conversion to a truth raster, and the accuracy classification using a 38 degree slope threshold are provided in the zipped geodatabase gis_fig2.gdb.zip. Figures 3-7 are ultimately based on synthetic_feature_maps_main.py and synthetic_feature_maps_functions.py. The former uses the latter to plot example classified maps along with how accuracy scores vary as a function of feature fraction for a given set of input parameters set by the user. Results are saved as a .csv file. Because these master scripts are designed for one set of input parameters, I provide a number of other scripts below that aid in reproducing the figures shown in the manuscript.Figure 3a and 3c can be reproduced using generate_fig3.py directly using input parameters of l = 100, scl = 1, sflag = 2, and fmap = 0.5. This plots the 'match scene' scenario only. Note that there is code that is commented out that will let you plot the 'all feature' scenario as well.Figure 3b and 3d can be reproduced using generate_fig3.py directly using input parameters of l = 100, scl = 10, sflag = 2, and fmap = 0.5. This plots the 'match scene' scenario only. Note that there is code that is commented out that will let you plot the 'all feature' scenario as well.Figure 4 can be reproduced using generate_Fig4.py. It uses saved results from synthetic_feature_maps_main.py that are stored in the folder results_rand_only.Figure 5 can be reproduced using generate_Fig5.py. It uses saved results from synthetic_feature_maps_main.py that are stored in the folder results_syst_only.Figure 6 can be reproduced using generate_Fig6.py. It uses saved results from synthetic_feature_maps_main.py that are stored in the folder results_rand_plus_syst.Figure 7 can be reproduced using generate_Fig7.py. It uses saved results from synthetic_feature_maps_main.py that are stored in the folders results_rand_only, results_syst_only, and results_rand_plus_syst.Figure 8 is conceptual. Figs. 8a-b were drawn in Adobe Illustrator. The plot shown in Fig. 8c can be reproduced using generate_Fig8c.py and requires the associated file fig8_examples.txt.Figure 9 is conceptual. Fig. 9a was drawn in Adobe Illustrator. The plot shown in Fig. 9b can be reproduced using generate_Fig9b.py. Because it is not using saved results and runs the 'systematic error' scenario from scratch using synthetic_feature_maps_functions.py, this script will take a bit of time to run.Table 1 uses the data from the classified map in Fig 2a and can be directly derived from eqs. 1-7.Table 2 requires merging two scenes with different feature fractions to produce and average feature fraction of 0.50. Each cell in the table can be calculated using generate_Table2_contents.py. It uses saved results from synthetic_feature_maps_main.py that are stored in the folders results_rand_only, results_syst_only, and results_rand_plus_syst.

The National Land Cover Database (NLCD), a product suite produced through the Multi-resolution Land Characteristics (MRLC) consortium, is an operational land cover monitoring program. The release of NLCD2019 extends the database to 18 years. We collected land cover reference data for the 2016 and 2019 components of the NLCD2019 database at Level II and Level I of the classification hierarchy. For both dates, Level II land cover overall accuracies (OA) were 77.5% ± 1% (± value is the standard error) when agreement was defined as a match between the map label and primary reference label only and increased to 87.1% ± 0.7% when agreement was defined as a match between the map label and either the primary or alternate reference label. At Level I of the classification hierarchy, land cover OA was 83.1% ± 0.9% for both 2016 and 2019 when agreement was defined as a match between the map label and primary reference label only and increased to 90.3% ± 0.7% when agreement also included the alternate reference label. The Level II and Level I OA for the 2016 land cover in the NLCD2019 database were 5% higher compared to the 2016 land cover component of the NLCD2016 database when agreement was defined as a match between the map label and primary reference label only. No improvement was realized by the NLCD2019 database when agreement also included the alternate reference label. User’s accuracies (UA) for forest loss and grass gain were 70% when agreement included either the primary or alternate label, and UA was generally 50% for all other change themes. Producer’s accuracies (PA) were 70% for grass loss and gain and water gain and generally 50% for the other change themes.

This data set includes an accuracy assessment of the repeatability of identifying heavy to severe tree canopy dieback in virtual plots located within the Ohia Dieback 83,603 hectare study area.