Want to keep the data in your Hosted Feature Service current? Not interested in writing a lot of code?Leverage this Python Script from the command line, Windows Scheduled Task, or from within your own code to automate the replacement of data in an existing Hosted Feature Service. It can also be leveraged by your Notebook environment and automatically managed by the MNCD Tool!See the Sampler Notebook that features the OverwriteFS tool run from Online to update a Feature Service. It leverages MNCD to cache the OverwriteFS script for import to the Notebook. A great way to jump start your Feature Service update workflow! RequirementsPython v3.xArcGIS Python APIStored Connection Profile, defined by Python API 'GIS' module. Also accepts 'pro', to specify using the active ArcGIS Pro connection. Will require ArcGIS Pro and Arcpy!Pre-Existing Hosted Feature ServiceCapabilitiesOverwrite a Feature Service, refreshing the Service Item and DataBackup and reapply Service, Layer, and Item properties - New at v2.0.0Manage Service to Service or Service to Data relationships - New at v2.0.0Repair Lost Service File Item to Service Relationships, re-enabling Service Overwrite - New at v2.0.0'Swap Layer' capability for Views, allowing two Services to support a View, acting as Active and Idle role during Updates - New at v2.0.0Data Conversion capability, able to invoke following a download and before Service update - New at v2.0.0Includes 'Rss2Json' Conversion routine, able to read a RSS or GeoRSS source and generate GeoJson for Service Update - New at v2.0.0Renamed 'Rss2Json' to 'Xml2GeoJSON' for its enhanced capabilities, 'Rss2Json' remains for compatability - Revised at v2.1.0Added 'Json2GeoJSON' Conversion routine, able to read and manipulate Json or GeoJSON data for Service Updates - New at v2.1.0Can update other File item types like PDF, Word, Excel, and so on - New at v2.1.0Supports ArcGIS Python API v2.0 - New at v2.1.2RevisionsSep 29, 2021: Long awaited update to v2.0.0!Sep 30, 2021: v2.0.1, Patch to correct Outcome Status when download or Coversion resulted in no change. Also updated documentation.Oct 7, 2021: v2.0.2, workflow Patch correcting Extent update of Views when Overwriting Service, discovered following recent ArcGIS Online update. Enhancements to 'datetimeUtil' Support script.Nov 30, 2021: v2.1.0, added new 'Json2GeoJSON' Converter, enhanced 'Xml2GeoJSON' Converter, retired 'Rss2Json' Converter, added new Option Switches 'IgnoreAge' and 'UpdateTarget' for source age control and QA/QC workflows, revised Optimization logic and CRC comparison on downloads.Dec 1, 2021: v2.1.1, Only a patch to Conversion routines: Corrected handling of null Z-values in Geometries (discovered immediately following release 2.1.0), improve error trapping while processing rows, and added deprecation message to retired 'Rss2Json' conversion routine.Feb 22, 2022: v2.1.2, Patch to detect and re-apply case-insensitive field indexes. Update to allow Swapping Layers to Service without an associated file item. Added cache refresh following updates. Patch to support Python API 2.0 service 'table' property. Patches to 'Json2GeoJSON' and 'Xml2GeoJSON' converter routines.Sep 5, 2024: v2.1.4, Patch service manager refresh failure issue. Added trace report to Convert execution on exception. Set 'ignore-DataItemCheck' property to True when 'GetTarget' action initiated. Hardened Async job status check. Update 'overwriteFeatureService' to support GeoPackage type and file item type when item.name includes a period, updated retry loop to try one final overwrite after del, fixed error stop issue on failed overwrite attempts. Removed restriction on uploading files larger than 2GB. Restores missing 'itemInfo' file on service File items. Corrected false swap success when view has no layers. Lifted restriction of Overwrite/Swap Layers for OGC. Added 'serviceDescription' to service detail backup. Added 'thumbnail' to item backup/restore logic. Added 'byLayerOrder' parameter to 'swapFeatureViewLayers'. Added 'SwapByOrder' action switch. Patch added to overwriteFeatureService 'status' check. Patch for June 2024 update made to 'managers.overwrite' API script that blocks uploads > 25MB, API v2.3.0.3. Patch 'overwriteFeatureService' to correctly identify overwrite file if service has multiple Service2Data relationships.Includes documentation updates!

Author: Titus, Maxwell (mtitus@esri.com)Last Updated: 3/4/2025Intended Environment: ArcGIS ProPurpose: This Notebook was designed to automate updates for Hosted Feature Services hosted in ArcGIS Online (or ArcGIS Portal) from ArcGIS Pro and a spatial join of two live datasets.Description: This Notebook was designed to automate updates for Hosted Feature Services hosted in ArcGIS Online (or ArcGIS Portal) from ArcGIS Pro. An associated ArcGIS Dashboard would then reflect these updates. Specifically, this Notebook would:First, pull two datasets - National Weather Updates and Public Schools - from the Living Atlas and add them to an ArcGIS Pro map.Then, the Notebook would perform a spatial join on two layers to give Public Schools features information on whether they fell within an ongoing weather event or alert. Next, the Notebook would truncate the Hosted Feature Service in ArcGIS Online - that is, delete all the data - and then append the new data to the Hosted Feature ServiceAssociated Resources: This Notebook was used as part of the demo for FedGIS 2025. Below are the associated resources:Living Atlas Layer: NWS National Weather Events and AlertsLiving Atlas Layer: U.S. Public SchoolsArcGIS Demo Dashboard: Demo Impacted Schools Weather DashboardUpdatable Hosted Feature Service: HIFLD Public Schools with Event DataNotebook Requirements: This Notebook has the following requirements:This notebook requires ArcPy and is meant for use in ArcGIS Pro. However, it could be adjusted to work with Notebooks in ArcGIS Online or ArcGIS Portal with the advanced runtime.If running from ArcGIS Pro, connect ArcGIS Pro to the ArcGIS Online or ArcGIS Portal environment.Lastly, the user should have editable access to the hosted feature service to update.

The Inventory from Model (White)Model Reservoirs (Red)Mary and Co's reservoirs (Blue)NHD water (Orange)CDL water (Yellow)The Inventory from Model map (1) represents all the water found within the three areas identified the 2023 project; (2) is where the model indicates a reservoir based on the elevation difference based on the Yeager et al. (2017) reservoirs; (3) is based on the Yeager et al. (2017) reservoir boundaries which were plugged into the fifth step of the model (the water identification steps were skipped). Then the model was used to evaluate and classify them into elevation classes. NHD Water map (4) is every waterbody polygon from the National Hydrography Dataset that falls within the three study area areas. The CDL Water map (5) is every open water and aquaculture polygon vectorized from the Cropland Data Layer raster within the three study areas.This Data is a zipped, ArcGIS Pro (V3.1.2) project. The project geodatabase contains the model. In this project there are five maps representing different datasets.NOTE (2024-05-13): ISO xml metadata files added to the root of the zipped folder, describing specific items in the project, saved in ArcGIS Pro V.3.2.2.

Raw lidar data consist of positions (x, y) and intensity values. They must undergo a classification process before individual points can be identified as belonging to ground, building, vegetation, etc., features. By completing this tutorial, you will become comfortable with the following skills:Converting .zlas files to .las for editing,Reassigning LAS class codes,Using automated lidar classification tools, andUsing 2D and 3D features to classify lidar data.Software Used: ArcGIS Pro 3.3Time to Complete: 60 - 90 minutesFile Size: 57mbDate Created: September 25, 2020Last Updated: September 27, 2024

ArcGIS Pro folder directory for use with the Learn ArcGIS lesson Automate fire damage assessment with deep learning.This is an archived copy of the tutorial data and will no longer be updated. For an up-to-date version, available only in English, please see Classifying Objects Using Deep Learning in ArcGIS Pro.Data sources:Building Features: Derived from Los Angeles County's Countywide Building Outlines (2017)Training Samples: Derived from Los Angeles County's Countywide Building Outlines (2017) dataset and manually classified as "Damaged" or "Undamaged"Imagery: Courtesy of USAA

We implemented automated workflows using Jupyter notebooks for each state. The GIS processing, crucial for merging, extracting, and projecting GeoTIFF data, was performed using ArcPy—a Python package for geographic data analysis, conversion, and management within ArcGIS (Toms, 2015). After generating state-scale LES (large extent spatial) datasets in GeoTIFF format, we utilized the xarray and rioxarray Python packages to convert GeoTIFF to NetCDF. Xarray is a Python package to work with multi-dimensional arrays and rioxarray is rasterio xarray extension. Rasterio is a Python library to read and write GeoTIFF and other raster formats. Xarray facilitated data manipulation and metadata addition in the NetCDF file, while rioxarray was used to save GeoTIFF as NetCDF. These procedures resulted in the creation of three HydroShare resources (HS 3, HS 4 and HS 5) for sharing state-scale LES datasets. Notably, due to licensing constraints with ArcGIS Pro, a commercial GIS software, the Jupyter notebook development was undertaken on a Windows OS.

Over 40,000 road crossings in Maine are maintained by Maine Department of Transportation (MaineDOT) managers, emergency managers, natural resource planners, and municipalities. Resource managers need a way to quickly and comprehensively assess, during the planning stages of potential transportation-related projects, how ecological, hydrologic, and structural characteristics of bridges and culverts and their watersheds could adversely affect project schedules and budgets. Factors that are critical to evaluate and incorporate into overall assessments of project risk include basin, land-use, and climatic characteristics; vulnerability to specific events, such as floods; and complicating factors in the watershed, such as endangered species, evacuation routes, and historical sites. A Python script tool has been built for ArcGIS Pro as an automated screening tool that draws on existing geographic information system (GIS) data layers to identify potential risk factors and quantify risk scores for bridges and culverts. This tool can help resource managers quickly evaluate projects, during early planning, in terms of variables that may adversely affect schedules or budgets.

Explore time-discrete statistical climate downscaling using regression tools and a Jupyter notebook with Python to automate temperature predictions and build a time-series mosaic. This has been created for the Learn ArcGIS lesson Downscale climate data with machine learning.This is an archived copy of the tutorial data and will no longer be updated. For an up-to-date version, available only in English, please see Regression Analysis: Building a Regression Model Using ArcGIS Pro, Regression Analysis: Performing Random Forest Regression Using ArcGIS Pro, and Downscaling a Prediction Model Using ArcGIS Notebooks and ArcGIS Pro.

Spatial data layers of stream crossing point locations, cross-section polyline, centerline polyline, and bank polyline shapefiles have been developed for selected stream crossings in the Squannacook River basin, Massachusetts. The spatial data and calculated attribute values are model input data for U.S. Army Corps of Engineer’s Hydrologic Engineering Center’s River Analysis System (HEC-RAS) hydraulic models. The stream crossing point locations were derived from the North Atlantic Aquatic Connectivity Collaboration (NAACC) database. The stream channel cross-sections, centerlines, and bank polylines were derived using automated methods in a Geographic Information System (GIS) using ArcGIS Pro and Python programming language. The polyline shapefiles are Z-enabled and have elevation data derived from Light Detection and Ranging (lidar) Digital Elevation Models (DEM) for Z-coordinate vertex values in units of feet. The polyline shapefiles are also M-enabled and have profile stationing values for the M-coordinate vertex values in units of feet. The automated GIS processes delineated a series of stream channel cross-sections along lidar-derived stream centerlines and have stream channel bathymetry estimated from Massachusetts bankfull channel geometry equations (Bent and Waite, 2013). The bankfull equations were also used to derive stream bank polylines. This data release contains the following shapefiles in the Spatial_Data_Layers.zip file: 1. Stream_Crossing_Locations.shp - Esri point shapefile derived from the NAACC stream crossing database. 2. Stream_Crossing_Watersheds.shp - Esri polygon shapefile of lidar-derived watershed boundaries that estimate the upstream drainage area for each stream crossing location. 3. Model_Cross_Sections.shp - Esri Z- and M-enabled polyline shapefile of the cross-section data used for hydraulic model input. 4. Model_Flowpaths.shp - Esri Z- and M-enabled polyline shapefile of the stream centerline and stream bank line data used for hydraulic model input. References: Bent, G.C., and Waite, A.M., 2013, Equations for estimating bankfull channel geometry and discharge for streams in Massachusetts: U.S. Geological Survey Scientific Investigations Report 2013–5155, 62 p., http://dx.doi.org/10.3133/sir20135155

A Geospatial Data Resource Site (GDRS) is a structured collection of GIS apps and data used as the data structure that supports the Minnesota Geospatial Commons (MGC) and MNDNR Quick Layers (for ArcGIS Pro and ArcMap). The GDRS Manager application allows anyone to create their own local GDRS, either to be shared on an organization's network drive or to use offline. There is no installation for the GDRSManager. Simply download the application and double-click the executable file to run it. Current version: 1.0.0.44

Use https://resources.gisdata.mn.gov/pub/gdrs as the source GDRS to access the full public data from the MGC. Copy everything or select which datasets you want to copy to a local GDRS destination. If you have disk space or download speed limiations, sorting the resources by size and unselecting the largest resources is a quick way to reduce the size of a the GDRS but still have access to most of the resources.

Once you have a GDRS set up, run the application again whenever you want to update existing datasets or add additional datasets. You can even keep your data current by scheduling automatic updates to run gdrsmanager.exe through a bat file. For your reference, a template bat file gdrsmanager_console.bat is included in the application download. Just make sure to update the bat file to include the path to your copy of GDRS Manager, and schedule the bat file to run as desired.

The application download includes a comprehensive help document, which you can also access separately here:
gdrsmanager_quicklayers_Pro_help.pdf
gdrsmanager_quicklayers_help.pdf (ArcMap)

An ArcGIS Pro project used by crime analysts to automate data imports from records management or computer-aided dispatch systems, which helps crime analysts free up time typically spent on data preparation.

This dataset was updated April, 2024.This ownership dataset was generated primarily from CPAD data, which already tracks the majority of ownership information in California. CPAD is utilized without any snapping or clipping to FRA/SRA/LRA. CPAD has some important data gaps, so additional data sources are used to supplement the CPAD data. Currently this includes the most currently available data from BIA, DOD, and FWS. Additional sources may be added in subsequent versions. Decision rules were developed to identify priority layers in areas of overlap.Starting in 2022, the ownership dataset was compiled using a new methodology. Previous versions attempted to match federal ownership boundaries to the FRA footprint, and used a manual process for checking and tracking Federal ownership changes within the FRA, with CPAD ownership information only being used for SRA and LRA lands. The manual portion of that process was proving difficult to maintain, and the new method (described below) was developed in order to decrease the manual workload, and increase accountability by using an automated process by which any final ownership designation could be traced back to a specific dataset.The current process for compiling the data sources includes: Clipping input datasets to the California boundary Filtering the FWS data on the Primary Interest field to exclude lands that are managed by but not owned by FWS (ex: Leases, Easements, etc) Supplementing the BIA Pacific Region Surface Trust lands data with the Western Region portion of the LAR dataset which extends into California. Filtering the BIA data on the Trust Status field to exclude areas that represent mineral rights only. Filtering the CPAD data on the Ownership Level field to exclude areas that are Privately owned (ex: HOAs) In the case of overlap, sources were prioritized as follows: FWS > BIA > CPAD > DOD As an exception to the above, DOD lands on FRA which overlapped with CPAD lands that were incorrectly coded as non-Federal were treated as an override, such that the DOD designation could win out over CPAD.In addition to this ownership dataset, a supplemental _source dataset is available which designates the source that was used to determine the ownership in this dataset.Data Sources: GreenInfo Network's California Protected Areas Database (CPAD2023a). https://www.calands.org/cpad/; https://www.calands.org/wp-content/uploads/2023/06/CPAD-2023a-Database-Manual.pdf US Fish and Wildlife Service FWSInterest dataset (updated December, 2023). https://gis-fws.opendata.arcgis.com/datasets/9c49bd03b8dc4b9188a8c84062792cff_0/explore Department of Defense Military Bases dataset (updated September 2023) https://catalog.data.gov/dataset/military-bases Bureau of Indian Affairs, Pacific Region, Surface Trust and Pacific Region Office (PRO) land boundaries data (2023) via John Mosley John.Mosley@bia.gov Bureau of Indian Affairs, Land Area Representations (LAR) and BIA Regions datasets (updated Oct 2019) https://biamaps.doi.gov/bogs/datadownload.htmlData Gaps & Changes:Known gaps include several BOR, ACE and Navy lands which were not included in CPAD nor the DOD MIRTA dataset. Our hope for future versions is to refine the process by pulling in additional data sources to fill in some of those data gaps. Additionally, any feedback received about missing or inaccurate data can be taken back to the appropriate source data where appropriate, so fixes can occur in the source data, instead of just in this dataset.24_1: Input datasets this year included numerous changes since the previous version, particularly the CPAD and DOD inputs. Of particular note was the re-addition of Camp Pendleton to the DOD input dataset, which is reflected in this version of the ownership dataset. We were unable to obtain an updated input for tribral data, so the previous inputs was used for this version.23_1: A few discrepancies were discovered between data changes that occurred in CPAD when compared with parcel data. These issues will be taken to CPAD for clarification for future updates, but for ownership23_1 it reflects the data as it was coded in CPAD at the time. In addition, there was a change in the DOD input data between last year and this year, with the removal of Camp Pendleton. An inquiry was sent for clarification on this change, but for ownership23_1 it reflects the data per the DOD input dataset.22_1 : represents an initial version of ownership with a new methodology which was developed under a short timeframe. A comparison with previous versions of ownership highlighted the some data gaps with the current version. Some of these known gaps include several BOR, ACE and Navy lands which were not included in CPAD nor the DOD MIRTA dataset. Our hope for future versions is to refine the process by pulling in additional data sources to fill in some of those data gaps. In addition, any topological errors (like overlaps or gaps) that exist in the input datasets may thus carry over to the ownership dataset. Ideally, any feedback received about missing or inaccurate data can be taken back to the relevant source data where appropriate, so fixes can occur in the source data, instead of just in this dataset.

These interpolated groundwater levels and altitudes product, for Monroe County, WV, was derived from groundwater-level data obtained from a U.S. Geological Survey (USGS) synoptic survey of 257 groundwater wells during October 23, 2017 through September 19, 2019, and selected points from the National Hydrography Dataset (NHD) to represent equal-altitude contour lines of groundwater altitudes in 50-foot intervals. Attributes include groundwater altitudes in decimal feet. Horizontal coordinates are referenced to UTM zone 17, NAD83, and groundwater altitudes are referenced to the North American Vertical Datum of 1988 (NAVD88). The potentiometric surface map, based on the 257 groundwater measurements, was constrained by the NHD streamlines and _location of known springs. ArcGIS Pro was used to make contour lines from point data, and the resulting contours were further edited in areas where automated methods were not as precise given fewer data points; the areas edited were where the land-surface elevation was lower than water-surface elevation, approximately.

In the Prince William Sound region of Alaska, recent glacier retreat started in the mid-1800s and began to accelerate in the mid-2000s in response to warming air temperatures (Maraldo and others, 2020). Prince William Sound is surrounded by the central Chugach Mountains and consists of numerous ocean-terminating glaciers, with rapid deglaciation increasingly exposing oversteepened bedrock walls of fiords. Deglaciation may accelerate the occurrence of rapidly moving rock avalanches (RAs), which have the potential to generate tsunamis and adversely impact maritime vessels, marine activities, and coastal infrastructure and populations in the Prince William Sound region. RAs have been documented in the Chugach Mountains in the past (Post, 1967; McSaveney, 1978; Uhlmann and others, 2013), but a time series of RAs in the Chugach Mountains is not currently available. A systematic inventory of RAs in the Chugach is needed as a baseline to evaluate any future changes in RA frequency, magnitude, and mobility. This data release presents a comprehensive historical inventory of RAs in a 4600 km2 area of the Prince William Sound. The inventory was generated from: (1) visual inspection of 30-m resolution Landsat satellite images collected between July 1984 and August 2024; and (2) the use of an automated image classification script (Google earth Engine supRaglAciaL Debris INput dEtector (GERALDINE, Smith and others, 2020)) designed to detect new rock-on-snow events from repeat Landsat images from the same time period. RAs were visually identified and mapped in a Geographic Information System (GIS) from the near-infrared (NIR) band of Landsat satellite images. This band provides significant contrast between rock and snow to detect newly deposited rock debris. A total of 252 Landsat images were visually examined, with more images available in recent years compared to earlier years (Figure 1). Calendar year 1984 was the first year when 30-m resolution Landsat data were available, and thus provided a historical starting point from which RAs could be detected with consistent certainty. By 2017, higher resolution (<5-m) daily Planet satellite images became consistently available and were used to better constrain RA timing and extent. Figure 1. Diagram showing the number of usable Landsat images per year. This inventory reveals 118 RAs ranging in size from 0.1 km2 to 2.3 km2. All of these RAs occurred during the months of May through September (Figure 2). The data release includes three GIS feature classes (polygons, points, and polylines), each with its own attribute information. The polygon feature class contains the entire extent of individual RAs and does not differentiate the source and deposit areas. The point feature class contains headscarp and toe locations, and the polyline feature class contains curvilinear RA travel distance lines that connect the headscarp and toe points. Additional attribute information includes the following: location of headscarp and toe points, date of earliest identified occurrence, if and when the RA was sequestered into the glacier, presence and delineation confidence levels (see Table 1 for definition of A, B, and C confidence levels), identification method (visual inspection versus automated detection), image platform, satellite, estimated cloud cover, if the RA is lobate, image ID, image year, image band, affected area in km2, length, height, length/height, height/length, notes, minimum and maximum elevation, aspect at the headscarp point, slope at the headscarp point, and geology at the headscarp point. Topographic information was derived from 5-m interferometric synthetic aperture radar (IfSAR) Digital Elevation Models (DEMs) that were downloaded from the USGS National Elevation Dataset website (U.S. Geological Survey, 2015) and were mosaicked together in ArcGIS Pro. The aspect and slope layers were generated from the downloaded 5-m DEM with the “Aspect” and “Slope” tools in ArcGIS Pro. Aspect and slope at the headscarp mid-point were then recorded in the attribute table. A shapefile of Alaska state geology was downloaded from Wilson and others (2015) and was used to determine the geology at the headscarp location. The 118 identified RAs have the following confidence level breakdown for presence: 66 are A-level, 51 are B-level, and 1 is C-level. The 118 identified RAs have the following confidence level breakdown for delineation: 39 are A-level and 79 are B-level. Please see the provided attribute table spreadsheet for more detailed information. Figure 2. Diagram showing seasonal timing of mapped rock avalanches. Table 1. Rock avalanche presence and delineation confidence levels Category Grade Justification Presence A Feature is clearly visible in one or more satellite images. B Feature is clearly visible in one or more satellite images but has low contrast with the surroundings and may be surficial debris from rock fall, rather than from a rock avalanche. C Feature presence is possible but uncertain due to poor quality of imagery (e.g., heavy cloud cover or shadows) or lack of multiple views. Delineation A Exact outline of the feature from headscarp to toe is clear. B General shape of the feature is clear but the exact headscarp or toe location is unclear (e.g., due to clouds or shadows). Disclaimer: Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the U.S. Government. References Maraldo, D.R., 2020, Accelerated retreat of coastal glaciers in the Western Prince William Sound, Alaska: Arctic, Antarctic, and Alpine Research, v. 52, p. 617-634, https://doi.org/10.1080/15230430.2020.1837715 McSaveney, M.J., 1978, Sherman glacier rock avalanche, Alaska, U.S.A. in Voight, B., ed., Rockslides and Avalanches, Developments in Geotechnical Engineering, Amsterdam, Elsevier, v. 14, p. 197–258. Post, A., 1967, Effects of the March 1964 Alaska earthquake on glaciers: U.S. Geological Survey Professional Paper 544-D, Reston, Virgina, p. 42, https://pubs.usgs.gov/pp/0544d/ Smith, W. D., Dunning, S. A., Brough, S., Ross, N., and Telling, J., 2020, GERALDINE (Google Earth Engine supRaglAciaL Debris INput dEtector): A new tool for identifying and monitoring supraglacial landslide inputs: Earth Surface Dynamics, v. 8, p. 1053-1065, https://doi.org/10.5194/esurf-8-1053-2020 Uhlmann, M., Korup, O., Huggel, C., Fischer, L., and Kargel, J. S., 2013, Supra-glacial deposition and flux of catastrophic rock-slope failure debris, south-central Alaska: Earth Surface Processes and Landforms, v. 38, p. 675–682, https://doi.org/10.1002/esp.3311 U.S. Geological Survey, 2015, USGS NED Digital Surface Model AK IFSAR-Cell37 2010 TIFF 2015: U.S. Geological Survey, https://elevation.alaska.gov/#60.67183:-147.68372:8 Wilson, F.H., Hults, C.P., Mull, C.G, and Karl, S.M, compilers, 2015, Geologic map of Alaska: U.S. Geological Survey Scientific Investigations Map 3340, pamphlet p. 196, 2 sheets, scale 1:1,584,000, https://pubs.usgs.gov/publication/sim3340

This data is related to a paper which evaluated different methods for detecting baobab trees in drone imagery, with the aim of comparing them to archaeological sites in eastern Africa.

This dataset contains:

1. An ArcGIS Pro geodatabase, with a shapefile of hand-digitized baobab locations for Unguja Island, shapefile data of archaeological sites, shapefiles of settlement data digitized from a 1907 map of the island, squares showing seasonal data for the ZMI imagery, and an outline of the island created from SRTM elevation data, shapefiles and rasters of the automated detection training and testing files, and the results of automated detection using the Max Likelihood classifier, sorted by circularity and pixel size.
2. Excel files showing the nearness analysis for baobab trees, with the raw data and tables available
3. Excel files with confusion matrices for the automated detection in different seasons
4. Python code used for the tree detection viewer
5. A ReadMe file with all workflows described in detail

Pockmarks are defined as depressions on the seabed and are usually formed by fluid expulsions. Recently discovered, pockmarks along the Aquitaine slope within the French EEZ, were manually mapped although two semi-automated methods were tested without convincing results. In order to potentially highlight different groups and possibly discriminate the nature of the fluids involved in their formation and evolution, a morphological study was conducted, mainly based on multibeam data and in particular bathymetry from the marine expedition GAZCOGNE1, 2013. Bathymetry and seafloor backscatter data, covering more than 3200 km², were acquired with the Kongsberg EM302 ship-borne multibeam echosounder of the R/V Le Suroît at a speed of ~8 knots, operated at a frequency of 30 kHz and calibrated with ©Sippican shots. Precision of seafloor backscatter amplitude is +/- 1 dB. Multibeam data, processed using Caraibes (©IFREMER), were gridded at 15x15 m and down to 10x10 m cells, for bathymetry and seafloor backscatter, respectively. The present table includes 11 morphological attributes extracted from a Geographical Information System project (Mercator 44°N conserved latitude in WGS84 Datum) and additional parameters related to seafloor backscatter amplitudes. Pockmark occurrence with regards to the different morphological domains is derived from a morphological analysis manually performed and based on GAZCOGNE1 and BOBGEO2 bathymetric datasets. The pockmark area and its perimeter were calculated with the “Calculate Geometry” tool of Arcmap 10.2 (©ESRI) (https://desktop.arcgis.com/en/arcmap/10.3/manage-data/tables/calculating-area-length-and-other-geometric-properties.htm). A first method to calculate pockmark internal depth developed by Gafeira et al. was tested (Gafeira J, Long D, Diaz-Doce D (2012) Semi-automated characterisation of seabed pockmarks in the central North Sea. Near Surface Geophysics 10 (4):303-315, doi:10.3997/1873-0604.2012018). This method is based on the “Fill” function from the Hydrology toolset in Spatial Analyst Toolbox Arcmap 10.2 (©ESRI), (https://pro.arcgis.com/en/pro-app/tool-reference/spatial-analyst/fill.htm) which fills the closed depressions. The difference between filled bathymetry and initial bathymetry produces a raster grid only highlighting filled depressions. Thus, only the maximum filling values which correspond to the internal depths at the apex of the pockmark were extracted. For the second method, the internal pockmark depth was calculated with the difference between minimum and maximum bathymetry within the pockmark. Latitude and longitude of the pockmark centroid, minor and major axis lengths and major axis direction of the pockmarks were calculated inside each depression with the “Zonal Geometry as Table” tool from Spatial Analyst Toolbox in ArcGIS 10.2 (©ESRI) (https://pro.arcgis.com/en/pro-app/tool-reference/spatial-analyst/zonal-statistics.htm). Pockmark elongation was calculated as the ratio between the major and minor axis length. Cell count is the number of cells used inside each pockmark to calculate statistics (https://pro.arcgis.com/en/pro-app/tool-reference/spatial-analyst/zonal-geometry.htm). Cell count and minimum, maximum and mean bathymetry, slope and seafloor backscatter values were calculated within each pockmark with “Zonal Statistics as Table” tool from Spatial Analyst Toolbox in ArcGIS 10.2 (©ESRI). Slope was calculated from bathymetry with “Slope” function from Spatial Analyst Toolbox in ArcGIS 10.2 (©ESRI) and preserves its 15 m grid size (https://pro.arcgis.com/en/pro-app/tool-reference/spatial-analyst/slope.htm). Seafloor backscatter amplitudes (minimum, maximum and mean values) of the surrounding sediments were calculated within a 100 m buffer around the pockmark rim.

Dataset for the textbook Computational Methods and GIS Applications in Social Science (3rd Edition), 2023 Fahui Wang, Lingbo Liu Main Book Citation: Wang, F., & Liu, L. (2023). Computational Methods and GIS Applications in Social Science (3rd ed.). CRC Press. https://doi.org/10.1201/9781003292302 KNIME Lab Manual Citation: Liu, L., & Wang, F. (2023). Computational Methods and GIS Applications in Social Science - Lab Manual. CRC Press. https://doi.org/10.1201/9781003304357 KNIME Hub Dataset and Workflow for Computational Methods and GIS Applications in Social Science-Lab Manual Update Log If Python package not found in Package Management, use ArcGIS Pro's Python Command Prompt to install them, e.g., conda install -c conda-forge python-igraph leidenalg NetworkCommDetPro in CMGIS-V3-Tools was updated on July 10,2024 Add spatial adjacency table into Florida on June 29,2024 The dataset and tool for ABM Crime Simulation were updated on August 3, 2023, The toolkits in CMGIS-V3-Tools was updated on August 3rd,2023. Report Issues on GitHub https://github.com/UrbanGISer/Computational-Methods-and-GIS-Applications-in-Social-Science Following the website of Fahui Wang : http://faculty.lsu.edu/fahui Contents Chapter 1. Getting Started with ArcGIS: Data Management and Basic Spatial Analysis Tools Case Study 1: Mapping and Analyzing Population Density Pattern in Baton Rouge, Louisiana Chapter 2. Measuring Distance and Travel Time and Analyzing Distance Decay Behavior Case Study 2A: Estimating Drive Time and Transit Time in Baton Rouge, Louisiana Case Study 2B: Analyzing Distance Decay Behavior for Hospitalization in Florida Chapter 3. Spatial Smoothing and Spatial Interpolation Case Study 3A: Mapping Place Names in Guangxi, China Case Study 3B: Area-Based Interpolations of Population in Baton Rouge, Louisiana Case Study 3C: Detecting Spatiotemporal Crime Hotspots in Baton Rouge, Louisiana Chapter 4. Delineating Functional Regions and Applications in Health Geography Case Study 4A: Defining Service Areas of Acute Hospitals in Baton Rouge, Louisiana Case Study 4B: Automated Delineation of Hospital Service Areas in Florida Chapter 5. GIS-Based Measures of Spatial Accessibility and Application in Examining Healthcare Disparity Case Study 5: Measuring Accessibility of Primary Care Physicians in Baton Rouge Chapter 6. Function Fittings by Regressions and Application in Analyzing Urban Density Patterns Case Study 6: Analyzing Population Density Patterns in Chicago Urban Area >Chapter 7. Principal Components, Factor and Cluster Analyses and Application in Social Area Analysis Case Study 7: Social Area Analysis in Beijing Chapter 8. Spatial Statistics and Applications in Cultural and Crime Geography Case Study 8A: Spatial Distribution and Clusters of Place Names in Yunnan, China Case Study 8B: Detecting Colocation Between Crime Incidents and Facilities Case Study 8C: Spatial Cluster and Regression Analyses of Homicide Patterns in Chicago Chapter 9. Regionalization Methods and Application in Analysis of Cancer Data Case Study 9: Constructing Geographical Areas for Mapping Cancer Rates in Louisiana Chapter 10. System of Linear Equations and Application of Garin-Lowry in Simulating Urban Population and Employment Patterns Case Study 10: Simulating Population and Service Employment Distributions in a Hypothetical City Chapter 11. Linear and Quadratic Programming and Applications in Examining Wasteful Commuting and Allocating Healthcare Providers Case Study 11A: Measuring Wasteful Commuting in Columbus, Ohio Case Study 11B: Location-Allocation Analysis of Hospitals in Rural China Chapter 12. Monte Carlo Method and Applications in Urban Population and Traffic Simulations Case Study 12A. Examining Zonal Effect on Urban Population Density Functions in Chicago by Monte Carlo Simulation Case Study 12B: Monte Carlo-Based Traffic Simulation in Baton Rouge, Louisiana Chapter 13. Agent-Based Model and Application in Crime Simulation Case Study 13: Agent-Based Crime Simulation in Baton Rouge, Louisiana Chapter 14. Spatiotemporal Big Data Analytics and Application in Urban Studies Case Study 14A: Exploring Taxi Trajectory in ArcGIS Case Study 14B: Identifying High Traffic Corridors and Destinations in Shanghai Dataset File Structure 1 BatonRouge Census.gdb BR.gdb 2A BatonRouge BR_Road.gdb Hosp_Address.csv TransitNetworkTemplate.xml BR_GTFS Google API Pro.tbx 2B Florida FL_HSA.gdb R_ArcGIS_Tools.tbx (RegressionR) 3A China_GX GX.gdb 3B BatonRouge BR.gdb 3C BatonRouge BRcrime R_ArcGIS_Tools.tbx (STKDE) 4A BatonRouge BRRoad.gdb 4B Florida FL_HSA.gdb HSA Delineation Pro.tbx Huff Model Pro.tbx FLplgnAdjAppend.csv 5 BRMSA BRMSA.gdb Accessibility Pro.tbx 6 Chicago ChiUrArea.gdb R_ArcGIS_Tools.tbx (RegressionR) 7 Beijing BJSA.gdb bjattr.csv R_ArcGIS_Tools.tbx (PCAandFA, BasicClustering) 8A Yunnan YN.gdb R_ArcGIS_Tools.tbx (SaTScanR) 8B Jiangsu JS.gdb 8C Chicago ChiCity.gdb cityattr.csv ...

Methods:This lidar derivative provides information about the bare surface of the earth. The 2-foot resolution hillshade raster was produced from the 2020 Digital Terrain Model using the hillshade geoprocessing tool in ArcGIS Pro.QL1 airborne lidar point cloud collected countywide (Sanborn)Point cloud classification to assign ground points (Sanborn)Ground points were used to create over 8,000 1-foot resolution hydro-flattened Raster DSM tiles. Using automated scripting routines within LP360, a GeoTIFF file was created for each tile. Each 2,500 x 2,500 foot tile was reviewed using Global Mapper to check for any surface anomalies or incorrect elevations found within the surface. (Sanborn)1-foot hydroflattened DTM tiles mosaicked together into a 1-foot resolution mosaiced hydroflattened DTM geotiff (Tukman Geospatial)1-foot hydroflattened DTM (geotiff) resampled to 2-foot hydro-flattened DTM using Bilinear interpolation and clipped to county boundary with 250-meter buffer (Tukman Geospatial)2-foot hillshade derived from DTM using the ESRI Spatial Analyst ‘hillshade’ function The data was developed based on a horizontal projection/datum of NAD83 (2011), State Plane, Feet and vertical datum of NAVD88 (GEOID18), Feet. Lidar was collected in early 2020, while no snow was on the ground and rivers were at or below normal levels. To postprocess the lidar data to meet task order specifications and meet ASPRS vertical accuracy guidelines, Sanborn Map Company, Inc., utilized a total of 25 ground control points that were used to calibrate the lidar to known ground locations established throughout the project area. An additional 125 independent accuracy checkpoints, 70 in Bare Earth and Urban landcovers (70 NVA points), 55 in Tall Grass and Brushland/Low Trees categories (55 VVA points), were used to assess the vertical accuracy of the data. These check points were not used to calibrate or post process the data.Uses and Limitations: The hillshade provides a raster depiction of the ground returns for each 2x2 foot raster cell across Santa Clara County. The layer is useful for hydrologic and terrain-focused analysis and is a helpful basemap when analyzing spatial data in relief.Related Datasets: This dataset is part of a suite of lidar of derivatives for Santa Clara County. See table 1 for a list of all the derivatives. Table 1. lidar derivatives for Santa Clara CountyDatasetDescriptionLink to DataLink to DatasheetCanopy Height ModelPixel values represent the aboveground height of vegetation and trees.https://vegmap.press/clara_chmhttps://vegmap.press/clara_chm_datasheetCanopy Height Model – Veg Returns OnlySame as canopy height model, but does not include lidar returns labelled as ‘unclassified’ (uses only returns classified as vegetation)https://vegmap.press/clara_chm_veg_returnshttps://vegmap.press/clara_chm_veg_returns_datasheetCanopy CoverPixel values represent the presence or absence of tree canopy or vegetation greater than or equal to 15 feet tall.https://vegmap.press/clara_coverhttps://vegmap.press/clara_cover_datasheetCanopy Cover – Veg Returns OnlySame as canopy height model, but does not include lidar returns labelled as ‘unclassified’ (uses only returns classified as vegetation)https://vegmap.press/clara_cover_veg_returnshttps://vegmap.press/clara_cover_veg_returns_datasheet HillshadeThis depicts shaded relief based on the Hillshade. Hillshades are useful for visual reference when mapping features such as roads and drainages and for visualizing physical geography. https://vegmap.press/clara_hillshadehttps://vegmap.press/clara_hillshade_datasheetDigital Terrain ModelPixel values represent the elevation above sea level of the bare earth, with all above-ground features, such as trees and buildings, removed. The vertical datum is NAVD88 (GEOID18).https://vegmap.press/clara_dtmhttps://vegmap.press/clara_dtm_datasheetDigital Surface ModelPixel values represent the elevation above sea level of the highest surface, whether that surface for a given pixel is the bare earth, the top of vegetation, or the top of a building.https://vegmap.press/clara_dsmhttps://vegmap.press/clara_dsm_datasheet