This resources contains PDF files and Python notebook files that demonstrate how to create geospatial resources in HydroShare and how to use these resources through web services provided by the built-in HydroShare GeoServer instance. Geospatial resources can be consumed directly into ArcMap, ArcGIS, Story Maps, Quantum GIS (QGIS), Leaflet, and many other mapping environments. This provides HydroShare users with the ability to store data and retrieve it via services without needing to set up new data services. All tutorials cover how to add WMS and WFS connections. WCS connections are available for QGIS and are covered in the QGIS tutorial. The tutorials and examples provided here are intended to get the novice user up-to-speed with WMS and GeoServer, though we encourage users to read further on these topic using internet searches and other resources. Also included in this resource is a tutorial designed to that walk users through the process of creating a GeoServer connected resource.

The current list of available tutorials: - Creating a Resource - ArcGIS Pro - ArcMap - ArcGIS Story Maps - QGIS - IpyLeaflet - Folium

This dataset contains WebMercator tiles which contain gray-scale shaded relief (hill shades), and nothing else. The tiles have a resolution of 256×256px, suitable for web mapping libraries such as Leaflet. The hill shades are generated from SRTM altitude data, which cover the land area between 60° northern and 58° southern latitude, and which lies in the public domain. Map material without political or infrastructural features can be desirable, for example, in use cases where historical data is visualized on a map. The concrete motivation for generating this map material was the Dhimmis & Muslims project (project page, home page, GitHub, DaRUS dataset), which analyzed peaceful coexistence of religious groups in the medieval Middle East. A particular goal with creating the dataset was to have map material available under a permissive license for screenshots and publications, instead of relying on proprietary mapping services such as Mapbox. Teaser image: The hillshades of Cyprus on zoom level 9. This image is hosted externally by GitHub, but is also present in the repository as teaser.png. Coverage. The dataset covers zoom level 0 (entire world in one tile) to 12 (entire world in 4096×4096 tiles). The total size of the dataset is 22,369,621 tiles. However, of those, 19,753,304 tiles (88.3%) are empty, either because the landscape there is fully flat (i.e., on water), or because they lie fully outside the latitude range covered by the SRTM altitude data. The empty tiles are not stored. Instead, a singular placeholder file is stored in the repository, alongside a list of the empty tiles. During extraction, the placeholder empty tile can be symbolically linked in the file system to all the places where it is needed. The total size of the non-empty tiles is about 103GB. Files. Besides the placeholder file and the list of empty tiles, the repository also contains a manifest file. This file lists all non-empty tiles by the ZIP file they are contained in. The tiles themselves are grouped into ZIP files by the following schema: All tiles from levels 0 to 5 are contained in one ZIP file. All tiles of level N, N≥6 are contained in a ZIP file which is named after the tile of level N-6 (block level) that contains the tile in question, named tiles_.zip. Hence, all tiles of level 6 are contained in a singular ZIP file named tiles_6_0_0_0.zip. The tiles of level 7 are split up into four group ZIP files named tiles_7_1_{0,1}_{0,1}.zip, the tiles of level 8 into 16 group ZIP files named tiles_8_2_{0..3}_{0..3}.zip, and so on. Both the manifest file and the commands to generate the distribution of tiles on ZIP files can be generated using the linked software repository. Usage. The tile ZIP files can be downloaded and extracted. By serving the extracted directory structure in a web server, a slippy map tile server can be created. The linked software repository also contains a command-line utility that generates the required shell commands to download the ZIP files, extract them, and softlink (ln -s) the empty tiles to the appropriate places. This command-line utility can also optionally read in a GeoJSON file of an area of interest. In this case, only tiles within that area are downloaded in a higher zoom level, whereas tiles completely outside the area are only downloaded to a lower zoom level; both zoom levels are also configurable. See the documentation in the repository and the command-line utility’s help (-h) output for more details.

Major libraries used: Osmar - To import data from open street maps Leaflet, mapview, sp - Mapping the network dplyr, plyr - To manipulate data frames Data used: The data used for creating the freeway network is obtained from open street maps, whereas, the data for sensors is obtained from PeMS. Methodology Creating the map required We start by downloading the bulk osm data for California (approx. 18GB). Next, using osmar library, we extract the required data using a bounding box, demarcating the latitude and longitude boundaries. bbox = corner_bbox(-118.0042, 33.6363, -117.7226, 33.9194) Then extracting only the freeway information (links and nodes) from the resulting data. The problem here is that most of the links contain more than two nodes, which would make the incidence matrix unnecessarily large. Thus, we extract the nodes that connect two links, and map them over the links. Creating the incidence matrix To create the incidence matrix, each link was looked up for the first and last node incident on it, keeping in mind the direction. The rows represent nodes, whereas the columns represent links. +1 was assigned at the head and -1 to the tail of the link, all other entries to 0. The incidence matrix has a dimension of 2973 by 2640. Assigning links to appropriate freeways The osm data does not explicitly mention about which freeway is a particular link part of. Thus, for the freeway links a combination of "name" and "ref" tag was used to obtain the information about the freeway. Adding ramps to the Fwy data frame So far, we only have freeway links assigned to the appropriate freeways. Now, we would like to add the immediate links that go off or on the freeway, aka ramps, to the data frame Fwy. This is done by checking for each node on the freeway segment, out of the links it is incident upon, which one is tagged as "motorway-link" in the osm data. If there is such a link, it was added to the Fwy data set with appropriate freeway values. Overlapping sensors over the network The sensor data is now used to add a layer over the existing graph to help visualize the complete network. Different types of sensors are grouped separately and can be viewed as per user's choice by clicking the check boxes. The legend shows the color used for each sensor type. Hovering upon the sensor, link or node highlights their IDs. By default, main line (ML) sensors are checked. Mapping sensors to the appropriate links Currently, by visualization we can figure out the link that contains a particular sensor. As PeMS sensor metadata does not interact with the osm data, the link-sensor relation is unknown. We need to create an algorithm such that each sensor automatically gets mapped to the link using the geographical properties. The algorithm for mapping sensors to the links is as follows: For each sensor location, extract all the links on the freeway segment in the direction sensor is installed For the nodes on each link, calculate 3 distances Distance between the nodes (d1) Distance between first node and sensor (d2) Distance between last node and sensor Calculate d1 - (d2+d3), call it d4 Calculate d4 for each link, and arrange d4 in ascending order The link for which d4 is smallest and lesser than a threshold (1e-4 in this case), assign it the sensor Repeat the above steps for each sensor location Finally, the results are stored as a form of a list (linkId). For illustration purpose, 5 ML sensors on I-5, highlighted on the map, are shown below with the appropriate link chosen by the algorithm. One can verify the IDs by hovering above the links in the map and cross checking with the table that appears below. Re-mapping ramp and freeway-freeway sensors Looking closely, one would figure out that the ramp sensors (OR/FR) are located on the freeways rather than ramps. Same for freeway-Freeway (FF) sensors. This was one of the tedious challenges I encountered in this project. But with a combination of a simple algorithm and manual work, the sensors were remapped. The details are omitted in this document. In the map below, all the remapped sensors are shown on their appropriate new links. Creating the adjacency matrix To create adjacency matrix, for each link, nodes having 1 or -1 were searched in the incidence matrix. The cell corresponding to these nodes in the adjacency matrix was assigned 1, else 0. Conclusion The results of this project, namely, incidence matrix, adjacency matrix and link-sensor relation data frame were used for the network sensor error estimation algorithm. This project led to the application of the error estimation algorithm on large networks, which is expected to result in an important contribution to the field of sensor bias estimation. The aim of this script is to automate the process of directed network graph formation, i.e., creation of incidence matrix, node adjacency matrix, and map the sensors to appropriate links. The data used fo...