StreetSurfaceVis

StreetSurfaceVis is an image dataset containing 9,122 street-level images from Germany with labels on road surface type and quality. The CSV file streetSurfaceVis_v1_0.csv contains all image metadata and four folders contain the image files. All images are available in four different sizes, based on the image width, in 256px, 1024px, 2048px and the original size.Folders containing the images are named according to the respective image size. Image files are named based on the mapillary_image_id.

You can find the corresponding publication here: StreetSurfaceVis: a dataset of crowdsourced street-level imagery with semi-automated annotations of road surface type and quality

Image metadata

Each CSV record contains information about one street-level image with the following attributes:

mapillary_image_id: ID provided by Mapillary (see information below on Mapillary)

user_id: Mapillary user ID of contributor

user_name: Mapillary user name of contributor

captured_at: timestamp, capture time of image

longitude, latitude: location the image was taken at

train: Suggestion to split train and test data. True for train data and False for test data. Test data contains data from 5 cities which are excluded in the training data.

surface_type: Surface type of the road in the focal area (the center of the lower image half) of the image. Possible values: asphalt, concrete, paving_stones, sett, unpaved

surface_quality: Surface quality of the road in the focal area of the image. Possible values: (1) excellent, (2) good, (3) intermediate, (4) bad, (5) very bad (see the attached Labeling Guide document for details)

Image source

Images are obtained from Mapillary, a crowd-sourcing plattform for street-level imagery. More metadata about each image can be obtained via the Mapillary API . User-generated images are shared by Mapillary under the CC-BY-SA License.

For each image, the dataset contains the mapillary_image_id and user_name. You can access user information on the Mapillary website by https://www.mapillary.com/app/user/ and image information by https://www.mapillary.com/app/?focus=photo&pKey=

If you use the provided images, please adhere to the terms of use of Mapillary.

Instances per class

Total number of images: 9,122

excellent good intermediate bad very bad

asphalt 971 1697 821

246

concrete 314 350 250

58

paving stones 385 1063 519

70

sett

129 694

540

unpaved

-

326 387 303

For modeling, we recommend using a train-test split where the test data includes geospatially distinct areas, thereby ensuring the model's ability to generalize to unseen regions is tested. We propose five cities varying in population size and from different regions in Germany for testing - images are tagged accordingly.

Number of test images (train-test split): 776

Inter-rater-reliablility

Three annotators labeled the dataset, such that each image was annotated by one person. Annotators were encouraged to consult each other for a second opinion when uncertain.1,800 images were annotated by all three annotators, resulting in a Krippendorff's alpha of 0.96 for surface type and 0.74 for surface quality.

Recommended image preprocessing

As the focal road located in the bottom center of the street-level image is labeled, it is recommended to crop images to their lower and middle half prior using for classification tasks.

This is an exemplary code for recommended image preprocessing in Python:

from PIL import Imageimg = Image.open(image_path)width, height = img.sizeimg_cropped = img.crop((0.25 * width, 0.5 * height, 0.75 * width, height))

License

CC-BY-SA

Citation

If you use this dataset, please cite as:

Kapp, A., Hoffmann, E., Weigmann, E. et al. StreetSurfaceVis: a dataset of crowdsourced street-level imagery annotated by road surface type and quality. Sci Data 12, 92 (2025). https://doi.org/10.1038/s41597-024-04295-9

@article{kapp_streetsurfacevis_2025, title = {{StreetSurfaceVis}: a dataset of crowdsourced street-level imagery annotated by road surface type and quality}, volume = {12}, issn = {2052-4463}, url = {https://doi.org/10.1038/s41597-024-04295-9}, doi = {10.1038/s41597-024-04295-9}, pages = {92}, number = {1}, journaltitle = {Scientific Data}, shortjournal = {Scientific Data}, author = {Kapp, Alexandra and Hoffmann, Edith and Weigmann, Esther and Mihaljević, Helena}, date = {2025-01-16},}

This is part of the SurfaceAI project at the University of Applied Sciences, HTW Berlin.

• Prof. Dr. Helena Mihajlević- Alexandra Kapp- Edith Hoffmann- Esther Weigmann

Contact: surface-ai@htw-berlin.de

https://surfaceai.github.io/surfaceai/

Funding: SurfaceAI is a mFund project funded by the Federal Ministry for Digital and Transportation Germany.

Im Rahmen des Forschungsprojekts SurfaceAI wurde eine Pipeline entwickelt, die auf der Grundlage von offen verfügbaren Straßenbildern ganze Straßennetze mittels neuronaler Netze bezüglich Oberflächenbelag und dessen Qualität klassifiziert. Der Quellcode kann hier eingesehen werden: https://github.com/SurfaceAI/road_network_classification. Das Projekt wird von November 2023 bis Ende Februar 2025 vom Programm mFUND des Bundesministeriums für Digitales und Verkehr (BMDV) gefördert. Ziel des Projekts ist es, eine datengestützte Grundlage zur Bewertung und Verbesserung der Straßeninfrastruktur zu schaffen. Der hier veröffentlichte Datensatz enthält beispielhaft die Ergebnisse für vier ausgewählte Städte/Stadtteile: München, Berlin-Neukölln, Osnabrück und Dresden.

Die Pipeline verwendet Bilder von der Crowdsourcing-Plattform Mapillary (https://www.mapillary.com) und klassifiziert diese mittels Convolutional Neural Networks (CNN). Panoramabilder wurden hierbei nicht berücksichtigt.

Für die Oberflächenklassifizierung stehen folgende Kategorien für den Belag zur Verfügung: Asphalt, Beton, Kopfsteinpflaster, Pflasterstein, unbefestigt (asphalt, concrete, sett, paving_stones, unpaved).

Desweiteren wird die Oberflächenqualität auf einer kontinuierlichen Skala von 1 (exzellent) bis 5 (sehr schlecht) bewertet. Detaillierte Informationen zu den Klassen stehen hier zur Verfügung: https://github.com/SurfaceAI/dataset_creation/blob/main/documentation/labeling_guide.md. (Anmerkung: die Skala von 1 bis 5 wurde für den Trainingsdatensatz verwendet. Das CNN-Modell kann potenziell Werte < 1 und > 5 zurückgeben. Diese wurden nicht bereinigt.)

Die einzeln klassifizierten Bilder werden anschließend dem OpenStreetMap(OSM)-Straßennetz zugeordnet und auf den OSM Segmenten aggregiert. Zusätzlich wurde in einer zweiten Variante feingliedriger aggregiert, indem OSM-Segmente in (max.) 60m-Subsegmente zerteilt wurden. Somit werden hier für jede Stadt/jeden Stadtteil zwei Datensätze zur Verfügung gestellt. Die hier veröffentlichten Shapefiles enthalten entsprechende Liniengeometrien mit den aggregierten Oberflächenklassifikationen.

Informationen zu einzelnen Attributen des Datensatzes sind in der Datenmodell-Beschreibung (Tab: Dateien und Inhalte) hinterlegt.

Die Datensätze wurden am 16.12.2024 erstellt.

Detaillierte Informationen zur Pipeline sind im GitHub Repository (https://github.com/SurfaceAI/road_network_classification) sowie in dieser Publikation zu finden:

Alexandra Kapp, Edith Hoffmann, Esther Weigmann, and Helena Mihaljević. 2024. SurfaceAI: Automated creation of cohesive road surface quality datasets based on open street-level imagery. In Proceedings of the 2nd ACM SIGSPATIAL International Workshop on Advances in Urban-AI (UrbanAI ‘24). Association for Computing Machinery, New York, NY, USA, 54–57.
https://dl.acm.org/doi/abs/10.1145/3681780.3697277

Weitere Informationen zum Trainingsdatensatz der CNN Modelle sind hier zu finden:

Alexandra Kapp, Edith Hoffmann, Esther Weigmann, and Helena Mihaljević. 2024. StreetSurfaceVis: a dataset of crowdsourced street-level imagery annotated by road surface type and quality. arXiv preprint arXiv:2407.21454. https://arxiv.org/abs/2407.21454

Trainingsdatensatz StreetSurfaceVis: https://zenodo.org/records/11449977