Doodleverse/Segmentation Zoo/Seg2Map Res-UNet models for DeepGlobe/7-class segmentation of RGB 512x512 high-res. images

These Residual-UNet model data are based on the DeepGlobe dataset

Models have been created using Segmentation Gym* using the following dataset**: https://www.kaggle.com/datasets/balraj98/deepglobe-land-cover-classification-dataset

Image size used by model: 512 x 512 x 3 pixels

classes: 1. urban 2. agricultural 3. rangeland 4. forest 5. water 6. bare 7. unknown

File descriptions

For each model, there are 5 files with the same root name:

1. '.json' config file: this is the file that was used by Segmentation Gym* to create the weights file. It contains instructions for how to make the model and the data it used, as well as instructions for how to use the model for prediction. It is a handy wee thing and mastering it means mastering the entire Doodleverse.

2. '.h5' weights file: this is the file that was created by the Segmentation Gym* function train_model.py. It contains the trained model's parameter weights. It can called by the Segmentation Gym* function seg_images_in_folder.py. Models may be ensembled.

3. '_modelcard.json' model card file: this is a json file containing fields that collectively describe the model origins, training choices, and dataset that the model is based upon. There is some redundancy between this file and the config file (described above) that contains the instructions for the model training and implementation. The model card file is not used by the program but is important metadata so it is important to keep with the other files that collectively make the model and is such is considered part of the model

4. '_model_history.npz' model training history file: this numpy archive file contains numpy arrays describing the training and validation losses and metrics. It is created by the Segmentation Gym function train_model.py

5. '.png' model training loss and mean IoU plot: this png file contains plots of training and validation losses and mean IoU scores during model training. A subset of data inside the .npz file. It is created by the Segmentation Gym function train_model.py

Additionally, BEST_MODEL.txt contains the name of the model with the best validation loss and mean IoU

References *Segmentation Gym: Buscombe, D., & Goldstein, E. B. (2022). A reproducible and reusable pipeline for segmentation of geoscientific imagery. Earth and Space Science, 9, e2022EA002332. https://doi.org/10.1029/2022EA002332 See: https://github.com/Doodleverse/segmentation_gym

**Demir, I., Koperski, K., Lindenbaum, D., Pang, G., Huang, J., Basu, S., Hughes, F., Tuia, D. and Raskar, R., 2018. Deepglobe 2018: A challenge to parse the earth through satellite images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 172-181).

Dataset

This dataset was created by hehe

Released under Data files © Original Authors

Contents

We observe that satellite imagery is a powerful source of information as it contains more structured and uniform data, compared to traditional images. Although computer vision community has been accomplishing hard tasks on everyday image datasets using deep learning, satellite images are only recently gaining attention for maps and population analysis. This workshop aims at bringing together a diverse set of researchers to advance the state-of-the-art in satellite image analysis.

To direct more attention to such approaches, we propose DeepGlobe Satellite Image Understanding Challenge, structured around three different satellite image understanding tasks. The datasets created and released for this competition may serve as reference benchmarks for future research in satellite image analysis. Furthermore, since the challenge tasks will involve "in the wild" forms of classic computer vision problems, these datasets have the potential to become valuable testbeds for the design of robust vision algorithms, beyond the area of remote sensing.

Accuracy comparisons in form of mIoU/OA on test set.

Land cover classification (LCC) is of paramount importance for assessing environmental changes in remote sensing images (RSIs) as it involves assigning categorical labels to ground objects. The growing availability of multi-source RSIs presents an opportunity for intelligent LCC through semantic segmentation, offering a comprehensive understanding of ground objects. Nonetheless, the heterogeneous appearances of terrains and objects contribute to significant intra-class variance and inter-class similarity at various scales, adding complexity to this task. In response, we introduce SLMFNet, an innovative encoder-decoder segmentation network that adeptly addresses this challenge. To mitigate the sparse and imbalanced distribution of RSIs, we incorporate selective attention modules (SAMs) aimed at enhancing the distinguishability of learned representations by integrating contextual affinities within spatial and channel domains through a compact number of matrix operations. Precisely, the selective position attention module (SPAM) employs spatial pyramid pooling (SPP) to resample feature anchors and compute contextual affinities. In tandem, the selective channel attention module (SCAM) concentrates on capturing channel-wise affinity. Initially, feature maps are aggregated into fewer channels, followed by the generation of pairwise channel attention maps between the aggregated channels and all channels. To harness fine-grained details across multiple scales, we introduce a multi-level feature fusion decoder with data-dependent upsampling (MLFD) to meticulously recover and merge feature maps at diverse scales using a trainable projection matrix. Empirical results on the ISPRS Potsdam and DeepGlobe datasets underscore the superior performance of SLMFNet compared to various state-of-the-art methods. Ablation studies affirm the efficacy and precision of SAMs in the proposed model.

Eurosat is a dataset and deep learning benchmark for land use and land cover classification. The dataset is based on Sentinel-2 satellite images covering 13 spectral bands and consisting out of 10 classes with in total 27,000 labeled and geo-referenced images.

Content

This repository contains pre-trained computer vision models, data labels, and images used in the pre-print publication "A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains":

1. ADPdevkit: a folder containing the 50 validation ("tuning") set and 50 evaluation ("segtest") set of images from the Atlas of Digital Pathology database formatted in the VOC2012 style--the full database of 17,668 images is available for download from the original website
2. VOCdevkit: a folder containing the relevant files for the PASCAL VOC2012 Segmentation dataset, with both the trainaug and test sets
3. DGdevkit: a folder containing the 803 test images of the DeepGlobe Land Cover challenge dataset formatted in the VOC2012 style
4. cues: a folder containing the pre-generated weak cues for ADP, VOC2012, and DeepGlobe datasets, as required for the SEC and DSRG methods
5. models_cnn: a folder containing the pre-trained CNN models
6. models_wsss: a folder containing the pre-trained SEC, DSRG, and IRNet models, along with dense CRF settings

More information

For more information, please refer to the following article. Please cite this article when using the data set.

@misc{chan2019comprehensive,
title={A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains},
author={Lyndon Chan and Mahdi S. Hosseini and Konstantinos N. Plataniotis},
year={2019},
eprint={1912.11186},
archivePrefix={arXiv},
primaryClass={cs.CV}
}

For the full code released on GitHub, please visit the repository at: https://github.com/lyndonchan/wsss-analysis

Contact

For questions, please contact:
Lyndon Chan
lyndon.chan@mail.utoronto.ca
http://orcid.org/0000-0002-1185-7961

This repository contains the data description and processing for the paper titled "SkySense++: A Semantic-Enhanced Multi-Modal Remote Sensing Foundation Model for Earth Observation." The code is in here

🔥🔥🔥 Last Updated on 2024.03.14 🔥🔥🔥

• [2025.3.14] updated optical images of JL-16 dataset (https://huggingface.co/datasets/KKKKKKang/JL-16)
• [2025.3.12] updated sentinel-1 images and labels of JL-16 dataset (https://zenodo.org/records/15010418)

We conduct semantic-enhanced pretraining on the RS-Semantic dataset, which consists of 13 datasets with pixel-level annotations. Below are the specifics of these datasets.

Dataset	Modalities	GSD(m)	Size	Categories	Download Link
Five Billion Pixels	Gaofen-2	4	6800x7200	24	Download
Potsdam	Airborne	0.05	6000x6000	5	Download
Vaihingen	Airborne	0.05	2494x2064	5	Download
Deepglobe	WorldView	0.5	2448x2448	6	Download
iSAID	Multiple Sensors	-	800x800 to 4000x13000	15	Download
LoveDA	Spaceborne	0.3	1024x1024	7	Download
DynamicEarthNet	WorldView	0.3	1024x1024	7	Download
	Sentinel-2*	10	32x32
	Sentinel-1*	10	32x33
Pastis-MM	WorldView	0.3	1024x1024	18	Download
	Sentinel-2*	10	32x32
	Sentinel-1*	10	32x33
C2Seg-AB	Sentinel-2*	10	128x128	13	Download
	Sentinel-1*	10	128x128
FLAIR	Spot-5	0.2	512x512	12	Download
	Sentinel-2*	10	40x40
DFC20	Sentinel-2	10	256x256	9	Download
	Sentinel-1	10	256x256
S2-naip	NAIP	1	512x512	32	Download
	Sentinel-2*	10	64x64
	Sentinel-1*	10	64x64
JL-16	Jilin-1	0.72	512x512	16	Download
	Sentinel-1*	10	40x40

* for time-series data.

We evaluate our SkySense++ on 12 typical Earth Observation (EO) tasks across 7 domains: agriculture, forestry, oceanography, atmosphere, biology, land surveying, and disaster management. The detailed information about the datasets used for evaluation is as follows.

Domain	Task type	Dataset	Modalities	GSD	Image size	Download Link	Notes
Agriculture	Crop classification	Germany	Sentinel-2*	10	24x24	Download
Foresetry	Tree species classification	TreeSatAI-Time-Series	Airborne,	0.2	304x304	Download
			Sentinel-2*	10	6x6
			Sentinel-1*	10	6x6
	Deforestation segmentation	Atlantic	Sentinel-2	10	512x512	Download
Oceanography	Oil spill segmentation	SOS	Sentinel-1	10	256x256	Download
Atmosphere	Air pollution regression	3pollution	Sentinel-2	10	200x200	Download
			Sentinel-5P	2600	120x120
Biology	Wildlife detection	Kenya	Airborne	-	3068x4603	Download
Land surveying	LULC mapping	C2Seg-BW	Gaofen-6	10	256x256	Download
			Gaofen-3	10	256x256
	Change detection	dsifn-cd	GoogleEarth	0.3	512x512	Download
Disaster management	Flood monitoring	Flood-3i	Airborne	0.05	256 × 256	Download
		C2SMSFloods	Sentinel-2, Sentinel-1	10	512x512	Download
	Wildfire monitoring	CABUAR	Sentinel-2	10	5490 × 5490	Download
	Landslide mapping	GVLM	GoogleEarth	0.3	1748x1748 ~ 10808x7424	Download
	Building damage assessment	xBD	WorldView	0.3	1024x1024	Download

* for time-series data.

Quantitative evaluation of results on DeepGlobe and Massachusetts dataset.