This dataset contains synthetic weather data generated for ten different locations, including New York, Los Angeles, Chicago, Houston, Phoenix, Philadelphia, San Antonio, San Diego, Dallas, and San Jose. The data includes information about temperature, humidity, precipitation, and wind speed, with 1 million data points generated for each parameter.

Features:

• Location: The city where the weather data was simulated.
• Date_Time: The date and time when the weather data was recorded.
• Temperature_C: The temperature in Celsius at the given location and time.
• Humidity_pct: The humidity in percentage at the given location and time.
• Precipitation_mm: The precipitation in millimeters at the given location and time.
• Wind_Speed_kmh: The wind speed in kilometers per hour at the given location and time.

Additional Information:

• Variability and Complexity: The dataset incorporates variability and complexity to simulate realistic weather patterns. For example, adjustments have been made to temperature and precipitation based on seasonal variations observed in certain locations. In New York, higher temperatures and precipitation are simulated during the summer months, while in Phoenix, lower temperatures and increased precipitation are simulated during the winter months.
• Data Generation Method: The dataset was generated using Python's Faker library to create synthetic weather data for each location. Random values within realistic ranges were generated for temperature, humidity, precipitation, and wind speed, with adjustments made to reflect seasonal variations.

Potential Use Cases:

• Weather Prediction Models: Researchers and data scientists can use this dataset to develop and train weather prediction models for various locations.
• Climate Studies: The dataset can be used for climate studies and analysis to understand weather patterns and trends in different regions.
• Educational Purposes: Students and educators can use this dataset to learn about data analysis, visualization, and modeling techniques in the context of weather data.

Acknowledgements:

• This dataset was generated using Python's Faker library.
• Special thanks to the Faker library developers for providing tools to create synthetic data for various purposes.

Image Credits :

Image by Mohamed Hassan from Pixabay

The dataset contains weather observation for the period from 01.07.2024 to 30.06.2025.

It can be used for: - Exploratory Data Analysis and Data Visualization - simple Machine Learning (e.g. Classification, Regression, Clustering),

The dataset contains: - observation dates, - temperature registered - day and night (Celsius), - air pressure (hPa), - wind speed (km/hour) and direction, - outlook (cloudy, rainy etc.).

There are two versions: clean and dirty (for Data Cleaning).

The data was artificially generated (the values should represent typical weather situation for Krakow, Poland).

📌 About this Dataset

This dataset provides real-time weather data collected from 78 cities across 63 provinces of Vietnam, reflecting the administrative boundaries prior to the provincial merger on June 12, 2025. The data is updated multiple times per day and compiled into a unified CSV file.

🧭 Motivation

Vietnam has diverse climate zones, yet city-level weather data is often underrepresented in global datasets.
This project aims to provide an open, granular, and continuously updated weather dataset for research, visualization, and machine learning applications.

📥 Source

• API: OpenWeatherMap API
• Method: Automated data collection using Python every 2–4 minutes for each city
• Author: Lê Duy Hải

🧾 License

This dataset is released under the CC0: Public Domain license.
You are free to use, modify, and distribute it without restrictions.

🔄 Update Plan

• Data is updated continuously without time limits.
• In rare cases, temporary interruptions may occur due to system issues. These gaps are natural in real-world data collection, and I always strive to fix them as quickly as possible to ensure reliable and transparent data.
• New versions will be released regularly.

🧪 Suggested Use Cases

• Weather pattern analysis in Vietnam
• Geospatial visualizations (city/province level)
• Time-series forecasting (LSTM, Prophet, etc.)
• Cross-city climate comparison

🙌 Community

This is a personal project maintained and shared openly.
If you find it useful, please consider upvoting so more people can discover and benefit from the dataset.
Your support is the biggest motivation for me to keep this project alive and growing. 🚀

This is a weather dataset. The dataset has 3 different features. There are 364 entries in the dataset. The dataset has Humidity and Temperature features, which can be used for time series analysis as well as forecasting.

The U.S. Geological Survey, in cooperation with the California Department of Water Resources (DWR), has constructed a new spatially distributed Precipitation-Runoff Modeling System (PRMS) for the Merced River Basin (Koczot and others, 2021), which is a tributary of the San Joaquin River in California. PRMS is a deterministic, distributed-parameter, physical-process-based modeling system developed to evaluate the response of streamflow and basin hydrology to various combinations of climate and land use (Markstrom and others, 2015). Although further refinement may be required to apply the Merced PRMS for official streamflow forecast operations, this application of PRMS is calibrated with intention to simulate (and eventually, forecast) year-to-year variations of inflows to Lake McClure during the critical April–July snowmelt season, and may become part of a suite of methods used by DWR for forecasting streamflow in and from the basin. The Merced application of PRMS is a high-resolution model defined spatially by discreet, georeferenced mapping units (i.e., "hydrologic response units"; HRUs). Daily inputs of precipitation, maximum and minimum temperatures are used to force the application. This application is designed to capture the effects of land use and climate change on streamflows and general hydrogeology from subareas of the model domain. As described in detail in Koczot and others (2021), simulations were calibrated against (1) solar radiation, (2) potential evapotranspiration, and (3) at 5 nodes representing locations of measured or reconstructed (at the outlet) streamflows. This application uses the PRMS 4.0.2 executable. Users should review the performance of this model to ensure applicability for their specific purpose. The PRMS application developed for this study can be operated through a customized Object User Interface (OUI; Markstrom and Koczot, 2008) coupled with a version of the Ensemble Streamflow Prediction (ESP; Day, 1985) forecasting tool, parameter-file editor, and data visualization tools. Furthermore, this includes daily-climate distribution preprocessing tools (Draper Climate-Distribution Software; Donovan and Koczot, 2019). Hereafter referred to as Merced OUI, this framework is the platform used to operate the Merced River Basin PRMS and perform streamflow simulations and forecasts.

This folder, titled "Data," contains the MATLAB code, final products, tables, and figures used in Parker, L.E., Zhang, N., Abatzoglou, J.T. et al. A variety-specific analysis of climate change effects on California winegrapes. Int J Biometeorol 68, 1559–1571 (2024). https://doi.org/10.1007/s00484-024-02684-8

Data Collection: Climatological data (daily maximum and minimum temperatures, precipitation, and reference evapotranspiration) were obtained from the gridMET dataset for the contemporary period (1991-2020) and from 20 global climate models (GCMs) for the mid-21st century (2040-2069) under RCP 4.5.Phenology Modeling: Variety-specific phenology models were developed using published climatic thresholds to assess chill accumulation, budburst, flowering, veraison, and maturity stages for the six winegrape varieties.Agroclimatic Metrics: Fourteen viticulturally important agroclimatic metrics were calculated, including Growing Degree Days (GDD), Cold Hardiness, Chilling Degree Days (CDD), Frost Damage Days (FDD), and others.Analysis Tools: MATLAB was used for data processing, analysis, and visualization. The MATLAB code provided in this dataset includes scripts for analyzing climate data, running phenology models, and generating visualizations.MATLAB Code: Scripts and functions used for data analysis and modeling.Processed Data: Results from phenology and agroclimatic analyses, including the projected changes in phenological stages and climate metrics for the selected varieties and AVAs.Tables: Detailed results of phenological changes and climate metrics, presented in a clear and structured format.Figures: Visual representations of the data and results, including charts and maps illustrating the impacts of climate change on winegrape development stages and agroclimatic conditions.

Research Description: This study investigates the impacts of climate change on the phenology and agroclimatic metrics of six winegrape varieties (Cabernet Sauvignon, Chardonnay, Pinot Noir, Zinfandel, Pinot Gris, Sauvignon Blanc) across multiple California American Viticultural Areas (AVAs). Using climatological data and phenology models, the research quantifies changes in key development stages and viticulturally important climate metrics for the mid-21st century.

Year:

Type: Numeric Description: Represents the year of the recorded data. This column is useful for time series analysis and observing trends over the years. Country:

Type: Categorical (Factor) Description: Indicates the country where the data was collected. This column helps in comparing data across different countries. Region:

Type: Categorical (Factor) Description: Specifies the region within the country. Useful for more granular analysis within countries, such as regional climate differences or yield variations. Crop_Type:

Type: Categorical (Factor) Description: Identifies the type of crop being analyzed (e.g., wheat, rice, corn). Helps in analyzing the impact of environmental factors on different crops. Average_Temperature_C:

Type: Numeric Description: The average temperature (in degrees Celsius) recorded during the growing season. Important for studying the impact of temperature on crop yield. Total_Precipitation_mm:

Type: Numeric Description: Total precipitation (in millimeters) during the growing season. Essential for understanding the effect of rainfall on crop growth and yield. CO2_Emissions_MT:

Type: Numeric Description: CO2 emissions (in metric tons) associated with agricultural activities or the region. Useful for studying the relationship between emissions and agricultural productivity. Crop_Yield_MT_per_HA:

Type: Numeric Description: The crop yield measured in metric tons per hectare. This is the target variable for understanding how environmental factors affect agricultural productivity. Extreme_Weather_Events:

Type: Categorical (Factor) or Numeric (Count) Description: Indicates the presence or number of extreme weather events (e.g., droughts, floods) that occurred during the growing season. Key for studying the impact of weather extremes on crop yield. Irrigation_Access_%:

Type: Numeric Description: Percentage of the crop area that has access to irrigation. This column helps in evaluating the impact of irrigation on crop yields and mitigating climate effects.

According to Cognitive Market Research, the Global Weather Forecast System market size 2025 was XX Million. Weather Forecast System Industry compound annual growth rate (CAGR) will be XX% from 2025 till 2033. Market Dynamics of Weather Forecast System Market

Key Drivers for Weather Forecast System Market

Technological Advancements Propel Market Growth 

The growth of technological advancements in weather forecasting is driving the market demand for weather forecast systems. A major factor contributing to this growth is the increasing use of artificial intelligence, machine learning, and big data analytics in weather prediction models. These technologies are improving the accuracy and reliability of weather forecasts, presenting new opportunities for industries that require precise weather information for efficient operation. For instance, in March 2023, The Weather Company introduced MAXimum Earth, a 3D visualization technology that provides immersive, interactive weather maps to enhance forecasting accuracy for industries like agriculture, aviation, and energy. (Source:https://www.weathercompany.com/blog/introducing-maximum-earth/) Modern advancements in meteorological instruments, satellite systems, and geospatial analytics are enhancing the accuracy of weather predictions. The use of big data and machine learning is transforming how weather patterns are monitored and forecasted, allowing for better preparation for extreme weather events. For instance, in May 2023, Tuvalu launched an AI-powered weather forecasting system with Atmo, Inc. and the Global Centre for Climate Mobility to provide more accurate and detailed forecasts, aiding in climate resilience efforts. (Source:https://www.businesswire.com/news/home/20240524276058/en/Tuvalu-Launches-AI-Weather-Forecasting-System-with-Atmo-Inc.-and-the-Global-Centre-for-Climate-Mobility)

Key Restraint for Weather Forecast System Market

Rising Infrastructure and Operational Costs to Hamper Market Growth 

The weather forecasting industry faces a significant challenge due to the high infrastructure and operational costs associated with advanced weather prediction technologies. These costs are needed for substantial investments in satellites, radar networks, supercomputers, and AI-driven systems to enhance forecasting accuracy. Maintaining weather stations, satellite networks, and data centers is expensive, limiting access for smaller enterprises and developing nations. For instance, in July 2023, NOAA highlighted the challenge of fragmented AI efforts in atmospheric sciences, with insufficient workforce training, lack of curated datasets, and limited resources hindering the development of advanced weather forecasting technologies. (Source:https://sab.noaa.gov/wp-content/uploads/SAB_MtgPres_Jul2023_John_Williams.pdf)
In response, organizations and countries with greater financial resources continue to dominate the use of sophisticated weather forecasting tools, leaving smaller players struggling to access similar levels of predictive insight and accuracy. This disparity further widens the gap between developed and developing nations, limiting the ability of smaller businesses and governments to effectively plan and mitigate weather-related risks, leading to higher operational costs. For instance, in May 2023, the World Bank reported that developing nations are struggling to adopt advanced weather forecasting technologies due to the unaffordability of infrastructure investments, limiting their climate resilience. (Source:https://openknowledge.worldbank.org/server/api/core/bitstreams/127de8c7-d367-59ac-9e54-27ee52c744aa/content)
Introduction of the Weather Forecast System Market

A Weather Forecast System refers to the process of predicting atmospheric conditions using advanced technologies, meteorological data, and analytical models to provide accurate weather information for various industries. These services utilize satellite observations, radar systems, numerical weather prediction (NWP) models, AI, and machine learning to analyze temperature, humidity, wind patterns, and precipitation trends. The demand for advanced weather technologies is expanding significantly. Recent reports highlight the increasing need for personalized, real-time updates in broadcasts. For instance, in April 2023, AccuWeather unveiled its new WeatherShow Enhancer at NAB 2023, a tool design...

Content:

Key meteorological parameters like mean, maximum, and minimum temperatures, precipitation, wind speed, wind gusts, and sunshine duration was all included in this dataset, which displays comprhensive daily weather records for Argentina. The information is arranged methodically to shed light on Argentina's changing climate over time. It can be applied to environmental studies, agricultural planning, trend analysis, climate research, and forecasting. A thorough grasp of local and regional weather patterns is made possible by the dataset's provision of both temperature extremes and atmospheric conditions.

Context

Argentina has a variety of climates, from cold, temperate climates in the south to subtropical areas in the north. Planning for natural disasters, managing water resources, assisting agriculture, and keeping an eye on climate change all depend on accurate weather data. Resarchers, decision-makers, and analysts can examine long-term climate behavior, seasonal variations, and extreme weather events thanks to this dataset, which offers an organize record of Argentina's meteorological parameters.

Dataset Structure:

Rows: Daily records of Argentina’s weather data

Columns: 9 variables

Key Columns:

Country: Country name (Argentina)

Date: Observation date (DD-MM-YYYY format)

Temp_Max: Maximum daily temperature (°C)

Temp_Min: Minimum daily temperature (°C)

Temp_Mean :Mean daily temperature (°C)

Precipitation_Sum: Total daily precipitation (mm)

Windspeed_Max:Maximum daily windspeed (km/h or m/s)

Windgusts_Max: Maximum daily wind gusts (km/h or m/s)

Sunshine_Duration:Duration of sunshine (in seconds or minutes)

Acknowledgement:

As a valuable resource for analyzing and learning

Types of Dataset:

Categorical: Country

Temporal: Date

Numerical (Continuous): Temp_Max, Temp_Min, Temp_Mean, Precipitation_Sum, Windspeed_Max, Windgusts_Max, Sunshine_Duration

Total: 1 categorical, 1 temporal, 7 numerical

This service is available to all ArcGIS Online users with organizational accounts. For more information on this service, including the terms of use, visit us online at http://goto.arcgisonline.com/earthobs2/REMSS_SeaSurfaceTempSea Surface Temperature is a key climate and weather measurement used for weather prediction, ocean forecasts, tropical cyclone forecasts, and in coastal applications such as fisheries, pollution monitoring and tourism. El Niño and La Niña are two examples of climate events which are forecast through the use of sea surface temperature maps. The Naval Oceanographic Office sea surface temperature dataset is calculated from satellite-based microwave and infrared imagery. These data are optimally interpolated to provide a daily, global map of the midday (12:00 pm) sea surface temperature. Learn more about the source data. Phenomenon Mapped: Sea Surface TemperatureUnits: Degrees CelsiusTime Interval: DailyTime Extent: 2008/04/01 12:00:00 UTC to presentCell Size: 11 kmSource Type: ContinuousPixel Type: Floating PointData Projection: GCS WGS84Mosaic Projection: Web Mercator Auxiliary SphereExtent: Global OceansSource: Naval Oceanographic OfficeUpdate Cycle: SporadicArcGIS Server URL: http://earthobs2.arcgis.com/arcgisTime: This is a time-enabled layer. It shows the average sea surface temperature during the map's time extent, or if time animation is disabled, a time range can be set using the layer's multidimensional settings. The map shows the average of all days in the time extent. Minimum temporal resolution is one day; maximum is one month.What can you do with this layer?Visualization: This layer can be used for visualization online in web maps and in ArcGIS Desktop.Analysis: This layer can be used as an input to geoprocessing tools and model builder. Units are in degrees Celsius, and there is a processing template to convert pixels to Fahrenheit. See this Esri blog post for more information on how to use this layer in your analysis. Do not use this layer for analysis while the Cartographic Renderer processing template is applied.This layer is part of the Living Atlas of the World that provides an easy way to explore the earth observation layers and many other beautiful and authoritative maps on hundreds of topics.

StormGPT Environmental Visualization Dataset

Overview

Professional-grade environmental intelligence visualizations generated using AI and real-time data integration. This dataset includes sample outputs demonstrating advanced geospatial analysis, weather forecasting, disaster impact modeling, and compliance monitoring capabilities.

What's Included

Visual Outputs (PNG Images)

• Hurricane Tracking & Impact Zones - Real-time storm position with projected impact areas
• Flood Impact Maps - Inundation modeling with infrastructure overlay
• Tsunami Simulations - Coastal impact visualization with evacuation zones
• 7-Day Weather Forecasts - City-specific predictions with precipitation probability
• Rainfall Analysis - Hourly precipitation graphs with compliance triggers
• 3D Earth Visualizations - Global weather systems and environmental events

Technical Documentation (CSV)

Complete catalog of 40+ technical capabilities used to generate these visualizations including Cesium 3D engine features, Leaflet mapping components, real-time data integrations (NOAA, USGS, NASA, EPA), and predictive analytics.

Use Cases

• Emergency Management - Visualize hurricane paths, flood zones, tsunami warnings
• Environmental Consulting - Create professional reports with custom visualizations
• Software Development - Reference implementation for environmental platforms
• Research & Education - Teaching materials for environmental science

Data Sources

Integrates data from NOAA, USGS, NASA, EPA, FEMA, and Cesium Ion

Applications

Perfect for stormwater compliance (CGP/MSGP/MS4), disaster response, infrastructure planning, and media communication.

License

CC BY-SA 4.0 - Commercial use allowed with attribution

Sea Surface Temperature is a key climate and weather measurement used for weather prediction, ocean forecasts, tropical cyclone forecasts, and in coastal applications such as fisheries, pollution monitoring and tourism. El Niño and La Niña are two examples of climate events which are forecast through the use of sea surface temperature maps. The Naval Oceanographic Office sea surface temperature dataset is calculated from satellite-based microwave and infrared imagery. These data are optimally interpolated to provide a daily, global map of the midday (12:00 pm) sea surface temperature. Learn more about the source data. Phenomenon Mapped: Sea Surface TemperatureUnits: Degrees CelsiusTime Interval: DailyTime Extent: 2008/04/01 12:00:00 UTC to presentCell Size: 11 kmSource Type: ContinuousPixel Type: Floating PointData Projection: GCS WGS84Mosaic Projection: Web Mercator Auxiliary SphereExtent: Global OceansSource: Naval Oceanographic OfficeUpdate Cycle: SporadicArcGIS Server URL: https://earthobs2.arcgis.com/arcgisTime: This is a time-enabled layer. It shows the average sea surface temperature during the map's time extent, or if time animation is disabled, a time range can be set using the layer's multidimensional settings. The map shows the average of all days in the time extent. Minimum temporal resolution is one day; maximum is one month.What can you do with this layer?Visualization: This layer can be used for visualization online in web maps and in ArcGIS Desktop.Analysis: This layer can be used as an input to geoprocessing tools and model builder. Units are in degrees Celsius, and there is a processing template to convert pixels to Fahrenheit. Do not use this layer for analysis while the Cartographic Renderer processing template is applied.This layer is part of the Living Atlas of the World that provides an easy way to explore the earth observation layers and many other beautiful and authoritative maps on hundreds of topics.

Predicting The Weather In Indonesia

5 Years of Historical Data

By Humanitarian Data Exchange [source]

About this dataset

This dataset contains daily summaries on a range of precipitation indicators for Indonesia. It is administered by the National Centers for Environmental Information (NCEI), a branch of the United States government’s National Oceanic and Atmospheric Administration (NOAA). With data compiled from thousands of base stations across Indonesia, users can gain insight into weather and climate conditions over the past five years.

More Datasets

For more datasets, click here.

Featured Notebooks

• 🚨 Your notebook can be here! 🚨!

Research Ideas

• This dataset can be used to analyze weather patterns and historical precipitation amounts in Indonesia over the past five years.
• It can be used as a tool to inform agricultural decision-making, such as planting crops that are more suited to a specific climate or amount of precipitation.
• The data can also be used to analyze potential changes in the environment due to climate change, such as how much rain is expected in a certain part of Indonesia at different times of the year

Acknowledgements

If you use this dataset in your research, please credit the original authors. Data Source

License

License: CC0 1.0 Universal (CC0 1.0) - Public Domain Dedication No Copyright - You can copy, modify, distribute and perform the work, even for commercial purposes, all without asking permission. See Other Information.

Columns

File: precipitation-idn-csv-1.csv | Column name | Description | |:--------------|:----------------------------------------------------------| | date | Date of the observation. (Date) | | datatype | Type of data being observed. (String) | | station | Location of the observation. (String) | | value | Value of the observation. (Float) | | fl_miss | Flag indicating if the observation is missing. (Boolean) | | fl_cmiss | Flag indicating if the observation is complete. (Boolean) | | country | Country of the observation. (String) | | indicator | Indicator of the observation. (String) |

Acknowledgements

If you use this dataset in your research, please credit the original authors. If you use this dataset in your research, please credit Humanitarian Data Exchange.

IntroductionIt has proven a hard challenge to stimulate climate action with climate data. While scientists communicate through words, numbers, and diagrams, artists use movement, images, and sound. Sonification, the translation of data into sound, and visualization, offer techniques for representing climate data with often innovative and exciting results. The concept of sonification was initially defined in terms of engineering, and while this view remains dominant, researchers increasingly make use of knowledge from electroacoustic music (EAM) to make sonifications more convincing.MethodsThe Aesthetic Perspective Space (APS) is a two-dimensional model that bridges utilitarian-oriented sonification and music. We started with a review of 395 sonification projects, from which a corpus of 32 that target climate change was chosen; a subset of 18 also integrate visualization of the data. To clarify relationships with climate data sources, we determined topics and subtopics in a hierarchical classification. Media duration and lexical diversity in descriptions were determined. We developed a protocol to span the APS dimensions, Intentionality and Indexicality, and evaluated its circumplexity.ResultsWe constructed 25 scales to cover a range of qualitative characteristics applicable to sonification and sonification-visualization projects, and through exploratory factor analysis, identified five essential aspects of the project descriptions, labeled Action, Technical, Context, Perspective, and Visualization. Through linear regression modeling, we investigated the prediction of aesthetic perspective from essential aspects, media duration, and lexical diversity. Significant regressions across the corpus were identified for Perspective (ß = 0.41***) and lexical diversity (ß = −0.23*) on Intentionality, and for Perspective (ß = 0.36***) and Duration (logarithmic; ß = −0.25*) on Indexicality.DiscussionWe discuss how these relationships play out in specific projects, also within the corpus subset that integrated data visualization, as well as broader implications of aesthetics on design techniques for multimodal representations aimed at conveying scientific data. Our approach is informed by the ongoing discussion in sound design and auditory perception research communities on the relationship between sonification and EAM. Through its analysis of topics, qualitative characteristics, and aesthetics across a range of projects, our study contributes to the development of empirically founded design techniques, applicable to climate science communication and other fields.

Retirement Notice: This item is in mature support as of April 2024 and will be retired in December 2026. A new version of this item is available for your use. Esri recommends updating your maps and apps to use the new version.Sea Surface Temperature is a key climate and weather measurement used for weather prediction, ocean forecasts, tropical cyclone forecasts, and in coastal applications such as fisheries, pollution monitoring and tourism. El Niño and La Niña are two examples of climate events which are forecast through the use of sea surface temperature maps. The Naval Oceanographic Office sea surface temperature dataset is calculated from satellite-based microwave and infrared imagery. These data are optimally interpolated to provide a daily, global map of the midday (12:00 pm) sea surface temperature. Learn more about the source data. Phenomenon Mapped: Sea Surface TemperatureUnits: Degrees CelsiusTime Interval: Daily Time Extent: 2008/04/01 12:00:00 UTC to presentCell Size: 11 kmSource Type: ContinuousPixel Type: Floating PointData Projection: GCS WGS84Mosaic Projection: Web Mercator Auxiliary SphereExtent: Global OceansSource: Naval Oceanographic OfficeArcGIS Server URL: https://earthobs2.arcgis.com/arcgis Time: This is a time-enabled layer. It shows the average sea surface temperature during the map's time extent, or if time animation is disabled, a time range can be set using the layer's multidimensional settings. The map shows the average of all days in the time extent. Minimum temporal resolution is one day; maximum is one month. What can you do with this layer? Visualization: This layer can be used for visualization online in web maps and in ArcGIS Desktop. Analysis: This layer can be used as an input to geoprocessing tools and model builder. Units are in degrees Celsius, and there is a processing template to convert pixels to Fahrenheit. Do not use this layer for analysis while the Cartographic Renderer processing template is applied.

This repository contains the long-term connectivity and link quality dataset collected on ChirpBox over 4 months (May -- September 2021) in the city of Shanghai, China.

In addition to the dataset itself, we provide evaluation scripts for data analysis and visualization, in order to facilitate data exploration and re-use. To make it clear how to use the scripts, we provide a Jupyter notebook -- dataset.ipynb for dataset visualization. Please check the notebook viewer for a preview.

List of files:

dataset_03052021_15092021.csv

The dataset includes LoRa connectivity and link quality, as well as environmental information, collected from May 3 to September 15, 2021.

data_analysis.py

The script for dataset analysis and visualization. One can use the functions in this script to derive network-level statistics (e.g., in terms of average number of correctly-exchanged packets), link-level statistics (e.g., in terms of SNR, RSS, and PRR), and node-level statistics(e.g., in terms of number of neighbours and temperature evolution over time).

metadata_processing.py

The script for pre-processing metadata into CSV files. One can use the functions in this script to convert metadata for each measurement saved in TXT and JSON formats to CSV files that include attributes such as link quality, connectivity, and environmental information, an example of which is dataset_03052021_15092021.csv.

dataset.ipynb

The Jupiter notebook contains examples of visualization and metadata pre-processing of datasets with functions in data_analysis.py and metadata_processing.py.

topology_map.png

The node deployment map used to create topology figures. A usage example is Figure 1 shown in the notebook dataset.ipynb.

dataset_metadata.zip

The dataset metadata is stored in TXT and JSON formats. Among them, link quality, connectivity and on-board sensor data are stored in TXT files and weather information are stored in JOSN files.

README.md

The README.md explains all the files in this repository and gives some examples of how to use the provided scripts to analyze the dataset.

This package contains data, filters and visualizations from Nilsson and Dokter et al. (2019).

Files

radar_metadata.csv: Metadata for the 84 European radars considered for this study. Includes radar code (odim_code = country + odim_code_3char and alternative radar code vp_radar), radar site location (location, latitude, longitude), radar site elevation (site_altitude_asl in meters above sea level) and radar altitude range used in this study (min_height_cut_asl and max_height_cut_asl in meters above sea level).

vp.zip: Vertical profiles of birds (vp) data, processed from the radar volume data following procedures described by Dokter et al. (2011), using the vol2bird algorithm in the R package bioRad. Zip file includes vp data for the 84 European radars considered for this study from September 19 to October 9, 2016 (21 days). This time period is characterized by strong passerine migration throughout Europe. Files are organized in radar (= odim_code), date and hour directories and follow the ODIM bird profile format specification. Data can be read with the R package bioRad.

vp_processing_settings.yaml: Data selection setting for this study, based on data quality criteria. File lists for each radar the altitudes to include (include_heights), time periods to exclude (exclude_datetimes) and reasons for exclusion (comments). 70 of the 84 radars were retained after filtering.

vp_processed_70_radars_20160919_20161009.csv: Processed vp data for 70 radars. Is the result of processing vp.zip with vp_processing_settings.yaml and radar_metadata.csv using vp-processing (Desmet & Nilsson 2018). Note: includes all timestamps: day and night & those marked for exclusion (marked in exclusion_reason). This data file forms the basis for analysis in the study.

Headers are:

radar_id: odim_code of the radar

datetime: timestamp

HGHT: lower altitude of altitude bin (m above sea level)

u: bird ground speed towards east (m/s)

v: bird ground speed towards north (m/s)

dens: bird density (birds/km3)

dd: bird flight direction (degrees from north)

ff: bird ground speed (m/s)

DBZH: reflectivity factor (dBZ) in horizontal polarisation

mtr: migration traffic rate (birds/km/h)

day_night: timestamp occurs during day or night (based on sunrise/sunset)

date_of_sunset: date at sunset, with night timestamps between midnight and sunrise belonging to the previous date

exclusion_reason: reason timestamp is excluded in vp_processing_settings.yaml (if applicable). Excluded timestamps have NA values for u, v, dens, dd, ff, DBZH, and mtr.

vp_flowviz.csv: Input data for visualizations. Is the result of processing vp_processed_70_radars_20160919_20161009.csv using vp-to-flowviz.Rmd in vp-processing (Desmet & Nilsson 2018). Aggregates data in hourly bins for 200-2000m (altitude_band = 1) and above (altitude_band = 2). Only altitude band 1 is used in visualizations.

flowviz.mov: Screencast of vp_flowviz.csv visualized with Bird migration flow visualization v2 (Desmet et al. 2016, Shamoun-Baranes et al. 2016). The visualization extrapolates the migration over the entire sampling range (cropped in the screencast due to technical limitations and thus excluding the Bulgarian radar), not taking topography or water bodies into account, and shows the ground speed (length of arrows) and direction of migration over time. Note that density is not shown: low density movements can therefore appear as strong as high density movements when ground speeds are similar.

cartoviz.mov: Screencast of vp_flowviz.csv visualized as an interactive map with CARTO. Visualization shows migration density (size of circles) and mean direction (colour) over time. The interactive map is available at https://inbo.carto.com/u/lifewatch/builder/8685140f-8d8c-4d06-9e1e-25d051d43748/embed.