Abstract [Related Publication]:

Models of dispersal potential are required to predict connectivity between populations of sessile organisms. However, to date, such models do not allow for time‐varying rates of acquisition and loss of competence to settle and metamorphose, and permit only a limited range of possible survivorship curves. We collect high‐resolution observations of coral larval survival and metamorphosis, and apply a piecewise modeling approach that incorporates a broad range of temporally‐varying rates of mortality and loss of competence. Our analysis identified marked changes in competence loss and mortality rates, whose timing implicates developmental failure and depletion of energy reserves. Asymmetric demographic rates suggest more intermediate‐range dispersal, less local retention, and less long‐distance dispersal than predicted by previously‐employed non‐piecewise models. Because vital rates are likely temporally asymmetric, at least for non‐feeding broadcast‐spawned larvae, piecewise analysis of demographic rates will likely yield more reliable predictions of dispersal potential.

Usage Notes [Dryad]:

TenuisGBR2012longtermsurvival.csv
Empirical long term survival data for A. Tenuis larvae. "temp" is the temperature that the larvae were raised at, "rep" is the replicate number, "day" is the date the observation was taken, "age (h)" is the larval age (h) at the time of the observation, "age (d)" is the larval age (d) at the time of the observation, "larvae" is the number of larvae at the beginning of the experiment, "surv" is the number of larvae alive at the time of the observation.

A.tenuisGBR2012metamorphosis.csv
Empirical metamorphosis data for A. Tenuis larvae. "temp" is the temperature that the larvae were raised at, "date" is the date the observation was taken, "hour" is the hour of the day that the observation was taken, "age (h)" is the larval age (h) at the time of the observation, "age (d)" is the larval age (d) at the time of the observation, "rep" is the replicate number, "larvae" is the number of larvae at the beginning of the experiment, "meta" is the number of larvae that had metamorphosed by the time of the observation, "swimming" is the number of larvae swimming at the time of the observation.

Survival model fit R code
This code fits the best-fitting survival model to the data.
(survival model code.R)

Competence model fit R code
This code fits the best-fitting competence model to the data.
(competence model code.R)

The dataset also includes 2 README files (MS Word format) for the R codes.

Coronavirus (covid-19) in Sierra Leone

This repository contains datasets relating to coronavirus in Sierra Leone, as well as on demographic and other information from the 2015 Population and Household Census (PHC). It also includes mapping shapefiles by district, so that you can map the district-level coronavirus statistics.

See here for a full description of how the data files have been created from the source data, including the R code.

Last updated: 10 June 2020.

Context

The novel 2019 coronavirus (covid-19) arrived late to West Africa and Sierra Leone in particular. This dataset provides the number of reported cases on a district-by-district basis for Sierra Leone, as well as various additional statistics at the country level. In addition, I provide district-by-district data on demographics and households' main sources of information, both from the 2015 census. For convenience, I also provide shapefiles for mapping the 14 districts of Sierra Leone.

Content

The dataset consists of four main files, which are in the output folder. See the column descriptions below for further details.

1. Coronavirus confirmed cases by district (sl_districts_coronavirus.csv). I found the original data by looking in the static/js/data folder in the source code for covid19.mic.gov.sl, last accessed 10 June 2020. The file contains the cumulative number of confirmed coronavirus cases in the 14 districts of Sierra Leone as a time series. I have used the R tidyverse to reshape the data and ensure naming is consistent with the other data files.

2. Demographic statistics by district (sl_districts_demographics.csv). Data from the 2015 Population and Housing Census (PHC), sourced from Open Data Sierra Leone. The dataset covers the 14 districts of Sierra Leone, which increased to 16 in 2017. Last accessed 10 June 2020.

3. Main Sources of Information by district (sl_districts_info_sources.csv). Data from the 2015 Population and Housing Census (PHC), sourced from Open Data Sierra Leone. The dataset presents the main sources of information, such as television or radio, for households in the 14 districts of Sierra Leone. Last accessed 2 June 2020. I note that I have made one correction to the source data (see R code with correction here).

4. Country-wide coronavirus statistics for Sierra Leone (sl_national_coronavirus.csv). The original data also comes from covid19.mic.gov.sl, last accessed 10 June 2020. The file contains numerous statistics as time series, listed in the Column Description section below. I note that there are various potential issues in the file which I leave the user to decide how to deal with (duplicate datetimes, inconsistent statistics).

Additionally I include a set of five files with district-by-district mapping (shapefiles) and other data, unchanged from their original source. Each file is labelled in the following way: sl_districts_mapping.*. These files come from Direct Relief Open Data on ArcGIS Hub. The data also include district-level data on maternal child health attributes, which was the original context of the mapping data.

Column Descriptions

Coronavirus confirmed cases by district sl_districts_coronavirus.csv:

1. date: Date of reporting
2. district: District of Sierra Leone (based on pre-2017 administrative boundaries)
3. confirmed_cases: Cumulative number of confirmed coronavirus cases; NA if no data reported
4. decrease: Dummy variable indicating whether the number of reported cases has been revised down. NA if no reported cases on that date; 1 if there is a decrease from the last reported cases; 0 otherwise

Demographic statistics by district sl_districts_demographics.csv:

1. district: District of Sierra Leone (based on pre-2017 administrative boundaries)
2. d_code: District code
3. d_id: District id
4. total_pop: Total population in district
5. pop_share: District's share of total country population
6. t_male: Total male population
7. t_female: Total female population
8. s_ratio: (*) Sex ratio at birth (number of males for every 100 females, under the age of 1)
9. t_urban: Total urban population
10. t_rural: Total rural population
11. prop_urban: Proportion urban
12. t_h_pop: Sum of h_male and h_female
13. h_male: (?)
14. h_female: (?)
15. t_i_pop: Sum of i_male and i_female
16. i_male: (?)
17. i_female: (?)
18. working_pop: Working population
19. depend_pop: Dependent population

...

Radar Image Dataset

Dataset Structure

Face Model parameters

Load Data

Acknowledgments

We would like to thank the Rohde & Schwarz GmbH & Co. KG (Munich, Germany) for providing the radar imaging devices and technical support that made this study possible.

NDVI Data Set (1. NDVI.nc)

• Global Vegetation Greenness (NDVI) from AVHRR GIMMS-3G+, 1981-2022
• Variable: Normalized Difference Vegetation Index (NDVI)
• Area: Global (60°S ~ 70°N, -180°W ~ 180°E)
• Period: 1982-01-01 ~ 2022-12-31
• Horizontal resolution: 0.25° × 0.25° (Regridded from original 0.0833° × 0.0833°)
• Temporal resolution: Bi-monthly (1st–15th and 16th–end of each month)
• Source: https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=2187

Meteorological Data Set (2.Temperature.nc, ... , 6.Cloud_cover.nc)

• Agrometeorological indicators from 1979 to present derived from reanalysis, Copernicus Climate Change Service
• Variable: Normalized Difference Vegetation Index (NDVI)
• Area: Global (60°S ~ 70°N, -180°W ~ 180°E)
• Period: 1982-01-01 ~ 2022-12-31
• Horizontal resolution: 0.25° × 0.25° (Regridded from original 0.1° × 0.1°)
• Temporal resolution: Bi-monthly (1st–15th and 16th–end of each month)
• The meteorological data were converted from daily values to bi-monthly average values.
• Variables: 2m temperature (K), 2m relative humidity (%), 10m wind speed (m s⁻¹), Precipitation flux (mm day⁻¹), Solar radiation flux (J m⁻² day⁻¹), Cloud cover (dimensionless)
• Source: https://cds.climate.copernicus.eu/datasets/sis-agrometeorological-indicators?tab=overview

Pre-processing code (Set_data_1~3)

• Set_data_1 : Combining raw data for NDVI and checking for missing values in the original data
• Set_data_2 : Combining annual data, calculating semi-monthly averages, and setting the latitude and longitude ranges for meteorological data.
• Set_data_3 : Synchronization of latitude, longitude, and resolution between NDVI and meteorological data.

Analysis code (code1~5)

• code_1 : This script processes climate data for variables by calculating their seasonal anomalies and time-averaged values. It performs the following steps:
  
  • Monthly Mean Calculation: The script first calculates the monthly mean for each variable over a period of 41 years.
  • Semi-Monthly Mean Calculation: It then computes the semi-monthly mean by combining the monthly mean data.
  • Anomaly Calculation: The script calculates the anomaly by subtracting the semi-monthly mean from the original data.
  • Time Mean Calculation: Finally, the time-mean for the entire time period is calculated for each variable.

• code_2 : This script calculates the linear regression slope, intercept, correlation coefficient (r-value), p-value, and standard error for a given climate variable (in this case, temperature anomaly) over time at each latitude and longitude point. The steps involved are:
  • Load Data: The script loads the input NetCDF file and extracts the time and temperature anomaly (or other climate data) values.
  • Linear Regression: For each spatial point (latitude, longitude), the script performs a linear regression between time and the corresponding climate data to determine the slope, intercept, r-value, p-value, and standard error.
  • Save Results: The regression results are saved into a new NetCDF file with variables for slope, intercept, r-value, p-value, and standard error for each latitude and longitude point.

• code_3 : This script processes NDVI (Normalized Difference Vegetation Index) data by performing the following steps:
  
  • Prepare Heatmap Data: It reshapes the NDVI data into a 4D array of the shape (latitude, longitude, years, 24 months), where each year contains 24 months of data.
  • Compute NDVI Histograms: It computes histograms of the NDVI data for each latitude, longitude, and year, adjusting the NDVI values into 20 bins for analysis.
  • Save Histogram Data: The histogram data is then saved to a .npy file, which stores the data for further analysis.

• code_4 : This script performs k-means clustering on NDVI data, based on histograms of NDVI values:
  1. Load Data: It loads NDVI data and histogram data (NDVI values in bins) from files.
  2. Filter Data: It filters out regions with zero values to focus on areas of interest.
  3. Reshape Data: The data is reshaped into a 2D format to prepare for clustering.
  4. K-Means Clustering: The script applies k-means clustering to the reshaped histogram data.
  5. Mean NDVI Calculation: It calculates the mean NDVI value for each cluster by extracting values from the NDVI data.
  6. Reordering Clusters: The clusters are reordered based on their mean NDVI values.
  7. Save Results: Finally, the script saves the cluster labels and non-zero indices into separate files.

• code_5 : This script processes NDVI (Normalized Difference Vegetation Index) data by clustering and saving the data for each cluster.
  
  • Load Data
    
    • Loads NDVI slope data (slope) from a NetCDF file.
    • Loads precomputed cluster labels (cluster_labels_8.npy) and valid data locations (non_zero_indices_8.npy).
  • Save NDVI Data by Cluster
    
    • Categorizes NDVI data based on clusters.
    • Creates a 2D array for each cluster and assigns NDVI data to the corresponding cluster coordinates.
    • Saves the clustered NDVI data as .npy files for further analysis.
  • Create Directory and Execute
    
    • Creates the output directory (if it does not exist).
    • Runs the main function to save the clustered NDVI data.

Acknowledgments

This work was also supported by Global - Learning & Academic research institution for Master’s·PhD students, and Postdocs (LAMP) Program of the National Research Foundation of Korea (NRF) grant funded by the Ministry of Education (No. RS-2023-00301914).

Mitosis Detections and Mitotic Network in TCGA

This dataset contains mitosis detections, mitotic network structures, and social network analysis (SNA) measures derived from 11,161 diagnostic slides in The Cancer Genome Atlas (TCGA). Mitoses were automatically identified using the MDFS algorithm [1], and each detected mitosis was converted into a node within a mitotic network. The resulting graphs are provided in JSON format, with each file representing a single diagnostic slide.

JSON Data Format

Each JSON file contains four primary fields:

1. edge_index
   Two parallel lists representing edges between nodes. The ii-th element in the first list corresponds to the source node index, and the ii-th element in the second list is the target node index.

2. coordinates
   A list of [x, y] positions for each node (mitosis). The (x,y) coordinates can be used for spatial visualization or further spatial analyses.

3. feats
   A list of feature vectors, with each row corresponding to a node. These features include:

   • type (an integer representing mitosis type. 1: typical mitosis, 2: atypical mitosis)
   • Node_Degree (the number of nodes connected to the node)
   • Clustering_Coeff (clustering coefficient of the node)
   • Harmonic_Cen (Harmonic centrality of the node)

4. feat_names
   The names of the features in feats. The order matches the columns in each node’s feature vector.

Example JSON Snippet

{
 "edge_index": [[1, 2, 6, 10], [2, 4, 8, 11]],
 "coordinates": [[27689.0, 12005.0], [24517.0, 17809.0], ...],
 "feats": [[1.0, 0.0, 0.0, 0.0], [1.0, 1.0, 0.0, 0.115], ...],
 "feat_names": ["type", "Node_Degree", "Clustering_Coeff", "Harmonic_Cen"]
}

Loading Data into NumPy

Below is a sample Python snippet to load one JSON file, extract node coordinates and the type feature, and combine them into a single NumPy array:

import json
import numpy as np

# Path to your JSON file
json_file_path = "example_graph.json"

with open(json_file_path, 'r') as f:
  data = json.load(f)

# Convert coordinates to NumPy
coordinates = np.array(data["coordinates"])

# Identify the "type" column
feat_names = data["feat_names"]
type_index = feat_names.index("type")

# Extract features and isolate the "type" column
feats = np.array(data["feats"])
node_types = feats[:, type_index].reshape(-1, 1)

# Combine x, y, and type into a single array (N x 3)
combined_data = np.hstack([coordinates, node_types])

print(combined_data)

Building a NetworkX Graph

To visualize or analyze the network structure, you can construct a NetworkX graph as follows:

import json
import networkx as nx
import matplotlib.pyplot as plt

json_file_path = "example_graph.json"

with open(json_file_path, "r") as f:
  data = json.load(f)

# Create a NetworkX Graph
G = nx.Graph()

# Add each node with position attributes
for i, (x, y) in enumerate(data["coordinates"]):
  G.add_node(i, pos=(x, y))

# Add edges using the parallel lists in edge_index
# (Adjust for 1-based indexing if necessary)
for src, dst in zip(data["edge_index"][0], data["edge_index"][1]):
  G.add_edge(src, dst)

Visualizing mitotic network using TIAToolbox

Having TIAToolbox installed, one can easily visualize the mitotic network on their respective whole slide images using the following command:

tiatoolbox visualize --slides path/to/slides --overlays path/to/overlays

The only thing to consider is that slides and overlays (provided graph json files) should have the same name. For more information, please refer to Visualization Interface Usage - TIA Toolbox 1.5.1 Documentation.

In case of using this dataset, please cite the following publication:

@article{jahanifar2024mitosis,
 title={Mitosis detection, fast and slow: robust and efficient detection of mitotic figures},
 author={Jahanifar, Mostafa and Shephard, Adam and Zamanitajeddin, Neda and Graham, Simon and Raza, Shan E Ahmed and Minhas, Fayyaz and Rajpoot, Nasir},
 journal={Medical Image Analysis},
 volume={94},
 pages={103132},
 year={2024},
 publisher={Elsevier}
}