Author: Andrew J. Felton
Date: 11/15/2024

This R project contains the primary code and data (following pre-processing in python) used for data production, manipulation, visualization, and analysis, and figure production for the study entitled:

"Global estimates of the storage and transit time of water through vegetation"

Please note that 'turnover' and 'transit' are used interchangeably. Also please note that this R project has been updated multiple times as the analysis has updated throughout the peer review process.

#Data information:

The data folder contains key data sets used for analysis. In particular:

"data/turnover_from_python/updated/august_2024_lc/" contains the core datasets used in this study including global arrays summarizing five year (2016-2020) averages of mean (annual) and minimum (monthly) transit time, storage, canopy transpiration, and number of months of data able as both an array (.nc) or data table (.csv). These data were produced in python using the python scripts found in the "supporting_code" folder. The remaining files in the "data" and "data/supporting_data" folder primarily contain ground-based estimates of storage and transit found in public databases or through a literature search, but have been extensively processed and filtered here. The "supporting_data"" folder also contains annual (2016-2020) MODIS land cover data used in the analysis and contains separate filters containing the original data (.hdf) and then the final process (filtered) data in .nc format. The resulting annual land cover distributions were used in the pre-processing of data in python.

#Code information

Python scripts can be found in the "supporting_code" folder.

Each R script in this project has a role:

"01_start.R": This script sets the working directory, loads in the tidyverse package (the remaining packages in this project are called using the `::` operator), and can run two other scripts: one that loads the customized functions (02_functions.R) and one for importing and processing the key dataset for this analysis (03_import_data.R).

"02_functions.R": This script contains custom functions. Load this using the `source()` function in the 01_start.R script.

"03_import_data.R": This script imports and processes the .csv transit data. It joins the mean (annual) transit time data with the minimum (monthly) transit data to generate one dataset for analysis: annual_turnover_2. Load this using the
`source()` function in the 01_start.R script.

"04_figures_tables.R": This is the main workhouse for figure/table production and supporting analyses. This script generates the key figures and summary statistics used in the study that then get saved in the "manuscript_figures" folder. Note that all maps were produced using Python code found in the "supporting_code"" folder. Also note that within the "manuscript_figures" folder there is an "extended_data" folder, which contains tables of the summary statistics (e.g., quartiles and sample sizes) behind figures containing box plots or depicting regression coefficients.

"supporting_generate_data.R": This script processes supporting data used in the analysis, primarily the varying ground-based datasets of leaf water content.

"supporting_process_land_cover.R": This takes annual MODIS land cover distributions and processes them through a multi-step filtering process so that they can be used in preprocessing of datasets in python.

LifeSnaps Dataset Documentation

Ubiquitous self-tracking technologies have penetrated various aspects of our lives, from physical and mental health monitoring to fitness and entertainment. Yet, limited data exist on the association between in the wild large-scale physical activity patterns, sleep, stress, and overall health, and behavioral patterns and psychological measurements due to challenges in collecting and releasing such datasets, such as waning user engagement, privacy considerations, and diversity in data modalities. In this paper, we present the LifeSnaps dataset, a multi-modal, longitudinal, and geographically-distributed dataset, containing a plethora of anthropological data, collected unobtrusively for the total course of more than 4 months by n=71 participants, under the European H2020 RAIS project. LifeSnaps contains more than 35 different data types from second to daily granularity, totaling more than 71M rows of data. The participants contributed their data through numerous validated surveys, real-time ecological momentary assessments, and a Fitbit Sense smartwatch, and consented to make these data available openly to empower future research. We envision that releasing this large-scale dataset of multi-modal real-world data, will open novel research opportunities and potential applications in the fields of medical digital innovations, data privacy and valorization, mental and physical well-being, psychology and behavioral sciences, machine learning, and human-computer interaction.

The following instructions will get you started with the LifeSnaps dataset and are complementary to the original publication.

Data Import: Reading CSV

For ease of use, we provide CSV files containing Fitbit, SEMA, and survey data at daily and/or hourly granularity. You can read the files via any programming language. For example, in Python, you can read the files into a Pandas DataFrame with the pandas.read_csv() command.

Data Import: Setting up a MongoDB (Recommended)

To take full advantage of the LifeSnaps dataset, we recommend that you use the raw, complete data via importing the LifeSnaps MongoDB database.

To do so, open the terminal/command prompt and run the following command for each collection in the DB. Ensure you have MongoDB Database Tools installed from here.

For the Fitbit data, run the following:

mongorestore --host localhost:27017 -d rais_anonymized -c fitbit 

For the SEMA data, run the following:

mongorestore --host localhost:27017 -d rais_anonymized -c sema 

For surveys data, run the following:

mongorestore --host localhost:27017 -d rais_anonymized -c surveys 

If you have access control enabled, then you will need to add the --username and --password parameters to the above commands.

Data Availability

The MongoDB database contains three collections, fitbit, sema, and surveys, containing the Fitbit, SEMA3, and survey data, respectively. Similarly, the CSV files contain related information to these collections. Each document in any collection follows the format shown below:

{
  _id: 

The HURRECON model estimates wind speed, wind direction, enhanced Fujita scale wind damage, and duration of EF0 to EF5 winds as a function of hurricane location and maximum sustained wind speed. Results may be generated for a single site or an entire region. Hurricane track and intensity data may be imported directly from the US National Hurricane Center's HURDAT2 database. HURRECON is available in R and Python. The R version is available on CRAN as HurreconR. The model is an updated version of the original HURRECON model written in Borland Pascal for use with Idrisi (see HF025). New features include support for: (1) estimating wind damage on the enhanced Fujita scale, (2) importing hurricane track and intensity data directly from HURDAT2, (3) creating a land-water file with user-selected geographic coordinates and spatial resolution, and (4) creating plots of site and regional results. The model equations for estimating wind speed and direction, including parameter values for inflow angle, friction factor, and wind gust factor (over land and water), are unchanged from the original HURRECON model. For more details and sample datasets, see the project website on GitHub (https://github.com/hurrecon-model).

The MNIST database of handwritten digits.

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('mnist', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

https://storage.googleapis.com/tfds-data/visualization/fig/mnist-3.0.1.png" alt="Visualization" width="500px">

This repository provides the R-MRIO database for the years 1995–2005 of the study "A highly resolved MRIO database for analyzing environmental footprints and Green Economy Progress".

https://doi.org/10.1016/j.scitotenv.2020.142587

The code to resolve the database and the data for the years 2006–2015 are stored under the repository http://doi.org/10.5281/zenodo.3993659

The folders "R-MRIO_year" provide the following files (*.mat-files) for each year from 1995–2005: A_RMRIO: the coefficient matrix Y_RMRIO: the final demand matrix Ext_RMRIO and Ext_hh_RMRIO: the satellite matrix of the economy and the final demand TotalOut_RMRIO: the total output vector The labels of the matrices are provided by the separate folder "Labels_RMRIO "

A script for importing and indexing the RMRIO database files in Python as Pandas DataFrames can be found here:

https://github.com/jbnsn/RMRIO-database-py-import

Fashion-MNIST is a dataset of Zalando's article images consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes.

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('fashion_mnist', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

https://storage.googleapis.com/tfds-data/visualization/fig/fashion_mnist-3.0.1.png" alt="Visualization" width="500px">

This dataset contains the data, documentation, and scripts that compose the SPATIAL SCALING CHALLENGE organized in the framework of the SENSECO COST Action CA17143 “Optical synergies for spatiotemporal SENsing of Scalable ECOphysiological traits” (https://www.senseco.eu/), by the Working Group 1. “Closing the scaling gap: from leaf measurements to satellite images” (https://www.senseco.eu/working-groups/wg1-scaling-gap/). The SPATIAL SCALING CHALLENGE is an open exercise where we challenge the remote sensing community to retrieve relevant vegetation biophysical and physiological variables such as leaf chlorophyll content (C_ab), leaf area index (LAI), maximal carboxylation rate (V_cmax,25), and non-photochemical quenching (NPQ) from simulated (hyperspectral reflectance (HDRF), sun-induced chlorophyll fluorescence (F) and land surface temperature (LST)) imagery. The dataset contains the simulated remote sensing and field data, their description, and scripts in Matlab, Python, and R languages to facilitate importing and handling the data and producing the standardized outputs necessary to participate. IMPORTANT: Additional data that can be used at the discretion of the participants have been released in https://doi.org/10.5281/zenodo.6530187 The SPATIAL SCALING CHALLENGE aims at gathering the community’s expertise and knowledge to tackle the scaling problems posed by variables of different nature. These experiences will be summarized in a journal article where all the participants are invited to contribute. The exercise is internationally open. Ph.D. students, early career and senior researchers, spin-offs, and companies working in the field of remote sensing of vegetation ecophysiology are welcome to participate. STILL OPEN FOR PARTICIPATION! New deadline 31^st of October 2022. Follow all the communications and updates of the SPATIAL SCALING CHALLENGE in the RG site: https://www.researchgate.net/project/Spatial-Scaling-Challenge-COST-Action-CA17134-SENSECO-Working-Group-1.

The First Public Data Release (DR1) of Transient Host Exchange (THEx) Dataset

Paper describing the dataset: “Linking Extragalactic Transients and their Host Galaxy Properties: Transient Sample, Multi-Wavelength Host Identification, and Database Construction” (Qin et al. 2021)

The data release contains four compressed archives.

“BSON export” is a binary export of the “host_summary” collection, which is the “full version” of the dataset. The schema was presented in the Appendix section of the paper.

You need to set up a MongoDB server to use this version of the dataset. After setting up the server, you may import this BSON file into your local database as a collection using “mongorestore” command.

You may find some useful tutorials for setting up the server and importing BSON files into your local database at:

https://docs.mongodb.com/manual/installation/

https://www.mongodb.com/basics/bson

You may run common operations like query and aggregation once you import this BSON snapshot into your local database. An official tutorial can be found at:

https://docs.mongodb.com/manual/tutorial/query-documents/

There are other packages (e.g., pymongo for Python) and software to perform these database operations.

“JSON export” is a compressed archive of JSON files. Each file, named by the unique id and the preferred name of the event, contains complete host data of a single event. The data schema and contents are identical to the “BSON” version.

“NumPy export” contains a series of NumPy tables in “npy” format. There is a row-to-row correspondence across these files. Except for the “master table” (THEx-v8.0-release-assembled.npy), which contains all the columns, each file contains the host properties cross-matched in a single external catalog. The meta info and ancillary data are summarized in THEx-v8.0-release-assembled-index.npy.

There is also a THEx-v8.0-release-typerowmask.npy file, which has rows co-indexed with other files and columns named after each transient type. The “rowmask” file allows you to select a subset of events under a specific transient type.

Note that in this version, we only include cataloged properties of the confirmed hosts or primary candidates. If the confirmed host (or primary candidate) cross-matched multiple sources in a specific catalog, we only use the representative source for host properties. Properties of other cross-matched groups are not included. Finally, table THEx-v8.0-release-MWExt.npy contains the calculated foreground extinction (in magnitudes) at host positions. These extinction values have not been applied to magnitude columns in our dataset. You need to perform this correction by yourself if desired.

“FITS export” includes the same individual tables as in “NumPy export”. However, the FITS standard limits the number of columns in a table. Therefore, we do not include the “master table” in “FITS export.”

Finally, in BSON and JSON versions, cross-matched groups (under the “groups” key) are ordered by the default ranking function. Even if the first group in this list (namely, the confirmed host or primary host candidate) is a mismatched or misidentified one, we keep it in its original position. The result of visual inspection, including our manual reassignments, has been summarized under the “vis_insp” key.

For NumPy and FITS versions, if we have manually reassigned the host of an event, the data presented in these tables are also updated accordingly. You may use the “case_code” column in the “index” file to find the result of visual inspection and manual reassignment, where the flags for this “case_code” column are summarized in case-code.txt. Generally, codes “A1” and “F1” are known and new hosts that passed our visual inspection, while codes “B1” and “G1” are mismatched known hosts and possibly misidentified new hosts that have been manually reassigned.

Abstract:

This dataset presents cumulative temperature masks that identify areas that are warm enough to stimulate tick questing activity, using a temperature threshold of 6 C. A series of scripts download and process VIIRS EO imagery of Land Surface Temperature to create temperature masks every 8 days using a combination of decadal (8-day) and daily satellite data. This dataset is an updated version for 2024.

These result in Boolean masks where suitable areas according to temperature limits on I. ricinus are identified as 1 and unsuitable areas as 0. This mask can then be applied to the existing seasonal Tick model to make a more timely prediction of tick activity based on recent temperatures.

This dataset will be updated every couple of months by the end of the year 2024.

Image acquisition:

Two different products are downloaded VIIRS Land Surface Temperature/Emissivity 8-day L3 Global 1 km SIN grid (VNP21A2, version 6) and VIIRS Land Surface Temperature/Emissivity Daily L3 Global 1 km SIN grid (VNP21A1D, version 6).

Processing:The cumulative temperature mask is processed in two steps through two separate scripts:

a. Acquisition of 1km VIIRS Land Surface Temperature imagery from NASA's data repository.b. Importing of the imagery into a suitable format from which regularly updated masks are calculated.

File naming schema:

ER+ year + 8-day number (46 in total) for example: ER2433C68.tif 24 refers to the year 2024 and 33 (decadal number) in this example refers to the 22nd of September.

ERCumTemp + year + month+ day: ERCumTemp240109.jpg, 240109 refers to the 8-day starting 9th of January.

gif file presents time-series animation for a whole year.

Projection + EPSG code:Latitude-Longitude/WGS84 (EPSG: 4326)Spatial extent:Extent -32.0000000000000000,10.0000000000000000 : 68.9999999999999574,81.9999999999999716Spatial resolution:0.0083333 deg (approx. 1000 m) Temporal resolution:8-day for year 2024

Pixel values:

Suitable areas as 1 and unsuitable areas as 0

Source: VIIRS NASA :VNP21A2 and VNP21A1D

Software used:Codes for modelling are in Python The software used for map production is ESRI ArcMap 10.8

License: CC-BY-SA 4.0Processed by:ERGO (Environmental Research Group Oxford) https://ergoonline.co.uk/ for the H2020 MOOD project

Dataset presented in the article "MOTU on FHIR: A 10-year data collection on the clinical rehabilitation pathway of 1006 trans-femoral amputees".

Data has been anonymised prior the publication. The data has been standardized in Fast Healthcare Interoperability Resources (FHIR) data standard.

This work has been conducted within the framework of the MOTU++ project (PR19-PAI-P2).

This research was co-funded by the Complementary National Plan PNC-I.1 "Research initiatives for innovative technologies and pathways in the health and welfare sector” D.D. 931 of 06/06/2022, DARE - DigitAl lifelong pRevEntion initiative, code PNC0000002, CUP: (B53C22006450001) and by the Italian National Institute for Insurance against Accidents at Work (INAIL) within the MOTU++ project (PR19-PAI-P2).

Authors express their gratitude to all the AlmaHealthDB Team.

Instruction MOTU-to-FHIR Importer