The monthly optimum interpolation (OI) fields are derived by a linear interpolation of the weekly OI fields to daily fields then averaging the daily values over a month. The monthly fields are in the same format and spatial resolution as the weekly fields.
The climatological monthly mean SST (January-December) was blended from ship, buoy, and bias-corrected satellite data from 1950-1979.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
The IRI Climate Data Library contains over 300 datasets from a variety of earth science disciplines and climate-related topics. It is web-accessible via a browser interface. Functions for display, analysis, and sub-setting of global climate datasets are available. Data extraction and download for multiple data and mapping formats are easily performed.
The OI sea surface temperature (SST) analysis is produced weekly (Sunday to Saturday) on a one-degree grid. The analysis uses in situ and satellite SST's plus SST's simulated by sea-ice cover. Before the analysis is computed, the satellite data is adjusted for biases using the method of Reynolds (1988) and Reynolds and Marsico (1993). A description of the OI analysis can be found in Reynolds and Smith (1994). The bias correction improves the large scale accuracy of the OI. Examples of the effect of recent corrections is given by Reynolds (1993).
For the more recent period, 1990-present, the in situ data were obtained from radio messages carried on the Global Telecommunication System. The satellite observations were obtained from operational data produced by the National Environmental Satellite, Data and Information Service (NESDIS)
During the period 1981-1989, the in situ data were obtained from the Comprehensive Ocean Atmosphere Data Set (COADS) for the 1980s. These data (see Slutz, et al., 1985, and Woodruff, et al., 1993) consist of logbook and radio reports. The satellite data were obtained from analyses of NESDIS data produced at the University of Miami's Rosentiel School of Marine and Atmospheric Sciences.
The OI analysis is done over all ocean areas. There is no analysis over land. The land values are filled by a Cressman interpolation to produce a complete grid for possible interpolation.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
The IRI Data Library is a powerful and freely accessible online data repository and analysis tool that allows a user to view, manipulate, and download over 400 climate-related data sets through a standard web browser. The Data Library contains a wide variety of publicly available data sets, including station and gridded atmospheric and oceanic observations and analyses, model-based analyses and forecasts, and land surface and vegetation data sets, from a range of sources. It includes a flexible, interactive data viewer that allows a user to visualize. multi-dimensional data sets in several combinations, create animations, and customize and download plots and maps in a variety of image formats. The Data Library is also a powerful computational engine that can perform analyses of varying complexity using an extensive array of statistical analysis tools. Online tutorials and function documentation are available to aid the user in applying these tools to the holdings available in the Data Library. Data sets and the results of any calculations performed by the user can be downloaded in a wide variety of file formats, from simple ascii text to GIS-compatible files to fully self-describing formats, or transferred directly to software applications that use the OPeNDAP protocol. This flexibility allows the Data Library to be used as a collaborative tool among different disciplines and to build new data discovery and analysis tools.
The Scripps Institution of Oceanography heat storage climatology was derived from the NODC Global Ocean Temperature/Salinity data set (1955-1988) and the Global Temperature/Salinity Pilot Project (GTSPP) data set (1989-1994). Temperatures were interpolated at 15 standard levels from 0 to 800 meters. The climatological series was based on the January 1980-December 1989 data.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
This collection consists of climate change indices including Southern Oscillation Index (SOI), North Atlantic Oscillation (NAO), sea level pressure anomalies (SLP), sea surface temperature (SST) anomalies, and global temperature. These indices are important in the monitoring of El Nino Southern Oscillation (ENSO) events.
Data sets consist of: Darwin Island Sea Level Pressure (SLP) anomalies: 1882-present Easter island Sea Level Pressure (SLP) anomalies: 1951-1995 Global Temperature Anomalies: 1867-1994 Summer Monsoon Rainfall: 1813-1998 North Atlantic Oscilation (NAO) Index: 1864-1995 El Nino Regions Sea Surface Temperature (SST) Anomalies: 1856-1991 Southern Oscillation Index (SOI): 1951-present Tahiti Sea Level Pressure (SLP) anomalies: 1951-present
Monthly mean surface pseudo-stress using variational, subjective, and objective-subjective techniques in the Indian, Pacific, and Atlantic oceans.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/, http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html, and http://iri.ldeo.columbia.edu/climate/monitoring/ipb/.
For 1856-1981 this is the analysis of Kaplan et al. [1998] which uses optimal estimation in the space of 80 empirical orthogonal functions (EOFs) in order to interpolate ship observations of the U.K. Met Office database [Parker et al. 1994]. The data after 1981 represents the projection of the NCEP OI analysis (which combines ship observations with remote sensing data) by Reynolds and Smith [1994] on the same set of 80 EOFs as used in Kaplan et al. [1998] in order to provide enhanced data quality of the former in the spatial resolution of the latter.
For more accessing the original NOAA/NCEP Reynolds OI SST data,
See: http://www.esrl.noaa.gov/psd/data/gridded/data.noaa.oisst.v2.html
Attribution-ShareAlike 4.0 (CC BY-SA 4.0)https://creativecommons.org/licenses/by-sa/4.0/
License information was derived automatically
Description: These data represent gridded monthly SST anomalies for 399 consecutive months from January 1970 through March 2003. The data were obtained from the IRI/LDEO Climate Data Library at Columbia University (http://iridl.ldeo.columbia.edu/). The data are gridded at a 2 degree by 2 degree resolution and represent anomalies from a January 1970 - December 1985 monthly (average) climatology. A more complete description can be found at http://iridl.ldeo.columbia.edu/SOURCES/.CAC/ References: In addition to Chapter 5 and 9 of Cressie and Wikle (2011), these data have been described and modeled in the following: Berliner, L.M., Wikle, C.K. and N. Cressie, (2000). Long-lead prediction of Pacific SSTs via Bayesian Dynamic Modeling. Journal of Climate, 13 , 3953-3968. Wikle, C.K. and M.B. Hooten, 2010: A general science-based framework for spatio-temporal dynamical models. Invited discussion paper for Test. 19, 417-451. Wikle, C.K. and S.H. Holan, 2011: Polynomial nonlinear spatio-temporal integro-difference equation models Journal of Time Series Analysis. DOI: 10.1111/j.1467-9892.2011.00729.x
Hemispheric land surface temperature anomalies from P.D. Jones at University of East Anglia/Climate Research Unit. A. Kaplan at LDEO/IRI modified the Jones temperature data set archived at the National Center for Atmospheric Research (NCAR). See: "http://dss.ucar.edu/datasets/ds215.0/"
Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
BACKGROUND Fluctuations in climate have been associated with variations in mosquito abundance. OBJECTIVES To analyse the influence of precipitation, temperature, solar radiation, wind speed and humidity on the oviposition dynamics of Aedes aegypti in three distinct environmental areas (Brasília Teimosa, Morro da Conceição/Alto José do Pinho and Dois Irmãos/Pintos) of the city of Recife and the Fernando de Noronha Archipelago northeastern Brazil. METHODS Time series study using a database of studies previously carried out in the areas. The eggs were collected using spatially distributed geo-referenced sentinel ovitraps (S-OVTs). Meteorological satellite data were obtained from the IRI climate data library. The association between meteorological variables and egg abundance was analysed using autoregressive models. FINDINGS Precipitation was positively associated with egg abundance in three of the four study areas with a lag of one month. Higher humidity (β = 45.7; 95% CI: 26.3 - 65.0) and lower wind speed (β = −125.2; 95% CI: −198.8 - −51.6) were associated with the average number of eggs in the hill area. MAIN CONCLUSIONS The effect of climate variables on oviposition varied according to local environmental conditions. Precipitation was a main predictor of egg abundance in the study settings.
Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
This repository contains the data and code used in "Drivers and impacts of westerly moisture transport events in East Africa" by Robert Peal and Emily Collier (submitted to Weather and Climate Dynamics)
wmteMasks.tar | The WMTE masks used in the paper, derived from ERA5 daily average 700 hPa moisture transport |
precipMasks.tar | The precipitation masks identified in ERA5 daily precipitation totals using the method of Konstali et al. (2024), implemented in the dynlib package (Spensberger (2021)) |
attributedPrecipMasks.tar | Masks showing the precipitation polygons that overlapped with WMTEs |
utils.tar | Utility programs used for data processing and for generating the WMTE masks |
figures.tar | Data and code for generating the figures in "Drivers and impacts of westerly moisture transport events in East Africa" by Robert Peal and Emily Collier (submitted to Weather and Climate Dynamics) |
Utility programs used for data processing and for generating the WMTE masks
NOTE: Nearly all the code requires the scripts tctools2.py, cartopy_local.py, and pytime.py and the folder cartopyData to be in the python path in order to run. These are all in utils.tar. If you want to run the code, I recommend to unpack utils.tar and then copy tctools2.py, cartopy_local.py, pytime.py and cartopyData into the folder of the script you are running so that they can definitely be imported. Otherwise you can edit the path using sys.path.append() to add the appropriate location.
wmte_detector --->detectorData --->event_detector3.py --->run_westerly_detector3.sh --->detector2d.py |
The WMTE detector code is in this folder detectorData is a folder with the specs of the filters used to generate the masks User options should be specified in the bash script. detector2d provides utility functions for the detector |
tctools2.py | Module with utility functions used extensively in the project. IMPORTANT: Most of the files will require tctools2.py, cartopy_local.py, and pytime.py to be in the path in order to run |
cartopy_local.py cartopyData |
Module for making cartopy plots using local shapefiles so it can be run without internet connection Folder containing some cartopy shapefile data for plots |
pytime.py | Module with some clock functions |
calculate_moist_adv.py gen_daily_moisture_transport.sh | Python code for calculating daily average moisture transport from files with hourly wind and specific humidity, and a bash script running the python code |
nctools | Folder containing some useful functions for calculating daily climatologies and anomalies of netcdf files |
run_swio_state swio_state.py swio_state5.csv | swio_state.py generates the csv file with information about the MJO phase and TCs present in SWIO on each day |
MJO.csv | Australian Bureau of meteorology (BOM) MJO indices |
ibtracs.since1980.list.v04r00.csv |
International Best Track Archive for Climate Stewardship (IBTrACS) Tropical cyclone locations from Knapp et. al., 2010 |
each folder contains the code and data for a different figure from the paper. Processing is done by the python script inside, and the figure is generated in the notebook.
detectorOverview | fig. 1 code repo. Plotting detection on an example day |
persistence_stats | fig. 2 code repo. Calculating and plotting basic statistics of WMTEs |
moistureComposite |
fig. 3 code repo. Plotting the composite moisture transport with and without WMTEs Needs the timeseries of days with a WMTE crossing the EEA line generated in tcWesterlyDays |
mjoWesterlyDays | fig. 4 code repo. Calculating and plotting the number of WMTE days in each season in each MJO phase |
tcWesterlyDays | fig. 5 code repo. Calculating the number of days with WMTE crossing the EEA line and plotting risk ratio to TCs. Also contains sensitivity analysis of the WMTE algorithm |
precipDays | fig. 6 code repo. Precipitation aggregation |
Bureau of Meteorology (BoM).: Real-time Multivariate MJO (RMM) Phase Index, https://iridl.ldeo.columbia.edu/SOURCES/.BoM/.MJO/.RMM/phase/index.html, available from IRI/LDEO Climate Data Library, Accessed 06.02.2024.
Knapp, K. R., Kruk, M. C., Levinson, D. H., Diamond, H. J., and Neumann, C. J.: The International Best Track Archive for Climate Stewardship (IBTrACS): Unifying Tropical Cyclone Data, Bulletin of the American Meteorological Society, 91, 363 – 376, https://doi.org/https://doi.org/10.1175/2009BAMS2755.1, 2010
Konstali, K., Spensberger, C., Spengler, T., and Sorteberg, A.: Global Attribution of Precipitation to Weather Features, Journal of Climate, 37, 1181 – 1196, https://doi.org/10.1175/JCLI-D-23-0293.1, 2024.
Spensberger, C.: Dynlib: A library of diagnostics, feature detection algorithms, plotting and convenience functions for dynamic meteorology. https://doi.org/10.5281/zenodo.4639624, 2021.
Vertically averaged (0/400m) heat storage anomaly computed from a 1979-1988 climatology are available as a dataset, including an interactive viewer and downloadable datafiles.
Data from the Joint World Meteorological
Organization/Intergovernmental Oceanographic Commission Technical
Commision for Oceanography and Marine Meteorology (JCOMM) Products
Bulletin Data Products. The organization was formally known as the
Integrated Global Ocean Services System (IGOSS) Data Products
Bulletin.
For further data products see:
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
and
"http://iri.ldeo.columbia.edu/climate/monitoring/ipb/"
Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
The accompanying data used for an analysis of animal-related outages in the state of Massachusetts from 2013-2018. All data formating and analysis took place in R version 4.0.3. The original outage data comes from Eversource Energy, National Grid, and Unitil Corporation, made available through the MA office of Energy and Environmental Affairs. The outage dataset used in this analysis is available on Columbia University's International Research Institute (IRI) for Climate and Society Data Library at http://iridl.ldeo.columbia.edu/SOURCES/.EOEEA/. The original bird abundance data comes from the eBird Basic Dataset (May 2020) and the modeled relative abundance estimates for Massachusetts towns are also available on the IRI Data Library at http://iridl.ldeo.columbia.edu/SOURCES/.PRISM/.eBird/.derived/.detectionProbability/.
Sea level pressure indices from Tahiti for monitoring El Nino Southern Oscillation (ENSO).
The Scripps Institution of Oceanography mixed layer depth climatology was derived from the NODC Global Ocean Temperature/Salinity data set (1955-1988) and the Global Temperature/Salinity Pilot Project (GTSPP) data set (1989-1994). Temperatures were interpolated at 15 standard levels from 0 to 800 meters. The climatological series was based on the January 1980-December 1989 data.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
Attribution 4.0 (CC BY 4.0)https://creativecommons.org/licenses/by/4.0/
License information was derived automatically
This repository contains replication data for Schwarzwald, Kevin and Richard Seager (2024), "'Revisiting the “East African Paradox': CMIP6 models also struggle to reproduce strong observed MAM long rain drying trends." (under revision at Journal of Climate). This includes the data necessary to replicate all main text figures and most figures in Supplementary Materials. Additional figures in Supplementary Materials require raw precipitation time series, detailed below. This repository also includes a copy of the code necessary to replicate the entire study; the latest version of the code, in addition to instructions on how to use it, is kept at this GitHub archive.
The repository is structured as follows:
code
: Static / stable version of reproduction code for Schwarzwald and Seager (2024) that uses and creates these data (see here for more detailed instructions)figures
: Static / stable versions of main and supplemental figures for Schwarzwald and Seager (2024).climate_raw
: "raw" (often pre-processed) climate data files; only some files are included, see below for more detailsclimate_proc
: Processed climate data files upon which the analysis is based, used by main and supplemental figure codeaux_data
: Certain auxiliary data files (fonts, critical values) and intermediate files for long code processes. Created and used by the replication code. Due to space limitations (and a desire to not create yet another cloud copy of CMIP6 data), this repository only contains processed CMIP6 data: the calculated linear trends of rainfall, sea surface temperatures, and 500 hPa geopotential height used in the analysis of Schwarzwald and Seager (2024). The raw data used to create these files can be downloaded from the ESGF, and consists of every available CMIP6 monthly rainfall, sea surface temperature (SST), and geopotential height file on the archive at the time of processing for the experiments detailed in the manuscript. For precipitation, only a bounding box (-3 S to 12.5 N, 32 E to 55 E) around East Africa was downloaded, and saved with the file suffix "_HoAfrica" (see replication code README for more details). One example precipitation file (one ensemble member of ACCESS-CM2 historical precipitation) is included in this repository for reference.
Similarly, this repository only contains processed NMME data: calculated linear trends of rainfall used in the analysis of Schwarzwald and Seager (2024). The raw data used to create these files can be downloaded from the IRI Data Library and consists of every available monthly rainfall hindcast / forecast file from the NMME archive for the models used (see manuscript Table S1). As above, only a bounding box around East Africa was downloaded, and files were standardized to a partial CMIP* file format (one file per variable, with CMIP file and variable name conventions, but with forecast lead time as an additional dimension). Preprocessing is a bit more extensive than for CMIP6 models: first, hindcasts and forecasts were concatenated into a single file (since hindcasts are saved in the archive only up to the time when the model was operationalized), and then reindexed such that the "time" variable refers to the time for which the forecast is made, and not the time at which the forecast is made. One example precipitation file (CanCM4i rainfall hindcasts/forecasts) is included in this repository for reference.
This repository does, however, contain preprocessed copies of the "raw" gridded observational rainfall data products used (in addition to the processed trends), since harmonizing and standardizing the data into a format easily compatible with the CMIP6 data was a nontrivial amount of work that would be tedious to replicate. For the ten gridded observational data products used, monthly rainfall was brought into the CMIP* file format (one file per variable, with CMIP file and variable name conventions), with one notable exception: all observational rainfall is saved in units of mm/day.
Reanalysis and gridded ocean observations can be downloaded from each product's respective repositories. As before, the code assumes the data have been preprocessed into something akin to the CMIP* file format.
Note that the repository includes a file (aux_data/pr_doyavg_CHIRPS_historical_seasstats_dunning_19810101-20141231_HoAfrica.nc
) containing CHIRPS seasonal characteristics in East Africa, created as part of Schwarzwald et al., 2023, Climate Dynamics. This file is primarily used to set the boundaries of the study region, see the manuscript for details of how it was calculated.
These data are offered under a CC 4.0 license, which allows redistribution and reuse as long as they are correctly cited; note that for much of these data (especially for "raw" data), this requires citations to the original creators.
For questions, please feel free to reach out to corresponding author Kevin Schwarzwald.
A Reduced Space Optimal Interpolation procedure has been applied to the global sea level pressure (SLP) record from the Comprehensive Ocean Atmosphere Data Set (COADS) averaged on a 4x4 degree grid. The SLP anomalies are with respect to the climatological annual cycle estimated from COADS data for the period 1951-1980. The data are presented as a monthly climatology.
Additional Kaplan SLP data include: - Optimal Interpolation 1854-1992 - Projected SLP anomalies based on linear best fit of the EOF patterns to the data 1854-1992 - Kaplan RF and KF Analysis errors and estimates of SLP 1854-1992
The Standardized Precipitation Index (SPI; McKee et al. 1993) is the number of standard deviations that observed cumulative precipitation deviates from the climatological average. The index is based entirely on monthly precipitation accumulations and its values can be compared across different climatic and geographic regions. These characteristics of the SPI have contributed to its popularity for application towards drought and water resource monitoring.
SPI analyses were performed on the following climatological data sets: NASA GPCP V2 NOAA NCEP CPC CAMS_OPI NOAA NCEP CPC Merged_Analyis (CMAP) UEA CRU New CRU05
Studies have suggested that a minimum of 50 years of precipitation data be used to calculate SPI values. It should therefore be noted that the UEA New data set is the only precipitation data set in the current set of analyses that meets that recommendation. Extreme SPI values in the other three datasets (CAMS OPI, CMAP, GPCP) may be suspect as they are based on roughly half of the recommended amount of data.
The analyses in this data set are based on a Pearson Type III (i.e., 3-parameter gamma) distribution as suggested by Guttman (1999). Fortran 77 source code made available in that reference was used to create the SPI analyses.
The SPI analyses are available from: http://ingrid.ldeo.columbia.edu/docfind/databrief/cat-atmos.html
Data set consists of a time series of sea height derived from altimeter measured on the TOPEX/Poseidon satellite.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"
The monthly optimum interpolation (OI) fields are derived by a linear interpolation of the weekly OI fields to daily fields then averaging the daily values over a month. The monthly fields are in the same format and spatial resolution as the weekly fields.
The climatological monthly mean SST (January-December) was blended from ship, buoy, and bias-corrected satellite data from 1950-1979.
Data from the Joint World Meteorological Organization/Intergovernmental Oceanographic Commission Technical Commision for Oceanography and Marine Meteorology (JCOMM) Products Bulletin Data Products. The organization was formally known as the Integrated Global Ocean Services System (IGOSS) Data Products Bulletin.
For further data products see: "http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/"
"http://ingrid.ldeo.columbia.edu/SOURCES/.IGOSS/.data_products.html"