Sample data for exercises in Further Adventures in Data Cleaning.

On an annual basis (individual hospital fiscal year), individual hospitals and hospital systems report detailed facility-level data on services capacity, inpatient/outpatient utilization, patients, revenues and expenses by type and payer, balance sheet and income statement.

Due to the large size of the complete dataset, a selected set of data representing a wide range of commonly used data items, has been created that can be easily managed and downloaded. The selected data file includes general hospital information, utilization data by payer, revenue data by payer, expense data by natural expense category, financial ratios, and labor information.

There are two groups of data contained in this dataset: 1) Selected Data - Calendar Year: To make it easier to compare hospitals by year, hospital reports with report periods ending within a given calendar year are grouped together. The Pivot Tables for a specific calendar year are also found here. 2) Selected Data - Fiscal Year: Hospital reports with report periods ending within a given fiscal year (July-June) are grouped together.

A dataset of synchrotron X-ray diffraction (SXRD) analysis files, recording the refinement of crystallographic texture from a number of Ti-6Al-4V (Ti-64) sample matrices, containing a total of 93 hot-rolled samples, from three different orthogonal sample directions. The aim of the work was to accurately quantify bulk macro-texture for both the α (hexagonal close packed, hcp) and β (body-centred cubic, bcc) phases across a range of different processing conditions.

Material

Prior to the experiment, the Ti-64 materials had been hot-rolled at a range of different temperatures, and to different reductions, followed by air-cooling, using a rolling mill at The University of Manchester. Rectangular specimens (6 mm x 5 mm x 2 mm) were then machined from the centre of these rolled blocks, and from the starting material. The samples were cut along different orthogonal rolling directions and are referenced according to alignment of the rolling directions (RD – rolling direction, TD – transverse direction, ND – normal direction) with the long horizontal (X) axis and short vertical (Y) axis of the rectangular specimens. Samples of the same orientation were glued together to form matrices for the synchrotron analysis. The material, rolling conditions, sample orientations and experiment reference numbers used for the synchrotron diffraction analysis are included in the data as an excel spreadsheet.

SXRD Data Collection

Data was recorded using a high energy 90 keV synchrotron X-ray beam and a 5 second exposure at the detector for each measurement point. The slits were adjusted to give a 0.5 x 0.5 mm beam area, chosen to optimally resolve both the α and β phase peaks. The SXRD data was recorded by stage-scanning the beam in sequential X-Y positions at 0.5 mm increments across the rectangular sample matrices, containing a number of samples glued together, to analyse a total of 93 samples from the different processing conditions and orientations. Post-processing of the data was then used to sort the data into a rectangular grid of measurement points from each individual sample.

Diffraction Pattern Averaging

The stage-scan diffraction pattern images from each matrix were sorted into individual samples, and the images averaged together for each specimen, using a Python notebook sxrd-tiff-summer. The averaged .tiff images each capture average diffraction peak intensities from an area of about 30 mm2 (equivalent to a total volume of ~ 60 mm3), with three different sample orientations then used to calculate the bulk crystallographic texture from each rolling condition.

SXRD Data Analysis

A new Fourier-based peak fitting method from the Continuous-Peak-Fit Python package was used to fit full diffraction pattern ring intensities, using a range of different lattice plane peaks for determining crystallographic texture in both the α and β phases. Bulk texture was calculated by combining the ring intensities from three different sample orientations.

A .poni calibration file was created using Dioptas, through a refinement matching peak intensities from a LaB6 or CeO2 standard diffraction pattern image. Two calibrations were needed as some of the data was collected in July 2022 and some of the data was collected in August 2022. Dioptas was then used to determine peak bounds in 2θ for characterising a total of 22 α and 4 β lattice plane rings from the averaged Ti-64 diffraction pattern images, which were recorded in a .py input script. Using these two inputs, Continuous-Peak-Fit automatically converts full diffraction pattern rings into profiles of intensity versus azimuthal angle, for each 2θ section, which can also include multiple overlapping α and β peaks.

The Continuous-Peak-Fit refinement can be launched in a notebook or from the terminal, to automatically calculate a full mathematical description, in the form of Fourier expansion terms, to match the intensity variation of each individual lattice plane ring. The results for peak position, intensity and half-width for all 22 α and 4 β lattice plane peaks were recorded at an azimuthal resolution of 1º and stored in a .fit output file. Details for setting up and running this analysis can be found in the continuous-peak-fit-analysis package. This package also includes a Python script for extracting lattice plane ring intensity distributions from the .fit files, matching the intensity values with spherical polar coordinates to parametrise the intensity distributions from each of the three different sample orientations, in the form of pole figures. The script can also be used to combine intensity distributions from different sample orientations. The final intensity variations are recorded for each of the lattice plane peaks as text files, which can be loaded into MTEX to plot and analyse both the α and β phase crystallographic texture.

Metadata

An accompanying YAML text file contains associated SXRD beamline metadata for each measurement. The raw data is in the form of synchrotron diffraction pattern .tiff images which were too large to upload to Zenodo and are instead stored on The University of Manchester's Research Database Storage (RDS) repository. The raw data can therefore be obtained by emailing the authors.

The material data folder documents the machining of the samples and the sample orientations.

The associated processing metadata for the Continuous-Peak-Fit analyses records information about the different packages used to process the data, along with details about the different files contained within this analysis dataset.

SYstem for iNTegrating Human dimensions, Ecosystem Services and Economic Assessment for Sustainability. CSIRO has developed this Shiny application to store metadata information relevant to datasets found in the Great Barrier Reef region which are primarily ecosystem service focused. The Shiny app allows users to filter the records describing the catalog of datasets by ES category (Provisioning, Regulating & Cultural) and Ecosystem service. It also allows users to filter by GBR User type (First Nations, Government, Household and industry). Users can also filter by various components in the Ecosystem Service Value Chain (ESVC) (eg. Use, measures, and derived values) to enable users to understand the various datasets which have been used to construct an ESVC. Lineage: The SEABORNE (Sustainable UsE And Benefits fOR mariNE) project has consolidatied and synthesised existing information about who is using the Reef, how it is being used and what the benefits are from this use. CSIRO's research on the Great Barrier Reef (GBR) has been identified as a Category 4 Mission for the organisation, with well-established investors and collaborators, an internal coordination architecture, and delivering impact from a large portfolio of research across several Business Units. This project is one of several key strategic outcomes of the Great Barrier Reef Platform. SEABORNE began in November 2021, with the project team initially developing and sourcing a list of potential datasets relevant to the research question. An Excel spreadsheet was trialled to make it more of a data entry form for users, however we encountered problems with dropdown fields not allowing multiple selections of values, due to the version of Excel and VBA programming. As part of the SCCPs (ERRFP-1322) we developed a Shiny (R) dashboard that allowed the database to be filtered and searched in a user-friendly manner. We exported the data from the MS ACCESS database as a CSV and used this in the Shiny app. This app also allows the spatial extent data from CSIRO DAP and the GBRMPA catalogue (online json file) to be read and displayed from the relevant website on a Leaflet map. Each record in the metadata database (has a UniqueID) and pertains to a dataset which has been used or considered in the SEABORNE project. This tool allows researchers a summary of what’s available, particularly in the GBR in relation to ecosystem services, where to get the data, what the data is about, the quality of the data etc, and who to contact to acquire the data. It is NOT a data warehouse, nor is it a data portal to download data from.