Matlab script for reading hdf5 files

General information on the data set

The dataset was generated with two different measurement systems at the ZeMA testbed for electromechanical cylinders.

All relevant information can be found within the hdf5 file.

Example for reading out the metadata of the hdf5 file in MATLAB:

available structures inside file

dataset = 'axis11_2kHz_ZeMA_PTB_SI.h5'; h5disp(dataset)

% general attributes about file attr = h5info(dataset).Attributes; project = jsondecode(attr(1,1).Value) person = jsondecode(attr(2,1).Value) publication = jsondecode(attr(3,1).Value) experiment = jsondecode(attr(4,1).Value)

Example for reading out the metadata of the hdf5 file in Python:

import h5py import json

open file

h5file = h5py.File("axis11_2kHz_ZeMA_PTB_SI.h5", "r")

general attributes about file

for key in h5file.attrs: print(key) val = json.loads(h5file.attrs[key]) for subkey, subval in val.items(): print(" ", subkey, " : ", subval)

available structures inside file

h5file.visit(print)

proper exit

h5file.close()

Metadata output of the hdf5 file:

For the dataset:

HDF5 axis11_2kHz_ZeMA_PTB_SI.h5 Group '/' Attributes: 'Project': '{ "fullTitle":"Metrology for the Factory of the Future", "acronym":"Met4FoF", "websiteLink":"www.met4fof.eu", "fundingSource":"European Commission (EC)", "fundingAdministrator":"EURAMET", "funding programme":"EMPIR", "fundingNumber":"17IND12", "acknowledgementText":"This work has received funding within the project 17IND12 Met4FoF from the EMPIR program co-financed by the Participating States and from the European Union's Horizon 2020 research and innovation program. The authors want to thank Clifford Brown, Daniel Hutzschenreuter, Holger Israel, Giacomo Lanza, Bj\u00f6rn Ludwig, and Julia Neumann fromPhysikalisch-Technische Bundesanstalt (PTB) for their helpful suggestions and support." }' 'Person': '{ "dc:author":[ "Tanja Dorst", "Maximilian Gruber", "Anupam Prasad Vedurmudi" ], "e-mail":[ "t.dorst@zema.de", "maximilian.gruber@ptb.de", "anupam.vedurmudi@ptb.de" ], "affiliation":[ "ZeMA gGmbH", "Physikalisch-Technische Bundesanstalt", "Physikalisch-Technische Bundesanstalt" ] }' 'Publication': '{ "dc:identifier":"10.5281/zenodo.5185953", "dc:license":"Creative Commons Attribution 4.0 International (CC-BY-4.0)", "dc:title":"Sensor data set of one electromechanical cylinder at ZeMA testbed (ZeMA DAQ and Smart-Up Unit)", "dc:description":"The data set was generated with two different measurement systems at the ZeMA testbed. The ZeMA DAQ unit consists of 11 sensors and the SmartUp-Unit has 13 differentsignals. A typical working cycle lasts 2.8s and consists of a forward stroke, a waiting time and a return stroke of the electromechanical cylinder. The data set does not consist of the entire working cycles. Only one second of the return stroke of every 100rd working cycle is included. The dataset consists of 4776 cycles. One row represents one second of the return stroke of one working cycle.", "dc:subject":[ "dynamic measurement", "measurement uncertainty", "sensor network", "digital sensors", "MEMS", "machine learning", "European Union (EU)", "Horizon 2020", "EMPIR" ], "dc:SizeOrDuration":"24 sensors, 4776 cycles and 2000 datapoints each", "dc:type":"Dataset", "dc:issued":"2021-09-10", "dc:bibliographicCitation":"T. Dorst, M. Gruber and A. P. Vedurmudi : Sensor data set of one electromechanical cylinder at ZeMA testbed (ZeMA DAQ and Smart-Up Unit), Zenodo [data set], https://doi.org/10.5281/zenodo.5185953, 2021." }' 'Experiment': '{ "date":"2021-03-29/2021-04-15", "DUT":"Festo ESBF cylinder", "identifier":"axis11", "label":"Electromechanical cylinder no. 11" }'

Example for one sensor (BMA 280, acceleration) of the PTB SmartUp Unit (SUU) and one sensor of ZeMA DAQ (pressure):

HDF5 axis11_2kHz_ZeMA_PTB_SI.h5 Group '/PTB_SUU' Group '/PTB_SUU/BMA_280' Group '/PTB_SUU/BMA_280/Acceleration' Attributes: 'qudt:hasQuantityKind': '[ "qudt:Acceleration", "qudt:Acceleration", "qudt:Acceleration" ]' 'misc': '{ "interpolation_scheme":"cubic" }' 'si:unit': '"\metre\second\tothe{-2}"' 'sosa:madeBySensor': '"BMA 280"' 'rdf:type': '"qudt:Quantity"' Dataset 'qudt:standardUncertainty' Size: 4766x1000x3 MaxSize: 4766x1000x3 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'si:label': '[ "X acceleration uncertainty", "Y acceleration uncertainty", "Z acceleration uncertainty" ]' Dataset 'qudt:value' Size: 4766x1000x3 MaxSize: 4766x1000x3 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'si:label': '[ "X acceleration", "Y acceleration", "Z acceleration" ]' Group '/ZeMA_DAQ' Group '/ZeMA_DAQ/Pressure' Attributes: 'qudt:hasQuantityKind': '"qudt:Pressure"' 'sosa:madeBySensor': '"Festo VPPM"' 'si:unit': '"\pascal"' 'rdf:type': '"qudt:Quantity"' Dataset 'qudt:standardUncertainty' Size: 4766x2000 MaxSize: 4766x2000 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'si:label': '"Pneumatic pressure uncertainty"' Dataset 'qudt:value' Size: 4766x2000 MaxSize: 4766x2000 Datatype: H5T_IEEE_F64LE (double) ChunkSize: [] Filters: none FillValue: 0.000000 Attributes: 'si:label': '"Pneumatic pressure"' 'misc': '{ "raw_data":false, "comment":"Converted from ADC values based on appropriate conversion." }'

Companion dataset of the manuscript: Fu Li, Umberto Villa, Seonyeong Park, Mark A. Anastasio. Three-dimensional stochastic numerical breast phantoms for enabling virtual imaging trials of ultrasound computed tomography. Arxiv preprint 2106.02744 (2021) This dataset includes a collection of 52 two-dimensional slices of numerical breast phantoms (NBPs) and corresponding ultrasound computed tomography (USCT) simulated measurement data. The anatomical structures of these NBPs were obtained by use of tools from the Virtual Imaging Clinical Trial for Regulatory Evaluation (VICTRE) project. More details on the modification and extension of the VICTRE NBPs for use in USCT studies can be found in the accompanying paper. The NBPs included in this dataset are representative of four ACR BI-RADS breast composition types: A. The breast is almost entirely fatty B. There are scattered areas of fibroglandular density C. The breasts is heterogeneously dense D. The breast is extremely dense Each NBP contains 2D maps of tissue labels, speed of sound, acoustic attenuation, density. A low-resolution speed-of-sound map is also provided to reproduce the FWI reconstruction results presented in the accompanying paper. Corresponding USCT measurement data were simulated by modeling 2D wave propagation in lossy heterogeneous media using a time explicit pseudospectral wave propagation solver. The dataset consists of three folders: The 2d_slices folder contains the 52 slices extracted from 3D NBPs. The measurements folder contains simulated measurement data corresponding to each slice. The imaging_system folder contains information about the 2D imaging system (excitation source, transducer coordinates) In addition, the following helper Matlab scripts are included: read_data.m: Helper function to load and visualize the excitation source and transducer locations. read_waveform_data.m: Helper function to read the .h5 files containing the measurement data. Each slice is saved as a binary MATLAB file (.mat) and contains the following variables label: tissue label map with [2560,2560] pixels and 0.1mm pixel size. Tissue types are denoted using the following labels: water (0), fat (1), skin (2), glandular (29), ligament (88), lesion (200). sos: speed of sound map (mm/μs) with [2560,2560] pixels and 0.1 mm pixel size. Data is stored as data type float32. aa: acoustic attenuation map (Np/m/MHzy) with [2560,2560] pixels and 0.1mm pixel size. Data is stored as data type float32. density: density map (kg/mm3) with [2560,2560] pixels and 0.1 mm pixel size. Data is stored as data type float32. sos_ini: low resolution speed of sound map (mm/μs) with [1280,1280] pixels and 0.2mm pixel size. Data is stored as data type float32. This is the initial guess used in the speed of sound reconstructions in our paper. y: attenuation exponent used for simulation. seed: phantom id type: breast composition type (A-D) The simulated measurement data is saved in hdf5 format. Measurement data corresponding the i-th emitting transducer is stored with hdf5 key equal to the transducer index as a two-dimensional array of size [1024,4250]. Here, the rows represent the receiver index, and the columns the time sample. The sampling frequency is 25MHZ. Because of file size limitations, measurement data for each slice has been divided into 8 chunks, containing data from 128 receivers each. The imaging_system folder contains information regarding the 2D imaging system. source300.mat describes the time profile of the exitation pulse. It consists of 300 time samples at a sampling frequency of 25Mhz. locations1024.mat provide the xy coordinates (mm)of the location of each transducer Data type is float32. Array size is [2x1024]. Warning: This is a very large dataset (~1TB). Please check out our download script written in python.

Testing dataset in hdf5 file format without the class labels. Scripts are provided for reading in the dataset in R, Python, and Matlab.

Overview This repository contains 742,688 samples of simulated Convergent Beam Electron Diffraction patterns (CBEDs); the training data for the paper "Phase Object Reconstruction for 4D-STEM using Deep Learning". The folder contains multiple hdf5 datasets. Each dataset has a corresponding Excel-sheet containing detailed information and simulation parameters for every datapoint, as well as a summary-report containing the parameter distributions, hdf5-infos and random number generator settings. This makes every dataset reproducible, using the simulation codes provided in https://github.com/ThFriedrich/ap_data_generation. Technical details Every Datapoint consists of a 3x3 set of adjacent Convergent Beam Electron Diffraction pattern (CBEDs), the coherent exit wave phase and amplitude in real and reciprocal space, and the probe functions phase and amplitude in real space. All patterns are 64x64 pixel in 16 bit unsigned integer data format. Every hdf5 file has the following structure: Attributes 'Seed': 6108236 'State': 251786606 ... 'Type': 'twister' 'arch': 'glnxa64' 'gpu': 'NVIDIA GeForce RTX 3080' 'matlab_ver': '2021a' Dataset 'features' Size: 64x64x9x5000 Datatype: H5T_STD_U16LE (uint16) Dataset 'labels_k' Size: 64x64x2x5000 Datatype: H5T_STD_U16LE (uint16) Dataset 'labels_r' Size: 64x64x2x5000 Datatype: H5T_STD_U16LE (uint16) Dataset 'probe_r' Size: 64x64x2x5000 Datatype: H5T_STD_U16LE (uint16) Dataset 'meta' Size: 19x5000 Datatype: H5T_IEEE_F32LE (single) The data was written to hdf5 in matlab. When reading from these files consider possibly different storage conventions (Row major vs. column major format). Data may need to be transposed accordingly. The integer arrays were scaled to use the full range of the uint16 datatype. The scaling values are stored under "meta". To restore the original values in floating point numbers, convert the arrays like this: Matlab: hdf_file = ['db_h5_b_5_Training.h5']; n = 128; % load n k-space exit waves x = single(h5read(hdf_file, '/labels_k', [1,1,1,1], [64,64,2,n])); % meta contains parameters and scaling factors for a given datapoint in following order: [E_0(keV), cond_lens_outer_aper_ang(mrad), collection angle(rA), step_size(A), scale_cbed_1 ... scale_cbed_9, scale_phase_k, scale_amp_k, scale_phase_r, scale_amp_r, scale_probe_phase_r, scale_probe_amp_r] s = h5read(hdf_file, '/meta', [14,1], [2,n]); amplitude = zeros(64,64,n); phase = zeros(64,64,n); for ix = 1:n phase(:,:,n) = (x(:,:,1,n)*s(1,ix) / 65536) - pi; amplitude(:,:,n) = (x(:,:,2,n)*s(2,ix)) / 65536; end % The 9 CBEDs correspond to a 3x3 kernel of patterns. The order in [x,y] is: %[[3, 6, 9]; % [2, 5, 8]; % [1, 4, 7]]

Training dataset in hdf5 file format including the class labels. Scripts are provided for reading in the dataset in R, Python, and Matlab.

This dataset contains data from a three-dimensional large eddy simulation of Mach 0.3 flow over a NACA 0012 airfoil at Reynolds number 23,000, which features a transitional boundary layer, separation over a recirculation bubble, and a turbulent wake. The dataset contains 16,000 time-resolved snapshots of the mid-span and spanwise-averaged velocity fields. All data are stored within hdf5 files, and a Matlab script showing how the data can be read and manipulated is provided. Please see the ‘airfoilLES_README.pdf’ file for more information. We recommend using the ‘airfoilLES_example.zip’ file as an entry point to the dataset.;The dataset is part of “A database for reduced-complexity modeling of fluid flows” (see references below) and is intended to aid in the conception, training, demonstration, evaluation, and comparison of reduced-complexity models for fluid mechanics. The paper introduces the flow setup and computational methods, describes the available data, and provides an example of how these data can be used for reduced-complexity modeling. Users of these data should cite the papers listed below.

The Imperial Valley Dark Fiber Project acquired Distributed Acoustic Sensing (DAS) seismic data on a ~28 km segment of dark fiber between the cities of Calipatria and Imperial in the Imperial Valley, Southern California. Dark fiber refers to unused optical fiber cables in telecommunications networks and is repurposed in this project for DAS applications. The objective, which is further detailed in the attached journal article from Ajo-Franklin et al., is to demonstrate dark fiber DAS as a tool for basin-scale geothermal exploration and monitoring. The included DAS data were recorded during two days at the beginning the project. Data is stored in the .h5 (HDF5) file format, readable using various software tools, including the 'h5read' and 'h5info' functions in Matlab. Provided here are examples of methods to read and use the data with the 'h5py' package in Python.