Stream Reasoning (SR) focuses on developing advanced approaches for applying inference to dynamic data streams; it has become increasingly relevant in various application scenarios such as IoT, Smart Cities, Emergency Management, and Healthcare, despite being a relatively new field of research.

The current lack of standardized formalisms and benchmarks has been hindering the comparison between different SR approaches. We propose a new benchmark, called EnviroStream, for evaluating SR systems on weather and environmental data from two European cities.

The benchmark includes queries and datasets of different sizes. We adopt I-DLV-sr, a recently released SR system based on Answer Set Programming, as a baseline experiment. We illustrate how the queries can be modeled via I-DLV-sr input language and report evaluation times. We also assess continuous online reasoning via a web application.

Data can and queries can be also downloaded via the GitHub repository: https://github.com/DeMaCS-UNICAL/EnviroStream

Real-time data can be visualized via the following link: https://experiments.demacs.unical.it/

Data streams in the JSON format of the Vélo en Libre Service (VLS) of Saint-Etienne-Métropole, VéliVert, according to the GBFS standard, make it possible to know in real time: — Information from the VéliVert service (system_information.json) — Name, location and capacity of stations (station_information.json) — Places and bikes available at the station (station_status.json) Further information: — Source: Régie VéliVert — Smoov — Reference coordinate system: WGS84 — EPSG:4326 — GBFS V1 Standard — Documentation: https://github.com/MobilityData/gbfs

This repository includes input and output data of the methodology presented in the paper for generating a calibrated dynamic microscopic traffic simulation.

• The input data includes the network, initial normalized origin-destination matrix, and hourly traffic counts from stationary city sensors.
• The output is a 24-hour calibrated microscopic traffic simulation for the city of Tartu, Estonia.

All source codes are available at https://github.com/Khoshkhah/NRTCalib.

Advances in fluorescence microscopy enable monitoring larger brain areas in-vivo with finer time resolution. The resulting data rates require reproducible analysis pipelines that are reliable, fully automated, and scalable to datasets generated over the course of months. We present CaImAn, an open-source library for calcium imaging data analysis. CaImAn provides automatic and scalable methods to address problems common to preprocessing, including motion correction, neural activity identification, and registration across different sessions of data collection. It does this while requiring minimal user intervention, with good scalability on computers ranging from laptops to high-performance computing clusters. CaImAn is suitable for two-photon and one-photon imaging, and also enables real-time analysis on streaming data.

To benchmark the performance of CaImAn we collected and combined a corpus of manual annotations from multiple labelers on nine mouse two-photon datasets, that are contained in this open access repository. We demonstrate that CaImAn achieves near-human performance in detecting locations of active neurons.

In order to reproduce the results of the paper or download the annotations and the raw movies, please refer to the readme.md at:

https://github.com/flatironinstitute/CaImAn/blob/master/use_cases/eLife_scripts/README.md

Measurements of Mains Frequency (aka Utility Frequency)

This repository provides measurements of the mains frequency (duration of every fully wave), taken in southern Germany (Plochingen), starting on week 38 in 2022. New data is uploaded weekly.

The data has been captured using a microcontroller with clock frequency synchronized to a 1-pps GPS signal. More details about the measuring methodology can be found below.

For each week, two data sets are uploaded:

1. raw data: a stream of type-length-value (TLV) records as received from the measuring device (micro-controller).
2. clean data: sanity-checked data produced from raw data and converted to user-friendly comma-separated values (CSV) format

More information about the data formats (TLV and CSV) and about the process to clean data (sanity check) can be found below.

Source Code

Source code including the microcontroller code, server code, and jupyter notebook to process the data is made available through Github: Github Repository

Measurement Methodology

The timer capture functionality of the microcontroller is used to capture the period of the mains sine waves and the 1-pps signal of a GPS device to calibrate the timer clock of the microcontroller.

Transformer --(sine wave)--> half-wave rectifier --(half sine wave)--> Schmitt-Trigger --(square wave)--> Microcontroller --(samples)--> Computer --> raw file 
                                                      ^ 
                                                      |
GPS Device <-------------------------------1-pps signal------------------------------------------------------

RAW Data Records -- TLV Files

Raw data is recorded in binary format (Little Endian) as a stream of type-length-value (TLV) records. Type is a uint16 number; length is a uint16 number defining the length of the value(s ) in bytes.

The interpretation of the value(s) depends on the type. The following types are defined:

• SAMPLES record (type 0): a batch of n = length/4 uint32 values defining the number of clock ticks of n consecutive waves. The clock ticks at a nominal rate of 42 MHz.
• ONEPPS record (type 1): a single uint32 value defining the number of clock ticks per second, calibrated by a 1-pps signal from a GPS device.
• WALLCLOCKTIME (type 2): a single uint64 value defining nanoseconds since the UNIX epoch (00:00:00 UTC, Jan. 1, 1970) referencing the stream roughly to wallclock (real) time. Note that this is just a rough reference to real-time. In particular, not every sample is timestamped, but wallclock timestamps are inserted into the stream every second.

C code to parse raw TLV files is provided in the Git.

Clean Data -- Processing and Format of CSV Files

Clean data, published as comma-separated values (CSV) files, is created from raw data as follows:

A sanity check is performed on 1-pps records calibrating the microcontroller clock. In seldom cases, the GPS device might not output a 1-pps signal for a short period. In these cases, the 1-pps value deviated significantly from the nominal value of 42 MHz (the nominal clock frequency of the microcontroller). Such 1-pps records are removed.

No further checks have been performed on the raw data, in particular, samples are not filtered. Recommended further post-processing includes downsampling the data to remove (very seldom) outliers affecting only one or two consecutive waves. To this end, a median filter calculating the median of five consecutive samples has shown to be very effective.

The clean CSV file contains the following fields:

• f_mains: mains frequency as captured by uncalibrated 42 MHz clock. The crystal oscillator can be expected to have an accuracy of 30 ppm over the full temperatur range (the calibration shows an accuracy better than 20 ppm, which is typical for for crystal oscillators at room temperature).
• f_mains_syncd: mains frequency, calibrated to 1-pps signal of GPS device (using the value f_clk_synd as described next).
• f_clk_syncd: calibrated clock frequency of microcontroller using 1-pps signal (nominal 42 MHz).
• clk_accuracy_ppm: estimated clock accuracy in ppm (relevant when interpreting f_mains instead of f_mains_syncd).
• t_wallclock: wallclock time in nanoseconds since UNIX epoch (00:00:00 UTC, Jan 1, 1970). Note that this timestamp is only taken once per second to roughly reference the samples to wallclock time. Therefore, several samples have the same wallclock timestamp! Still, it is useful for selecting all samples of one day, hour, minute, etc.
• t_wallclock_str: string representation of t_wallclock value (in UTC).

Gaps in Data

No or incomplete data is available during these periods due to technical maintenance:

• Nov 5-8, 2022
• Nov 23, 2022
• Dec 19, 2022

A Toolbox designed around neural networks for time series analysis of TreeNet data. The software containing the machine learning models is stored as a repository on GitHub (https://github.com/treenet/treenetai) and licensed under GNU GENERAL PUBLIC LICENSE version 3. TreeNet (treenet.info) is an international monitoring and research network in which automated tree stem radius fluctuations measured with point dendrometers are analyzed in terms of forest ecosystem responses to climate change. A continuous stream of microclimate and tree physiology data provides realtime information on tree water relations and tree growth. TreeNet aims to link research results from carbon flux sites with dendrometer data to entire landscapes. Further it provides online-tools to its partners to automatically interpret stem radius fluctuations in terms of tree water deficit, wood growth and related indicators for forest ecosystem carbon sink and drought stress. The project was initiated by Roman Zweifel, WSL and Werner Eugster, ETHZ in 2009.

Abstract

This dataset was supplied to the Bioregional Assessment Programme by a third party and is presented here as originally supplied. Metadata was not provided and has been compiled by the Bioregional Assessment Programme based on the known details at the time of acquisition.

The data includes level, salinity and temperature from gauge 203450 and 203470 in the Richmond catchment. This data is plotted against time for water quality analysis purposes

This is a download from the open access NSW database at http://realtimedata.water.nsw.gov.au/water.stm

Dataset History

This data is a download from the open access NSW database

http://realtimedata.water.nsw.gov.au/water.stm

The data includes level, salinity and temperature from gauge 203450 and 203470 in the Richmond catchment.

Data is was downloaded on 18/3/2015.

Dataset Citation

NSW Office of Water (2015) CLM - Richmond stream gauge data. Bioregional Assessment Source Dataset. Viewed 07 April 2016, http://data.bioregionalassessments.gov.au/dataset/03f59f6b-8d06-4513-b662-db7c4c2d2909.

This record describes sub bottom profiler data collected on RV Investigator voyage IN2024_T01, 'Transit Voyage: Fremantle to Hobart.' The primary objective of the voyage was the safe and timely transit of RV Investigator from Fremantle to Hobart in preparation for the following research voyage. During the transit various research projects were conducted, along with outreach, familiarisation and training activities. The voyage departed Fremantle on the 09 March and returned to Hobart on the 20 March. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged [continuously/sporadically] for the extent of the voyage. Data are stored in .raw (260 files 5.77 GB) raw and .seg (257 files 5.69 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

These are the data and resources used for a Twitch Emote recommendation system using a Word2Vec model. The nature and exploration of the data is described in Emotes-2-Vec: A Large Scale Embedding of Twitch Chat Data. To protect the privacy of the users whose messages were scraped to build this corpus, names and timestamps have been removed and only the message bodies are included. However, a tutorial for this project is included on the project GitHub: https://github.com/KoroshM/Emote-Recommender.

embeddings.tsv and labeled_metadata.tsv may be used in TensorFlow's embedding projector to visualize the embedding space.

Note: Model files are the following:
embeddings.tsv
labeled_metadata.tsv
model
model.model**
model.wv.vectors.npy

**Located here: https://drive.google.com/drive/folders/1RZC4JA4CpAcwoo6dOwq_jobTd6dNi_n2?usp=sharing

2020 US Presidential Election Tweet IDs The repository contains an ongoing collection of tweets IDs associated with the 2020 United States presidential elections, with our data collection starting on May 20, 2019. We leveraged Twitter’s streaming API to follow specified accounts and also collect in real-time tweets that mention specific keywords. To comply with Twitter’s Terms of Service, we are only publicly releasing the Tweet IDs of the collected Tweets. The data is released for non-commercial research use. We currently have a backlog of historical twitter files that we are working on pre-processing and extracting Tweet IDs from; we will be releasing both past and future data sets as the data becomes available and as we finish pre-processing the data. Thank you for your patience! Note that the compressed files must be first uncompressed in order to use included scripts. This dataset is release v1.3 and is not actively maintained -- the actively maintained dataset can be found here: https://github.com/echen102/us-pres-elections-2020. This release contains Tweet IDs collected from 3/01/20 - 11/13/20. Please refer to the README for more details regarding data, data organization and data usage agreement. This dataset is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License . By using this dataset, you agree to abide by the stipulations in the license, remain in compliance with Twitter’s Terms of Service, and cite the following manuscript: Emily Chen, Ashok Deb, Emilio Ferrara. #Election2020: The First Public Twitter Dataset on the 2020 US Presidential Election. Arxiv (2020)

This record describes sub bottom profiler data collected on RV Investigator voyage IN2024_V01, 'Multidisciplinary Investigations of the Southern Ocean' which departed Hobart on 05/01/2024 and returned to Fremantle on 05/03/2024. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor from Hobart, along the Antarctic ice edge to Fremantle. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage. Data are stored in .raw and .seg formats at CSIRO. There are 775 files totalling 29.8 GB of raw data in this dataset. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on RV Investigator voyage IN2024_V03, titled "Untangling the causes of change over 25 years in the southeast marine ecosystem (SEA-MES Voyage 2)." The voyage took place between May 01, 2024 and May 31, 2024 (AEST), departing from Hobart and returning to Sydney. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage. Data are stored in .raw (1368 files 30.5 GB) raw and .seg (1352 files 30.2 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on RV Investigator voyage IN2023_V03, SOTS: Southern Ocean Time Series automated moorings for climate and carbon cycle studies southwest of Tasmania which departed Hobart on May 12, 2023 and returned to Hobart on May 25, 2023. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage. Data are stored in .raw (219 files 4.87 GB) raw and .seg (218 files 4.85 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes multibeam echosounder data collected on RV Investigator voyage IN2023_V07, [FOCUS] which departed Hobart on the 15/11/23 and returned to Hobart on the 20/12/23. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged sporadically for the extent of the voyage. Data are stored in .raw (231 files 4.87 GB) raw and .seg (236 files 4.83 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on RV Investigator voyage IN2023_V01, titled "Antarctic Bottom Water Production in the past: Records from marine sediments, Cape Darnley, East Antarctica". The voyage took place between 25th January, 2023 and 2nd March, 2023 departing from Henderson's (WA) and arriving in Hobart (TAS). The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage. Data are stored in .raw (2,284 files 41.4 GB) raw and .seg (1,078 files 20.8 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on the RV Investigator voyage IN2023_V04, International nutrient inter-comparison Voyage (INIV) which departed Hobart on the 05/06/23 and returned to Hobart on the 18/06/23. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged sporadically for the extent of the voyage. Data are stored in .raw (247 files 5.39 GB) raw and .seg (255 files 5.34 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request.

This record describes sub bottom profiler data collected on RV Investigator voyage IN2023_V06, titled: "Understanding Eddy Interactions and Their Impacts In the East Australian Current System." The voyage took place between October 9 and November 2, 2023, departing from Sydney (NSW) and returning to Sydney. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged [continuously/sporadically] for the extent of the voyage. Data are stored in .raw (492 files 9.12 GB) raw and .seg (491 files 9.06 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request.

This record describes sub-bottom profiler data collected on RV Investigator voyage IN2022_V09, titled "Valuing Australia’s new Gascoyne Marine Park." The voyage took place between 0000 November 19, 2022 and 0700 December 19, 2022 (UTC), departing from Fremantle and returning to Fremantle. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage. Data are stored in .raw (1736 files 37.8 GB) raw and .seg (1721 files 37.8 GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on RV Investigator voyage IN2019_T01, CAPSTAN which departed Hobart on the 29th April 2019 and returned to Fremantle on the 11th May 2019. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged continuously for the extent of the voyage except for period mapping the shipping sealane from Albany to near Fremantle. Data are stored in .raw (130 files 5.35 GB) raw and .seg (130 files 5GB) segy formats at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request

This record describes sub bottom profiler data collected on RV Investigator voyage IN2017_C02, ['Hogan Group Hydrographic Survey – Bass Strait'] which departed Hobart on the 4th May 2017 and came alongside Bell Bay following completion of the survey on the 14th May 2017. The Kongsberg SBP120 (sub bottom profiler) was used to acquire data containing the specular reflections at different sediment interfaces below the seafloor. The SBP120 provides a 3° by 3° angular resolution. The echosounder's frequency sweep range is from 2.5 to 7 kHz. The SBP120 was logged sporadically for the extent of the voyage. Data are stored in .raw and .seg formats (4537 files 70 GB) at CSIRO. The segy format data had a real time processing stream applied, which applies gain, a gain correction, matched filter with replica shaping, an attribute calculation for instantaneous amplitude and time variable gain. Additional information regarding this dataset is contained in the GSM data acquisition and processing report. Additional data products may be available on request