The OECD has initiated PISA for Development (PISA-D) in response to the rising need of developing countries to collect data about their education systems and the capacity of their student bodies. This report aims to compare and contrast approaches regarding the instruments that are used to collect data on (a) component skills and cognitive instruments, (b) contextual frameworks, and (c) the implementation of the different international assessments, as well as approaches to include children who are not at school, and the ways in which data are used. It then seeks to identify assessment practices in these three areas that will be useful for developing countries. This report reviews the major international and regional large-scale educational assessments: large-scale international surveys, school-based surveys and household-based surveys. For each of the issues discussed, there is a description of the prevailing international situation, followed by a consideration of the issue for developing countries and then a description of the relevance of the issue to PISA for Development.

This dataset presents the quantitative raw data that was collected under the H2020 RRI2SCALE project for the D1.4 - “Large scale regional citizen surveys report”. The dataset includes the answers that were provided by almost 8,000 participants from 4 pilot European regions (Kriti, Vestland, Galicia, and Overijssel) regarding the general public's views, concerns, and moral issues about the current and future trajectories of their RTD&I ecosystem. The original survey questionnaire was created by White Research SRL and disseminated to the regions through supporting pilot partners. Data collection took place from June 2020 to September 2020 through 4 different waves – one for each region. Based on the conclusion of a consortium vote during the kick-off meeting, it was decided that instead of resource-intensive methods that would render data collection unduly expensive, to fill in the quotas responses were collected through online panels by survey companies that were used for each region. For the statistical analysis of the data and the conclusions drawn from the analysis, you can access the "Large scale regional citizen surveys report" (D1.4).

BackgroundLarge cross-sectional household surveys are common for measuring indicators of neglected tropical disease control programs. As an alternative to standard paper-based data collection, we utilized novel paperless technology to collect data electronically from over 12,000 households in Ethiopia.MethodologyWe conducted a needs assessment to design an Android-based electronic data collection and management system. We then evaluated the system by reporting results of a pilot trial and from comparisons of two, large-scale surveys; one with traditional paper questionnaires and the other with tablet computers, including accuracy, person-time days, and costs incurred.Principle FindingsThe electronic data collection system met core functions in household surveys and overcame constraints identified in the needs assessment. Pilot data recorders took 264 (standard deviation (SD) 152 sec) and 260 sec (SD 122 sec) per person registered to complete household surveys using paper and tablets, respectively (P = 0.77). Data recorders felt a lack of connection with the interviewee during the first days using electronic devices, but preferred to collect data electronically in future surveys. Electronic data collection saved time by giving results immediately, obviating the need for double data entry and cross-correcting. The proportion of identified data entry errors in disease classification did not differ between the two data collection methods. Geographic coordinates collected using the tablets were more accurate than coordinates transcribed on a paper form. Costs of the equipment required for electronic data collection was approximately the same cost incurred for data entry of questionnaires, whereas repeated use of the electronic equipment may increase cost savings.Conclusions/SignificanceConducting a needs assessment and pilot testing allowed the design to specifically match the functionality required for surveys. Electronic data collection using an Android-based technology was suitable for a large-scale health survey, saved time, provided more accurate geo-coordinates, and was preferred by recorders over standard paper-based questionnaires.

Index to the BGS collection of large scale or large format plans of all types including those relating to mining activity, including abandonment plans and site investigations. The Plans Database Index was set up c.1983 as a digital index to the collections of Land Survey Plans and Plans of Abandoned Mines. There are entries for all registered plans but not all the index fields are complete, as this depends on the nature of the original plan. The index covers the whole of Great Britain.

Throughout the Upper Mississippi River System (UMRS), floods in 1993 and 2019 were record-setting events in terms of flood stage and duration of flooding. Floodplain tree species are adapted to survive moderate frequencies, intensities, and durations of inundation. These events generated interest in quantifying levels of tree mortality and presented an opportunity to study how differences in flood attributes may affect tree survivorship and forest community dynamics post-flood. In 1995, 547 plots randomly located within forested land cover were established within seven reaches on the Upper Mississippi River (Pool 4, Pool 8, Pool 13, Pool 17, Pool 22, Pool 26, Open River) and one reach of the Illinois River (La Grange Pool). In 2021, 39-46 sites (n = 342) from each of the eight reaches were navigated to using GPS coordinates from the original study and resurveyed using the same field sampling protocol used in 1995. In many cases (n = 287), the original posts and tree tags installed in 1995 could not be found and thus plots in 2021 were re-established by navigated as close to the original GPS coordinates as possible and referencing the 1995 data. In some of these cases (n = 24), plots were inaccessible (e.g., in a water body) due to landform changes that had occurred in the intervening years between field sample collections, and in these instances, plots were relocated to a nearby site with similar forest composition, elevation, and river geomorphology. Plots were re-established in order to maintain a consistent number of plots per reach for potential future monitoring, however, those that were moved were noted along with original 1995 coordinates in the provided ‘Plot_2021.csv’ file and can be included or removed in subsequent analyses depending on research questions and goals. Plots were sampled using a fixed radius sampling methodology with a radius of 10 meters, and data collected included: overstory species composition, diameters, and health status; canopy cover; seedling counts and composition; cover classes for different forest height strata; ground cover; and herbaceous cover. Seedlings, ground cover, and herbaceous cover were measured within ten randomly located 0.5 m x 0.5 m subplots.

This study engaged 409 participants over a period spanning from July 10 to August 8, 2023, ensuring representation across various demographic factors: 221 females, 186 males, 2 non-binary, year of birth between 1951 and 2005, with varied annual incomes and from 15 Spanish regions. The MobileWell400+ dataset, openly accessible, encompasses a wide array of data collected via the participants' mobile phone, including demographic, emotional, social, behavioral, and well-being data. Methodologically, the project presents a promising avenue for uncovering new social, behavioral, and emotional indicators, supplementing existing literature. Notably, artificial intelligence is considered to be instrumental in analysing these data, discerning patterns, and forecasting trends, thereby advancing our comprehension of individual and population well-being. Ethical standards were upheld, with participants providing informed consent.

The following is a non-exhaustive list of collected data:

Data continuously collected through the participants' smartphone sensors: physical activity (resting, walking, driving, cycling, etc.), name of detected WiFi networks, connectivity type (WiFi, mobile, none), ambient light, ambient noise, and status of the device screen (on, off, locked, unlocked).

Data corresponding to an initial survey prompted via the smartphone, with information related to demographic data, effects and COVID vaccination, average hours of physical activity, and answers to a series of questions to measure mental health, many of them taken from internationally recognised psychological and well-being scales (PANAS, PHQ, GAD, BRS and AAQ), social isolation (TILS) and economic inequality perception.

Data corresponding to daily surveys prompted via the smartphone, where variables related to mood (valence, activation, energy and emotional events) and social interaction (quantity and quality) are measured.

Data corresponding to weekly surveys prompted via the smartphone, where information on overall health, hours of physical activity per week, lonileness, and questions related to well-being are asked.

Data corresponding to an final survey prompted via the smartphone, consisting of similar questions to the ones asked in the initial survey, namely psychological and well-being items (PANAS, PHQ, GAD, BRS and AAQ), social isolation (TILS) and economic inequality perception questions.

For a more detailed description of the study please refer to MobileWell400+StudyDescription.pdf.

For a more detailed description of the collected data, variables and data files please refer to MobileWell400+FilesDescription.pdf.

This is a test collection for passage and document retrieval, produced in the TREC 2023 Deep Learning track. The Deep Learning Track studies information retrieval in a large training data regime. This is the case where the number of training queries with at least one positive label is at least in the tens of thousands, if not hundreds of thousands or more. This corresponds to real-world scenarios such as training based on click logs and training based on labels from shallow pools (such as the pooling in the TREC Million Query Track or the evaluation of search engines based on early precision).Certain machine learning based methods, such as methods based on deep learning are known to require very large datasets for training. Lack of such large scale datasets has been a limitation for developing such methods for common information retrieval tasks, such as document ranking. The Deep Learning Track organized in the previous years aimed at providing large scale datasets to TREC, and create a focused research effort with a rigorous blind evaluation of ranker for the passage ranking and document ranking tasks.Similar to the previous years, one of the main goals of the track in 2022 is to study what methods work best when a large amount of training data is available. For example, do the same methods that work on small data also work on large data? How much do methods improve when given more training data? What external data and models can be brought in to bear in this scenario, and how useful is it to combine full supervision with other forms of supervision?The collection contains 12 million web pages, 138 million passages from those web pages, search queries, and relevance judgments for the queries.

This paper documents the findings of the March 12-14, 2001 Workshop on New Visions for Large-Scale Networks: Research and Applications. The workshops objectives were to develop a vision for the future of networking 10 to 20 years out and to identify needed Federal networking research to enable that vision...

Index to BGS geological map 'Standards', manuscript and published maps for Great Britain produced by the Survey on County Series (1:10560) and National Grid (1:10560 & 1:10000) Ordnance Survey base maps. 'Standards' are the best interpretation of the geology at the time they were produced. The Oracle index was set up in 1988, current holdings are over 41,000 maps. There are entries for all registered maps, but not all fields are complete on all entries.

Version 162 of the dataset. NOTES: Data for 3/15 - 3/18 was not extracted due to unexpected and unannounced downtime of our university infrastructure. We will try to backfill those days by next release. FUTURE CHANGES: Due to the imminent paywalling of Twitter's API access this might be the last full update of this dataset. If the API access is not blocked, we will be stopping updates for this dataset with release 165 - a bit more than 3 years after our initial release. It's been a joy seeing all the work that uses this resource and we are glad that so many found it useful.

The dataset files: full_dataset.tsv.gz and full_dataset_clean.tsv.gz have been split in 1 GB parts using the Linux utility called Split. So make sure to join the parts before unzipping. We had to make this change as we had huge issues uploading files larger than 2GB's (hence the delay in the dataset releases). The peer-reviewed publication for this dataset has now been published in Epidemiologia an MDPI journal, and can be accessed here: https://doi.org/10.3390/epidemiologia2030024. Please cite this when using the dataset.

Due to the relevance of the COVID-19 global pandemic, we are releasing our dataset of tweets acquired from the Twitter Stream related to COVID-19 chatter. Since our first release we have received additional data from our new collaborators, allowing this resource to grow to its current size. Dedicated data gathering started from March 11th yielding over 4 million tweets a day. We have added additional data provided by our new collaborators from January 27th to March 27th, to provide extra longitudinal coverage. Version 10 added ~1.5 million tweets in the Russian language collected between January 1st and May 8th, gracefully provided to us by: Katya Artemova (NRU HSE) and Elena Tutubalina (KFU). From version 12 we have included daily hashtags, mentions and emoijis and their frequencies the respective zip files. From version 14 we have included the tweet identifiers and their respective language for the clean version of the dataset. Since version 20 we have included language and place location for all tweets.

The data collected from the stream captures all languages, but the higher prevalence are: English, Spanish, and French. We release all tweets and retweets on the full_dataset.tsv file (1,395,222,801 unique tweets), and a cleaned version with no retweets on the full_dataset-clean.tsv file (361,748,721 unique tweets). There are several practical reasons for us to leave the retweets, tracing important tweets and their dissemination is one of them. For NLP tasks we provide the top 1000 frequent terms in frequent_terms.csv, the top 1000 bigrams in frequent_bigrams.csv, and the top 1000 trigrams in frequent_trigrams.csv. Some general statistics per day are included for both datasets in the full_dataset-statistics.tsv and full_dataset-clean-statistics.tsv files. For more statistics and some visualizations visit: http://www.panacealab.org/covid19/

More details can be found (and will be updated faster at: https://github.com/thepanacealab/covid19_twitter) and our pre-print about the dataset (https://arxiv.org/abs/2004.03688)

As always, the tweets distributed here are only tweet identifiers (with date and time added) due to the terms and conditions of Twitter to re-distribute Twitter data ONLY for research purposes. They need to be hydrated to be used.

3D Map Data Collection Sensor Market Outlook

According to our latest research, the global 3D Map Data Collection Sensor market size reached USD 4.2 billion in 2024, with a robust year-on-year growth rate. The market is anticipated to expand at a CAGR of 13.7% from 2025 to 2033, culminating in a forecasted market value of USD 13.1 billion by 2033. The major growth driver for this market is the increasing demand for high-resolution geospatial data across industries such as automotive, urban planning, and environmental monitoring, propelled by advancements in sensor technologies and the proliferation of autonomous systems worldwide.

The primary growth factor fueling the 3D Map Data Collection Sensor market is the rapid adoption of autonomous vehicles and advanced driver-assistance systems (ADAS) in the automotive sector. As automotive manufacturers strive to enhance safety and navigation capabilities, the integration of LiDAR, radar, and high-definition cameras has become indispensable. These sensors are critical for real-time 3D mapping, object detection, and environmental perception, enabling vehicles to operate autonomously with greater accuracy and reliability. Additionally, the surge in demand for electric vehicles and connected mobility solutions further amplifies the need for sophisticated 3D mapping technologies, driving sustained investment and innovation in sensor development.

Another significant growth catalyst is the widespread application of 3D mapping in urban planning, construction, and infrastructure management. Governments and private enterprises are increasingly leveraging 3D map data collection sensors for smart city initiatives, land surveying, and construction project management. These sensors enable accurate spatial data acquisition, facilitating efficient planning, design, and monitoring of urban environments. The integration of aerial and mobile platforms with advanced sensor arrays allows for rapid, large-scale data collection, supporting infrastructure development and environmental sustainability goals. As urbanization accelerates globally, the demand for precise 3D mapping solutions is expected to rise exponentially.

Technological advancements in sensor miniaturization, data processing, and cloud-based analytics are also propelling the market forward. The evolution of compact, high-performance sensors has made it feasible to deploy 3D map data collection systems across diverse platforms, including unmanned aerial vehicles (UAVs), terrestrial vehicles, and handheld devices. Enhanced data fusion techniques and artificial intelligence-driven analytics are enabling real-time processing and interpretation of vast geospatial datasets, unlocking new use cases in agriculture, disaster management, and environmental monitoring. These innovations are reducing operational costs, improving data accuracy, and expanding the accessibility of 3D mapping technologies to a broader spectrum of end-users.

Regionally, North America continues to dominate the 3D Map Data Collection Sensor market, accounting for the largest revenue share in 2024, followed closely by Europe and Asia Pacific. The presence of leading technology companies, robust R&D investments, and early adoption of autonomous solutions are key factors contributing to the region's market leadership. Meanwhile, the Asia Pacific region is witnessing the fastest growth, driven by rapid urbanization, infrastructure development, and increasing investments in smart city projects. Emerging markets in Latin America and the Middle East & Africa are also exhibiting promising growth potential, supported by government initiatives and expanding industrial applications.

Sensor Type Analysis

The 3D Map Data Collection Sensor market is segmented by sensor type into LiDAR, radar, camera, GNSS, ultrasonic, and others, each playing a pivotal role in the acquisition of spatial data. LiDAR sensors have emerged as the most prominent segment due to their exceptional ability to generate high-resolution, acc

Count of total number of Large-Scale Loach (Paramisgurnus dabryanus) observed in a two hour period

Roads are one of the most transforming linear infrastructures in human-dominated landscapes, with animal road-kills as their most studied impact. Therefore, there is the need to gather road-kill data and in this sense, citizen science is gaining popularity as an easy and cheap source of data collection that allows large scale studies that may otherwise be unattainable. However, citizen science projects that focus on road-kills tends to be geographically localised, therefore, there is a debate about whether large-scale data collected by citizen scientists can identify spatial and temporal road-kill patterns, and thus, be used as a reliable conservation tool. We aim to assess whether citizen science data contained in the Spanish Atlas of Terrestrial Mammals (henceforth “Atlas”), can be as valuable and accurate as road-kill surveys undertaken by experts in detecting road-kill hotspots and establishing road-kill rates for different species of carnivores. Using Linear Models, we compared species-richness, diversity and abundance of road-killed carnivores between Atlas data and our own road-kill survey database. We also compared (per species) the observed road-kills in our road survey with the expected road-kills based on the species abundance from the Atlas. In our Linear Models we did not find a significant relation between the road-kill data and the Atlas data. This suggests that data from the Atlas are unsuitable to determine road-kills patterns in our study area. This could be due to the lack of control over the sampling effort in the Atlas data, and the fact that the Atlas has a sampling scope that is not fitted for road mortality studies. When we compared observed road-kills (per species) with those expected based on Atlas abundance, we found that some species are road-killed more (or less) than expected. This may be due to ecological or behavioural traits that make some species more (or less) prone to be road-killed. To summarize, our findings suggest that occurrence in Atlas data does not mirror road-kill patterns, likely due to both several biases in Atlas data and to species-specific responses to roads. Thus, to study road-kill rates and patterns, we suggest the use classical road-kill surveys, unless correcting approaches to citizen science datasets are applied. This is especially important when the study aims to determine species’ specific road-kill patterns.

Dataset that contains vessel position information transmitted by vessels of different types and collected via the Automatic Identification System (AIS). The AIS dataset comes along with spatially and temporally correlated data about the vessels and the area of interest, including weather information

The motivation for the preprocessing of large-scale CAD models for assembly-by-disassembly approaches. The assembly-by-disassembly is only suitable for assemblies with a small number of parts (n_{parts} < 22). However, when dealing with large-scale products with high complexity, the CAD models may not contain feasible subassemblies (e.g. with connected and interference-free parts) and have too many parts to be processed with assembly-by-disassembly. Product designers' preferences during the design phase might not be ideal for assembly-by-disassembly processing because they do not consider subassembly feasibility and the number of parts per subassembly concisely. An automated preprocessing approach is proposed to address this issue by splitting the model into manageable partitions using community detection. This will allow for parallelised, efficient and accurate assembly-by-disassembly of large-scale CAD models. However, applying community detection methods for automatically splitting CAD models into smaller subassemblies is a new concept and research on the suitability for ASP needs to be conducted. Therefore, the following underlying research question will be answered in this experiments:

Underlying research question 2: Can automated preprocessing increase the suitability of CAD-based assembly-by-disassembly for large-scale products?

A hypothesis is formulated to answer this research question, which will be utilised to design experiments for hypothesis testing.

Hypothesis 2: Community detection algorithms can be applied to automatically split large-scale assemblies in suitable candidates for CAD-based AND/OR graph generation.}

Current discussions of the ethical aspects of big data are shaped by concerns regarding the social consequences of both the widespread adoption of machine learning and the ways in which biases in data can be replicated and perpetuated. We instead focus here on the ethical issues arising from the use of big data in international neuroscience collaborations. Neuroscience innovation relies upon neuroinformatics, large-scale data collection and analysis enabled by novel and emergent technologies. Each step of this work involves aspects of ethics, ranging from concerns for adherence to informed consent or animal protection principles and issues of data re-use at the stage of data collection, to data protection and privacy during data processing and analysis, and issues of attribution and intellectual property at the data-sharing and publication stages. Significant dilemmas and challenges with far-reaching implications are also inherent, including reconciling the ethical imperative for openness and validation with data protection compliance and considering future innovation trajectories or the potential for misuse of research results. Furthermore, these issues are subject to local interpretations within different ethical cultures applying diverse legal systems emphasising different aspects. Neuroscience big data require a concerted approach to research across boundaries, wherein ethical aspects are integrated within a transparent, dialogical data governance process. We address this by developing the concept of “responsible data governance,” applying the principles of Responsible Research and Innovation (RRI) to the challenges presented by the governance of neuroscience big data in the Human Brain Project (HBP).

Login Data Set for Risk-Based Authentication

Synthesized login feature data of >33M login attempts and >3.3M users on a large-scale online service in Norway. Original data collected between February 2020 and February 2021.

This data sets aims to foster research and development for Risk-Based Authentication (RBA) systems. The data was synthesized from the real-world login behavior of more than 3.3M users at a large-scale single sign-on (SSO) online service in Norway.

The users used this SSO to access sensitive data provided by the online service, e.g., a cloud storage and billing information. We used this data set to study how the Freeman et al. (2016) RBA model behaves on a large-scale online service in the real world (see Publication). The synthesized data set can reproduce these results made on the original data set (see Study Reproduction). Beyond that, you can use this data set to evaluate and improve RBA algorithms under real-world conditions.

WARNING: The feature values are plausible, but still totally artificial. Therefore, you should NOT use this data set in productive systems, e.g., intrusion detection systems.

Overview

The data set contains the following features related to each login attempt on the SSO:

Feature	Data Type	Description	Range or Example
IP Address	String	IP address belonging to the login attempt	0.0.0.0 - 255.255.255.255
Country	String	Country derived from the IP address	US
Region	String	Region derived from the IP address	New York
City	String	City derived from the IP address	Rochester
ASN	Integer	Autonomous system number derived from the IP address	0 - 600000
User Agent String	String	User agent string submitted by the client	Mozilla/5.0 (Windows NT 10.0; Win64; ...
OS Name and Version	String	Operating system name and version derived from the user agent string	Windows 10
Browser Name and Version	String	Browser name and version derived from the user agent string	Chrome 70.0.3538
Device Type	String	Device type derived from the user agent string	(mobile, desktop, tablet, bot, unknown)1
User ID	Integer	Idenfication number related to the affected user account	[Random pseudonym]
Login Timestamp	Integer	Timestamp related to the login attempt	[64 Bit timestamp]
Round-Trip Time (RTT) [ms]	Integer	Server-side measured latency between client and server	1 - 8600000
Login Successful	Boolean	True: Login was successful, False: Login failed	(true, false)
Is Attack IP	Boolean	IP address was found in known attacker data set	(true, false)
Is Account Takeover	Boolean	Login attempt was identified as account takeover by incident response team of the online service	(true, false)

Data Creation

As the data set targets RBA systems, especially the Freeman et al. (2016) model, the statistical feature probabilities between all users, globally and locally, are identical for the categorical data. All the other data was randomly generated while maintaining logical relations and timely order between the features.

The timestamps, however, are not identical and contain randomness. The feature values related to IP address and user agent string were randomly generated by publicly available data, so they were very likely not present in the real data set. The RTTs resemble real values but were randomly assigned among users per geolocation. Therefore, the RTT entries were probably in other positions in the original data set.

• The country was randomly assigned per unique feature value. Based on that, we randomly assigned an ASN related to the country, and generated the IP addresses for this ASN. The cities and regions were derived from the generated IP addresses for privacy reasons and do not reflect the real logical relations from the original data set.

• The device types are identical to the real data set. Based on that, we randomly assigned the OS, and based on the OS the browser information. From this information, we randomly generated the user agent string. Therefore, all the logical relations regarding the user agent are identical as in the real data set.

• The RTT was randomly drawn from the login success status and synthesized geolocation data. We did this to ensure that the RTTs are realistic ones.

Regarding the Data Values

Due to unresolvable conflicts during the data creation, we had to assign some unrealistic IP addresses and ASNs that are not present in the real world. Nevertheless, these do not have any effects on the risk scores generated by the Freeman et al. (2016) model.

You can recognize them by the following values:

• ASNs with values >= 500.000

• IP addresses in the range 10.0.0.0 - 10.255.255.255 (10.0.0.0/8 CIDR range)

Study Reproduction

Based on our evaluation, this data set can reproduce our study results regarding the RBA behavior of an RBA model using the IP address (IP address, country, and ASN) and user agent string (Full string, OS name and version, browser name and version, device type) as features.

The calculated RTT significances for countries and regions inside Norway are not identical using this data set, but have similar tendencies. The same is true for the Median RTTs per country. This is due to the fact that the available number of entries per country, region, and city changed with the data creation procedure. However, the RTTs still reflect the real-world distributions of different geolocations by city.

See RESULTS.md for more details.

Ethics

By using the SSO service, the users agreed in the data collection and evaluation for research purposes. For study reproduction and fostering RBA research, we agreed with the data owner to create a synthesized data set that does not allow re-identification of customers.

The synthesized data set does not contain any sensitive data values, as the IP addresses, browser identifiers, login timestamps, and RTTs were randomly generated and assigned.

Publication

You can find more details on our conducted study in the following journal article:

Pump Up Password Security! Evaluating and Enhancing Risk-Based Authentication on a Real-World Large-Scale Online Service (2022)
Stephan Wiefling, Paul René Jørgensen, Sigurd Thunem, and Luigi Lo Iacono.
ACM Transactions on Privacy and Security

Bibtex

@article{Wiefling_Pump_2022,
 author = {Wiefling, Stephan and Jørgensen, Paul René and Thunem, Sigurd and Lo Iacono, Luigi},
 title = {Pump {Up} {Password} {Security}! {Evaluating} and {Enhancing} {Risk}-{Based} {Authentication} on a {Real}-{World} {Large}-{Scale} {Online} {Service}},
 journal = {{ACM} {Transactions} on {Privacy} and {Security}},
 doi = {10.1145/3546069},
 publisher = {ACM},
 year  = {2022}
}

License

This data set and the contents of this repository are licensed under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. See the LICENSE file for details. If the data set is used within a publication, the following journal article has to be cited as the source of the data set:

Stephan Wiefling, Paul René Jørgensen, Sigurd Thunem, and Luigi Lo Iacono: Pump Up Password Security! Evaluating and Enhancing Risk-Based Authentication on a Real-World Large-Scale Online Service. In: ACM Transactions on Privacy and Security (2022). doi: 10.1145/3546069

1. Few (invalid) user agents strings from the original data set could not be parsed, so their device type is empty. Perhaps this parse error is useful information for your studies, so we kept these 1526 entries.↩︎

Water features that relate to the interior of the country. Multiple points that describe a feature’s centreline or edge. NOTE: Landgate no longer maintains large scale topographic features. The large scale topographic data capture programme ceased in 2016. Please consider carefully the suitability of the data within this service for your purpose. © Western Australian Land Information Authority (Landgate). Use of Landgate data is subject to Personal Use License terms and conditions unless otherwise authorised under approved License terms and conditions.

The self-documenting aspects and the ability to reproduce results have been touted as significant benefits of Jupyter Notebooks. At the same time, there has been growing criticism that the way notebooks are being used leads to unexpected behavior, encourages poor coding practices and that their results can be hard to reproduce. To understand good and bad practices used in the development of real notebooks, we analyzed 1.4 million notebooks from GitHub. Based on the results, we proposed and evaluated Julynter, a linting tool for Jupyter Notebooks.

Papers:

• PIMENTEL, J. F.; MURTA, L.; BRAGANHOLO, V.; FREIRE, J.; A large-scale study about quality and reproducibility of jupyter notebooks. In: International Conference on Mining Software Repositories (MSR), 2019, Montreal, Canada.
• PIMENTEL, J. F.; MURTA, L.; BRAGANHOLO, V.; FREIRE, J.; Understanding and Improving the Quality and Reproducibility of Jupyter Notebooks. Empirical Software Engineering, 2021 (in press)

This repository contains three files:

• db2020-09-22.dump.gz
• sample.tar.gz
• julynter_reproducility.tar.gz

Reproducing the Notebook Study

The db2020-09-22.dump.gz file contains a PostgreSQL dump of the database, with all the data we extracted from notebooks. For loading it, run:

gunzip -c db2020-09-22.dump.gz | psql jupyter

Note that this file contains only the database with the extracted data. The actual repositories are available in a google drive folder, which also contains the docker images we used in the reproducibility study. The repositories are stored as content/{hash_dir1}/{hash_dir2}.tar.bz2, where hash_dir1 and hash_dir2 are columns of repositories in the database.

For scripts, notebooks, and detailed instructions on how to analyze or reproduce the data collection, please check the instructions on the Jupyter Archaeology repository (tag 1.0.0)

The sample.tar.gz file contains the repositories obtained during the manual sampling.

Reproducing the Julynter Experiment

The julynter_reproducility.tar.gz file contains all the data collected in the Julynter experiment and the analysis notebooks. Reproducing the analysis is straightforward:

• Uncompress the file: $ tar zxvf julynter_reproducibility.tar.gz
• Install the dependencies: $ pip install julynter/requirements.txt
• Run the notebooks in order: J1.Data.Collection.ipynb; J2.Recommendations.ipynb; J3.Usability.ipynb.

The collected data is stored in the julynter/data folder.

Changelog

2019/01/14 - Version 1 - Initial version
2019/01/22 - Version 2 - Update N8.Execution.ipynb to calculate the rate of failure for each reason
2019/03/13 - Version 3 - Update package for camera ready. Add columns to db to detect duplicates, change notebooks to consider them, and add N1.Skip.Notebook.ipynb and N11.Repository.With.Notebook.Restriction.ipynb.
2021/03/15 - Version 4 - Add Julynter experiment; Update database dump to include new data collected for the second paper; remove scripts and analysis notebooks from this package (moved to GitHub), add a link to Google Drive with collected repository files

Portable Traffic Data Collection Systems Market Outlook

According to our latest research, the global market size for Portable Traffic Data Collection Systems reached USD 1.62 billion in 2024, and is anticipated to expand at a CAGR of 8.1% from 2025 to 2033. By the end of the forecast period, the market is projected to achieve a value of USD 3.23 billion by 2033. The primary growth factor driving this market is the increasing demand for real-time and accurate traffic data to support smart city initiatives, urban mobility planning, and enhanced road safety measures, as per our latest research and industry analysis.

One of the most significant growth drivers for the Portable Traffic Data Collection Systems market is the rapid urbanization and the consequent rise in vehicular density across major cities globally. As urban centers continue to expand, the need for efficient traffic management has become paramount. Governments and urban planners are increasingly relying on advanced traffic data collection to optimize signal timings, reduce congestion, and improve road safety. The ability of portable systems to be quickly deployed and relocated makes them ideal for dynamic and temporary data collection needs, such as during roadworks, special events, or in areas experiencing sudden changes in traffic flow. Furthermore, the integration of these systems with smart city frameworks and intelligent transportation systems (ITS) has amplified their adoption, as municipalities strive for data-driven decision-making to address urban mobility challenges.

Technological advancements are also propelling the market forward. Innovations in sensor technologies, data analytics, wireless communication, and cloud-based platforms have significantly enhanced the accuracy, reliability, and flexibility of portable traffic data collection systems. Modern systems now offer real-time data transmission, remote monitoring, and seamless integration with existing traffic management infrastructure. This has enabled stakeholders to access actionable insights quickly and efficiently, supporting proactive interventions and policy formulation. The shift towards video-based and radar-based solutions, in particular, is driven by their ability to provide granular data on vehicle speed, classification, and count, further fueling market growth. As the cost of these technologies continues to decline, their adoption is expected to increase across both developed and developing regions.

Another key factor contributing to market expansion is the increased focus on sustainable transportation and environmental monitoring. Portable traffic data collection systems are being leveraged to assess the impact of traffic on air quality, noise pollution, and carbon emissions. This data is critical for designing low-emission zones, promoting public transportation, and implementing congestion pricing schemes. Additionally, the growing trend of public-private partnerships in the transportation sector has spurred investments in advanced traffic monitoring solutions. Private companies, research institutes, and urban planning organizations are collaborating to develop innovative applications, further diversifying the market landscape. As regulatory frameworks evolve to mandate comprehensive traffic data collection for infrastructure projects, the demand for portable solutions is expected to witness sustained growth.

Regionally, North America currently dominates the Portable Traffic Data Collection Systems market, accounting for the largest share in 2024. This leadership position is attributed to the early adoption of smart traffic management technologies, substantial government investments in infrastructure modernization, and the presence of leading technology providers. Europe follows closely, driven by stringent regulatory requirements for road safety and environmental monitoring. The Asia Pacific region, however, is poised for the fastest growth during the forecast period, fueled by rapid urbanization, increasing vehicle ownership, and large-scale smart city projects in countries such as China, India, and Japan. Latin America and the Middle East & Africa are also witnessing growing adoption, supported by urban development initiatives and cross-border transportation projects.

Product Type Analysis

The Product Type segment of the Portable Traffic Data Collection Systems market is highly diversified, encompassing radar-based sys