As one of the research directions at OLIVES Lab @ Georgia Tech, we focus on the robustness of data-driven algorithms under diverse challenging conditions where trained models can possibly be depolyed. To achieve this goal, we introduced a large-sacle (1.M images) object recognition dataset (CURE-OR) which is among the most comprehensive datasets with controlled synthetic challenging conditions. In CURE-OR dataset, there are 1,000,000 images of 100 objects with varying size, color, and texture, captured with multiple devices in different setups. The majority of images in the dataset were acquired with smartphones and tested with off-the-shelf applications to benchmark the recognition performance of devices and applications that are used in our daily lives. Please refer to our GitHub page for code, papers, and more information. Some data specifications are provided below:

Image Name Format :

"backgroundID_deviceID_objectOrientationID_objectID_challengeType_challengeLevel.jpg"

Background ID:

1: White 2: Texture 1 - living room 3: Texture 2 - kitchen 4: 3D 1 - living room 5: 3D 2 – office

Object Orientation ID:

1: Front (0 º) 2: Left side (90 º) 3: Back (180 º) 4: Right side (270 º) 5: Top

Object ID:

1-100

Challenge Type:

No challenge 02: Resize 03: Underexposure 04: Overexposure 05: Gaussian blur 06: Contrast 07: Dirty lens 1 08: Dirty lens 2 09: Salt & pepper noise 10: Grayscale 11: Grayscale resize 12: Grayscale underexposure 13: Grayscale overexposure 14: Grayscale gaussian blur 15: Grayscale contrast 16: Grayscale dirty lens 1 17: Grayscale dirty lens 2 18: Grayscale salt & pepper noise

Challenge Level:

A number between [0, 5], where 0 indicates no challenge, 1 the least severe and 5 the most severe challenge. Challenge type 1 (no challenge) and 10 (grayscale) has a level of 0 only. Challenge types 2 (resize) and 11 (grayscale resize) has 4 levels (1 through 4). All other challenges have levels 1 to 5.

Model Zoo (PyTorch) of non-adversarially trained models for Robust Models are less Over-Confident (NeurIPS'22)

Abstract: "Regardless of the success of convolutional neural networks (CNNs) in many academic benchmarks of computer vision tasks, their application in real-world is still facing fundamental challenges, like the inherent lack of robustness as unveiled by adversarial attacks. These attacks target to manipulate the network's prediction by adding a small amount of noise onto the input. In turn, adversarial training (AT) aims to achieve robustness against such attacks by including adversarial samples in the trainingset. However, a general analysis of the reliability and model calibration of these robust models beyond adversarial robustness is still pending. In this paper, we analyze a variety of adversarially trained models that achieve high robust accuracies when facing state-of-the-art attacks and we show that AT has an interesting side-effect: it leads to models that are significantly less overconfident with their decisions even on clean data than non-robust models. Further, our analysis of robust models shows that not only AT but also the model's building blocks (activation functions and pooling) have a strong influence on the models' confidence."

Top 10 models in terms of RMSE - different out-of-sample periods. Nowcasting: 2 steps ahead.

This repository contains the code and data to replicate all the analyses in our paper "Recalculating ... : How Uncertainty in Local Labor Market Definitions Affects Empirical Findings." Some of the data can also be used in other researchers' analyses to investigate the robustness of their results when they use commuting zones to aggregate or collect data.

Quantitative fatty acid signature analysis (QFASA; Iverson et al. 2004. Ecological Monographs 74:211-235) has become a common method of estimating diet composition, especially for marine mammals, but the performance of the method has received limited investigation. This software was developed to compare the bias of several QFASA estimators using computer simulation and develop recommendations regarding estimator selection (Bromaghin et al. 2015. Assessing the robustness of quantitative fatty acid signature analysis to assumption violations. Methods in Ecology and Evolution (publication expected in late 2015 or early 2016).

This repository contains the data and metadata for accompanying the publication of Santos Neves, P., Lambert, J. W., Valente, L., & Etienne, R. S. (2021). The robustness of a simple dynamic model of island biodiversity to geological and eustatic change. BiorXiv.

All files apart from metadata contained within this repository were obtained via computation at University of Groningen Peregrine High Performance Computing Cluster (HPCC).
Data was generated using the pipeline implemented on the R package DAISIErobustness, which itself greatly depends on the R package DAISIE. The code for these packages is version controlled on GitHub and is freely available in open-source repositories. See Related Identifiers section for links to relevant archived versions of both these packages.

This dataset contains fatty acid (FA) data expressed as mass percent of total FA for bearded seals, ringed seals and walrus. This is one of many datasets used in Bromaghin et al., In press, Assessing the robustness of quantitative fatty acid signature analysis to assumption violations, Methods in Ecology and Evolution. These supplemental data were used in computer simulations to compare the bias of several quantitative fatty acid signature analysis (QFASA) estimators and develop recommendations regarding estimator selection.

In the last 15 years, a complex networks perspective has been increasingly used in the robustness assessment of ecological systems. It is therefore crucial to assess the reliability of such tools. Based on the traditional simulation of node (species) removal, mutualistic pollination networks are considered to be relatively robust because of their 1) truncated power-law degree distribution, 2) redundancy in the number of pollinators per plant and 3) nested interaction pattern. However, species removal is only one of several possible approaches to network robustness assessment. Empirical evidence suggests a decline in abundance prior to the extinction of interacting species, arguing in favour of an interaction removal-based approach (i.e. interaction disruption), as opposed to traditional species removal. For simulated networks, these two approaches yield radically different conclusions, but no tests are currently available for empirical mutualistic networks. This study compared this new robustness evaluation approach based on interaction extinction versus the traditional species removal approach for 12 alpine and subalpine pollination networks. In comparison with species removal, interaction removal produced higher robustness in the worst-case extinction scenario but lower robustness in the best-case extinction scenario. Our results indicate that: 1) these two approaches yield very different conclusions and 2) existing assessments of ecological network robustness could be overly optimistic, at least those based on a disturbance affecting species at random or beginning with the least connected species. Therefore, further empirical study of plant–pollinator interactions in disturbed ecosystems is imperative to understand how pollination networks are disassembled.

The concept of robustness in biology has gained much attention recently, but a mechanistic understanding of how genetic networks regulate phenotypic variation has remained elusive. One approach to understand the genetic architecture of variability has been to analyze dispensable gene deletions in model organisms; however, the most important genes cannot be deleted. Here, we have utilized two systems in yeast whereby essential genes have been altered to reduce expression. Using high‐throughput microscopy and image analysis, we have characterized a large number of morphological phenotypes, and their associated variation, for the majority of essential genes in yeast. Our results indicate that phenotypic robustness is more highly dependent upon the expression of essential genes than on the presence of dispensable genes. Morphological robustness appears to be a general property of a genotype that is closely related to pleiotropy. While the fitness profile across a range of expression levels is idiosyncratic to each gene, the global pattern indicates that there is a window in which phenotypic variation can be released before fitness effects are observable.

Dataset contains CAD model of the robot leg, and data analysis codes. Publication link: https://doi.org/10.48550/arXiv.2212.00475

Abstract
Stochastic noise in gene expression causes variation in the development of phenotypes, making such noise a potential target of stabilizing selection. Here we develop a new simulation model of gene networks to study the adaptive landscape underlying the evolution of robustness to noise. We find that epistatic interactions between the determinants of the expression of a gene and its downstream effect impose significant constraints on evolution, but these interactions do allow the gradual evolution of increased robustness. Despite strong sign epistasis, adaptation rarely proceeds via deleterious intermediate steps, but instead occurs primarily through small beneficial mutations. A simple mathematical model captures the relevant features of the single-gene fitness landscape and explains counterintuitive patterns, such as a correlation between the mean and standard deviation of phenotypes. In more complex networks, mutations in regulatory regions provide evolutionary pathways to increased robustness. These results chart the constraints and possibilities of adaptation to reduce expression noise and demonstrate the potential of a novel modeling framework for gene networks.

This archive contains part 2 of Shift Benchmark on Multiple Sclerosis lesion segmentation data. This dataset is provided by the Shifts Project to enable assessment of the robustness of models to distributional shift and the quality of their uncertainty estimates. This part is contains data collected from several different sources and distributed under a CC BY NC SA 4.0 license. Part 1 of the data is available here. A full description of the benchmark is available in https://arxiv.org/pdf/2206.15407. To find out more about the Shifts Project, please visit https://shifts.ai .

Number of nowcasting and forecasting models selected in the MCS at the 90% confidence level, using the statistic and the MSE loss function, as well as number of selected Google based models.

This page includes R codes for all studies discussed in the manuscript Robustness of normality-based likelihood ratio tests for interaction in two-mode data and a permutation-based alternative.

The ability of a species to cope with both long-term and short-term environmental fluctuations might vary with the species' life history. While some life-history characteristics promote large and stable population sizes despite interannual environmental fluctuations, other life-history strategies might allow to evolve quickly in response to long-term gradual changes. In a theoretical study, we show that there is a tradeoff between both properties. Life-history characteristics that promote fast rates of evolution come at the expense of a poor response to short-term environmental fluctuations, and vice versa. We demonstrated the presence of this tradeoff by the use of a mathematical analysis and individual-based simulations.

These data represent simulated ecological drought conditions for current climate, and for future climate represented by all available climate models at two time periods during the 21st century. These data were used to: 1) describe geographic patterns in ecological drought under historical climate conditions, 2) quantify the direction and magnitude of change in ecological drought, 3) identify areas and ecological drought metrics with projected changes that are robust across climate models, defined as drought metrics and locations where >90% of climate models agree in the direction of change.

The field of numerical algebraic geometry consists of algorithms for numerically solving systems of polynomial equations. When the system is exact, such as having rational coefficients, the solution set is well-defined. However, for a member of a parameterized family of polynomial systems where the parameter values may be measured with imprecision or arise from prior numerical computations, uncertainty may arise in the structure of the solution set, including the number of isolated solutions, the existence of higher dimensional solution components, and the number of irreducible components along with their multiplicities. The loci where these structures change form a stratification of exceptional algebraic sets in the space of parameters. We describe methodologies for making the interpretation of numerical results more robust by searching for nearby parameter values on an exceptional set. We demonstrate these techniques on several illustrative examples and then treat several more substantial problems arising from the kinematics of mechanisms and robots.

Datasets for Semantic Robustness of Models of Source Code.

Includes the c2s/java-small, csn/java, csn/python, and sri/py150 in the following representations:

Raw [in raw.tar.gz]

Normalized [in normalized.tar.gz]

Pre-processed (ast-paths and tokens) [in preprocessed.tar.gz]

Transformed [in transformed.tar.gz]

Normalized

transforms.All

transforms.ShuffleLocalVariables

transforms.ShuffleParameters

transforms.RenameLocalVariables

transforms.RenameFields

transforms.RenameParameters

transforms.ReplaceTrueFalse

transforms.InsertPrintStatements

transforms.Identity

Pre-processed (ast-paths and tokens)

transforms.Identity

transforms.InsertPrintStatements

transforms.ReplaceTrueFalse

transforms.RenameParameters

transforms.RenameFields

transforms.RenameLocalVariables

transforms.ShuffleParameters

transforms.ShuffleLocalVariables

transforms.All

This dataset contains results represented in the work titled 'A Robust, Over-the-Air Test Bed for Radio-Frequency Fingerprinting of Cellular Devices', whose abstract sample is below. We present a characterized test bed and algorithms for non-destructive, over-the-air fingerprinting of commercial cellular user equipment (UE). This test bed is designed to repeatably collect radiated fields from cellular devices in a 4G long term evolution (LTE) network configuration. We describe a straightforward classification algorithm to determine the model of each cellular device that allows for a direct correlation between input data from test cellular phones and identification efficacy. Additionally, by controlling the radio channel conditions, we provide a framework for transparently studying dominant uncertainties and sensitivities in data-driven cellular device fingerprinting. The algorithm performs classification with either the error vector magnitude, a quantity derived from demodulated data, or the out-of-band frequency _domain response of the cellular devices. We have investigated the robustness over time of this fingerprinting method and show over 95% accuracy in identifying UE models from different manufacturers and gaining insight into parameters that can cause a reduction in this level of accuracy and in data-driven approaches in general. This work is part of a larger effort to identify and create a database of genuine off-the-shelf cellular devices to help mitigate counterfeiting and hardware security tampering using RF fingerprinting. As such, the raw data are text files in comma separated value (CSV) format. The text files have varying numbers of columns depending on the figure it is attributed to.

We analyze the effects of neutral and investment-specific technology shocks on hours and output. Long cycles in hours are removed in a variety of ways. Hours robustly fall in response to neutral shocks and robustly increase in response to investment-specific shocks. The percentage of the variance of hours (output) explained by neutral shocks is small (large); the opposite is true for investment-specific shocks. News shocks are uncorrelated with the estimated technology shocks.