Abstract

In the last years, neural networks have evolved from laboratory environments to the state-of-the-art for many real-world problems. Our hypothesis is that neural network models (i.e., their weights and biases) evolve on unique, smooth trajectories in weight space during training. Following, a population of such neural network models (refereed to as “model zoo”) would form topological structures in weight space. We think that the geometry, curvature and smoothness of these structures contain information about the state of training and can be reveal latent properties of individual models. With such zoos, one could investigate novel approaches for (i) model analysis, (ii) discover unknown learning dynamics, (iii) learn rich representations of such populations, or (iv) exploit the model zoos for generative modelling of neural network weights and biases. Unfortunately, the lack of standardized model zoos and available benchmarks significantly increases the friction for further research about populations of neural networks. With this work, we publish a novel dataset of model zoos containing systematically generated and diverse populations of neural network models for further research. In total the proposed model zoo dataset is based on six image datasets, consist of 24 model zoos with varying hyperparameter combinations are generated and includes 47’360 unique neural network models resulting in over 2’415’360 collected model states. Additionally, to the model zoo data we provide an in-depth analysis of the zoos and provide benchmarks for multiple downstream tasks as mentioned before.

Dataset

This dataset is part of a larger collection of model zoos and contains the zoos trained on CIFAR10. All zoos with extensive information and code can be found at www.modelzoos.cc.

This repository contains two types of files: the raw model zoos as collections of models (file names beginning with "cifar_"), as well as preprocessed model zoos wrapped in a custom pytorch dataset class (filenames beginning with "dataset"). Zoos are trained with small and large CNN models, in three configurations varying the seed only (seed), varying hyperparameters with fixed seeds (hyp_fix) or varying hyperparameters with random seeds (hyp_rand). The index_dict.json files contain information on how to read the vectorized models.

For more information on the zoos and code to access and use the zoos, please see www.modelzoos.cc.

🖼️ CIFAR10 (Extracted from PyTorch Vision)

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

  ℹ️ Dataset Details





  📖 Dataset Description

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The classes are completely mutually exclusive. There is no… See the full description on the dataset page: https://huggingface.co/datasets/p2pfl/CIFAR10.

Different Artificial Neural Networks (saved weights), some only pre-trained either on rwave-1024 or rwave-4096 or FractalDB1000 datasets; some fine-tuned or trained from scratch (pt_none_ft... or pt_ft... or ...scratch...) on CIFAR10/100 or ImageNet1k. Retinal Waves for Pre-Training Artificial Neural Networks Mimicking Real Prenatal Development - see https://github.com/BennyCa/ReWaRD for filter visualization and further fine-tuning possibilities Pre-training and fine-tuning was conducted using the codebase https://github.com/hirokatsukataoka16/FractalDB-Pretrained-ResNet-PyTorch

Dataset Specifications

Contains the entire CIFAR10 dataset, downloaded via PyTorch, then split and saved as .png files representing 32x32 images. There a three splits, perfectly balanced class-wise:

train: 49,000 out of the original 50,000 samples from the training set of CIFAR10; calibration: 1,000 left-out samples from the training set; test: 10,000 samples, the entire original test set.

Every sample has a unique filename XXX.png where XXX goes from 0 to 59,999.

Cifar10Corrupted is a dataset generated by adding 15 common corruptions + 4 extra corruptions to the test images in the Cifar10 dataset. This dataset wraps the corrupted Cifar10 test images uploaded by the original authors.

To use this dataset:

import tensorflow_datasets as tfds

ds = tfds.load('cifar10_corrupted', split='train')
for ex in ds.take(4):
 print(ex)

See the guide for more informations on tensorflow_datasets.

https://storage.googleapis.com/tfds-data/visualization/fig/cifar10_corrupted-brightness_1-1.0.0.png" alt="Visualization" width="500px">

Dataset Card for cifar10_augmented

This dataset contains augmented versions of CIFAR-10 to benchmark the impact of classical and synthetic augmentation techniques on deep image classification models. Provided as a .zip file, the dataset must be unzipped before use. It follows a standard ImageFolder structure for compatibility with PyTorch and TensorFlow pipelines.

  Dataset Details





  Dataset Sources

Repository:… See the full description on the dataset page: https://huggingface.co/datasets/ianisdev/cifar10_augmented.