TensorFlow's TFRecord Format

## Overview

TensorFlow's TFRecord Format is a dataset for object detection tasks - it contains Traffic Signs annotations for 219 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Dataset

This dataset was created by pa928human

Released under CC0: Public Domain

Contents

Tfrecord Data Format

## Overview

Tfrecord Data Format is a dataset for object detection tasks - it contains Weapons And Animals annotations for 1,333 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Coco To TFRecord

## Overview

Coco To TFRecord is a dataset for object detection tasks - it contains Objects annotations for 677 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Dataset

This dataset was created by pa928human

Released under CC0: Public Domain

Contents

FRC Coco To Tfrecord

## Overview

FRC Coco To Tfrecord is a dataset for object detection tasks - it contains Corals Algae annotations for 418 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

TFRecord Files for Training From the INGC – Predict Volanic Eruptions Competition (https://www.kaggle.com/c/predict-volcanic-eruptions-ingv-oe) - Each TFRecord contains 80 Examples - Each Example is a CSV file from the training dataset of the competition - The following code can decode the TFRecord files into a tf.data.Dataset ``python def decode(serialized_example, is_test=False): """ Parses a set of features and label from the givenserialized_example`.

  It is used as a map function for `dataset.map`

Args:
  serialized_example (tf.Example): A serialized example containing the
    following features:
      – sensor_feature_0 – [int64]
      – sensor_feature_1 – [int64]
      – sensor_feature_2 – [int64]
      – sensor_feature_3 – [int64]
      – sensor_feature_4 – [int64]
      – sensor_feature_5 – [int64]
      – sensor_feature_6 – [int64]
      – sensor_feature_7 – [int64]
      – sensor_feature_8 – [int64]
      – sensor_feature_9 – [int64]
      – label_feature  – int64
  is_test (bool, optional): Whether to allow for the label feature

Returns:
  A decoded tf.data.Dataset object representing the tfrecord dataset
"""
# Defaults are not specified since both keys are required.
feature_dict = {
  'sensor_feature_0': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_1': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_2': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_3': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_4': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_5': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_6': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_7': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_8': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  'sensor_feature_9': tf.io.FixedLenSequenceFeature(shape=(), dtype=tf.int64, allow_missing=True),
  }

if not is_test:
  feature_dict['label_feature'] = tf.io.FixedLenFeature(shape=(), dtype=tf.int64)

# Define a parser
features = tf.io.parse_single_example(serialized_example, features=feature_dict)

# Decode the data and capture the label feature
sensors = [tf.cast(features[f"sensor_feature_{i}"], tf.int16) for i in range(10)]

if is_test:
  return sensors
else:
  label = tf.cast(features["label_feature"], tf.int32)
  return sensors, label

def get_tfrecord_ds(tfrecord_dir): tfrecord_paths = [os.path.join(tfrecord_dir, f_name)
for f_name in os.listdir(tfrecord_dir)
if f_name.endswith('.tfrec')] return tf.data.TFRecordDataset(tfrecord_paths) ```

E621 TFRecords to train classifiers and other stuff with my codebases. TFRecord serialization/deserialization code: NUM_CLASSES = 8783

Function to convert value to bytes_list

def _bytes_feature(value): if isinstance(value, type(tf.constant(0))): value = value.numpy() elif isinstance(value, str): value = value.encode() return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

Function to convert bool/enum/int/uint to int64_listdef… See the full description on the dataset page: https://huggingface.co/datasets/SmilingWolf/e621-v1-tfrecord.

Danbooru TFRecords to train classifiers and other stuff with my codebases. TFRecord serialization/deserialization code: NUM_CLASSES = 12822

Function to convert value to bytes_list

def _bytes_feature(value): if isinstance(value, type(tf.constant(0))): value = value.numpy() elif isinstance(value, str): value = value.encode() return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))

Function to convert bool/enum/int/uint to int64_listdef… See the full description on the dataset page: https://huggingface.co/datasets/SmilingWolf/danbooru-v4-tfrecord.

YOLO To TFrecord

## Overview

YOLO To TFrecord is a dataset for object detection tasks - it contains Drone annotations for 1,337 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [Public Domain license](https://creativecommons.org/licenses/Public Domain).

Dataset

This dataset was created by junseonglee11

Contents

Dataset

This dataset was created by Tyson

Released under CC0: Public Domain

Contents

It contains the following files:

Multimodal dataset (tfrecord file) and trained weight (h5 file) are used to reproduce multi attention weight.

This dataset is built for time-series Sentinel-2 cloud detection and stored in Tensorflow TFRecord (refer to https://www.tensorflow.org/tutorials/load_data/tfrecord).

Each file is compressed in 7z format and can be decompressed using Bandzip or 7-zip software.

Dataset Structure:

Each filename can be split into three parts using underscores. The first part indicates whether it is designated for training or validation ('train' or 'val'); the second part indicates the Sentinel-2 tile name, and the last part indicates the number of samples in this file.

For each sample, it includes:

Sample ID;

Array of time series 4 band image patches in 10m resolution, shaped as (n_timestamps, 4, 42, 42);

Label list indicating cloud cover status for the center (6\times6) pixels of each timestamp;

Ordinal list for each timestamp;

Sample weight list (reserved);

Here is a demonstration function for parsing the TFRecord file:

import tensorflow as tf

init Tensorflow Dataset from file name

def parseRecordDirect(fname): sep = '/' parts = tf.strings.split(fname,sep) tn = tf.strings.split(parts[-1],sep='_')[-2] nn = tf.strings.to_number(tf.strings.split(parts[-1],sep='_')[-1],tf.dtypes.int64) t = tf.data.Dataset.from_tensors(tn).repeat().take(nn) t1 = tf.data.TFRecordDataset(fname) ds = tf.data.Dataset.zip((t, t1)) return ds

keys_to_features_direct = { 'localid': tf.io.FixedLenFeature([], tf.int64, -1), 'image_raw_ldseries': tf.io.FixedLenFeature((), tf.string, ''), 'labels': tf.io.FixedLenFeature((), tf.string, ''), 'dates': tf.io.FixedLenFeature((), tf.string, ''), 'weights': tf.io.FixedLenFeature((), tf.string, '') }

The Decoder (Optional)

class SeriesClassificationDirectDecorder(decoder.Decoder): """A tf.Example decoder for tfds classification datasets.""" def init(self) -> None: super()._init_()

def decode(self, tid, ds): parsed = tf.io.parse_single_example(ds, keys_to_features_direct) encoded = parsed['image_raw_ldseries'] labels_encoded = parsed['labels'] decoded = tf.io.decode_raw(encoded, tf.uint16) label = tf.io.decode_raw(labels_encoded, tf.int8) dates = tf.io.decode_raw(parsed['dates'], tf.int64) weight = tf.io.decode_raw(parsed['weights'], tf.float32) decoded = tf.reshape(decoded,[-1,4,42,42]) sample_dict = { 'tid': tid, # tile ID 'dates': dates, # Date list 'localid': parsed['localid'], # sample ID 'imgs': decoded, # image array 'labels': label, # label list 'weights': weight } return sample_dict

simple function

def preprocessDirect(tid, record): parsed = tf.io.parse_single_example(record, keys_to_features_direct) encoded = parsed['image_raw_ldseries'] labels_encoded = parsed['labels'] decoded = tf.io.decode_raw(encoded, tf.uint16) label = tf.io.decode_raw(labels_encoded, tf.int8) dates = tf.io.decode_raw(parsed['dates'], tf.int64) weight = tf.io.decode_raw(parsed['weights'], tf.float32) decoded = tf.reshape(decoded,[-1,4,42,42]) return tid, dates, parsed['localid'], decoded, label, weight

t1 = parseRecordDirect('filename here') dataset = t1.map(preprocessDirect, num_parallel_calls=tf.data.experimental.AUTOTUNE)

#

Class Definition:

0: clear

1: opaque cloud

2: thin cloud

3: haze

4: cloud shadow

5: snow

Dataset Construction:

First, we randomly generate 500 points for each tile, and all these points are aligned to the pixel grid center of the subdatasets in 60m resolution (eg. B10) for consistence when comparing with other products. It is because that other cloud detection method may use the cirrus band as features, which is in 60m resolution.

Then, the time series image patches of two shapes are cropped with each point as the center.The patches of shape (42 \times 42) are cropped from the bands in 10m resolution (B2, B3, B4, B8) and are used to construct this dataset.And the patches of shape (348 \times 348) are cropped from the True Colour Image (TCI, details see sentinel-2 user guide) file and are used to interpreting class labels.

The samples with a large number of timestamps could be time-consuming in the IO stage, thus the time series patches are divided into different groups with timestamps not exceeding 100 for every group.

tdcsfog train folder data converted to tfrecord Dataset. Since data is greater than 20 GB, it is split into two datasets.

Part 2: https://www.kaggle.com/datasets/coderrkj/parkinson-fog-pred-timeseries-tfrecord-dataset-2

Class Dataset

## Overview

Class Dataset is a dataset for classification tasks - it contains Faces annotations for 6,551 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [Public Domain license](https://creativecommons.org/licenses/Public Domain).

DLTr

## Overview

DLTr is a dataset for classification tasks - it contains Fruits annotations for 3,242 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Collaboration

## Overview

Collaboration is a dataset for classification tasks - it contains Cells annotations for 1,805 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Generate TFRecord For Masks

## Overview

Generate TFRecord For Masks is a dataset for object detection tasks - it contains Masks annotations for 1,340 images.

## Getting Started

You can download this dataset for use within your own projects, or fork it into a workspace on Roboflow to create your own model.

  ## License

  This dataset is available under the [CC BY 4.0 license](https://creativecommons.org/licenses/CC BY 4.0).

Dataset

This dataset was created by Marcos Novaes

Released under CC0: Public Domain

Contents