Amazon Bin Image Dataset

The Amazon Bin Image Dataset contains 536,434 images and metadata from bins of a pod in an operating Amazon Fulfillment Center. The bin images in this dataset are captured as robot units carry pods as part of normal Amazon Fulfillment Center operations. This dataset has many images and the corresponding medadata.

The image files have three groups according to its naming scheme.

• A file name with 1~4 digits (1,200): 1.jpg ~ 1200.jpg
• A file name with 5 digits (99,999): 00001.jpg ~ 99999.jpg
• A file name with 6 digits (435,235): 100000.jpg ~ 535234.jpg

Amazon Bin Image Dataset File List dataset aims to provide a CSV file to contain all file locations and the quantity to help the analysis and distributed learning.

Documentation

• https://github.com/awslabs/open-data-docs/tree/main/docs/aft-vbi-pds

Download

• https://registry.opendata.aws/amazon-bin-imagery/
• https://github.com/awslabs/open-data-registry/blob/main/datasets/amazon-bin-imagery.yaml

GHCN

• The Global Historical Climatology Network (GHCN) is an integrated database of climate summaries from land surface stations across the globe.
• GHCN-Daily contains records from over 100,000 stations in 180 countries and territories.
• The data are obtained from more than 20 sources. Some data are more than 175 years old.
• NCEI provides numerous daily variables, including maximum and minimum temperature, total daily precipitation, snowfall, and snow depth; however, about one half of the stations report precipitation only

Data description

https://www.ncdc.noaa.gov/ghcn-daily-description

Collection

The data can be collected from S3 buckets. Here I collected it for 2019.
For detail information the link is as below:
https://docs.opendata.aws/noaa-ghcn-pds/readme.html
Question for data quality should be addressed at noaa.bdp@noaa.gov.

Summary of Date format

ID = 11 character station identification code. Please see ghcnd-stations section below for an explantation
YEAR/MONTH/DAY = 8 character date in YYYYMMDD format (e.g. 19860529 = May 29, 1986)
ELEMENT = 4 character indicator of element type
DATA VALUE = 5 character data value for ELEMENT
M-FLAG = 1 character Measurement Flag
Q-FLAG = 1 character Quality Flag
S-FLAG = 1 character Source Flag
OBS-TIME = 4-character time of observation in hour-minute format (i.e. 0700 =7:00 am)

The fields are comma delimited and each row represents one station-day.

Variable and feature details

These variables have the following definitions:

• ID = the station identification code.
  The first two characters denote the FIPS country code. Details for FIPS country code https://www.geodatasource.com/resources/tutorials/international-country-code-fips-versus-iso-3166/
  The third character is a network code that identifies the station numbering system used
  0 = unspecified (station identified by up to eight alphanumeric characters)
  1 = Community Collaborative Rain, Hail,and Snow (CoCoRaHS) based identification number. To ensure consistency with with GHCN Daily, all numbers in the original CoCoRaHS IDs have been left-filled to make them all four digits long. In addition, the characters “-” and “_” have been removed to ensure that the IDs do not exceed 11 characters when preceded by “US1”. For example, the CoCoRaHS ID “AZ-MR-156” becomes “US1AZMR0156” in GHCN-Daily
• LATITUDE = latitude of the station (in decimal degrees).
• LONGITUDE = longitude of the station (in decimal degrees).
• STATE = U.S. postal code for the state (for U.S. and Canadian stations only).
• NAME = name of the station.
• GSN FLAG = flag that indicates whether the station is part of the GCOS Surface Network (GSN).
• HCN/CRN FLAG = flag that indicates whether the station is part of the U.S. Historical Climatology Network (HCN). T
• WMO ID is the World Meteorological Organization (WMO) number for the station. If the station has no WMO number (or one has not
  yet been matched to this station), then the field is blank.

GHCND inventory master data

This is the periods of record for each station and element

Data structure

• ID = the station identification code. Please see “ghcnd-stations.txt” for a complete list of stations and their metadata.
• LATITUDE = the latitude of the station (in decimal degrees).
• LONGITUDE = the longitude of the station (in decimal degrees).
• ELEMENT = the element type. See section III for a definition of elements.
• FIRSTYEAR = the first year of unflagged data for the given element.
• LASTYEAR = the last year of unflagged data for the given element.

Acknowledgements

Referenced from AWS open source data storage in S3 and NOAA data domain.

Inspiration

NOAA weather stations, weather transaction data

The Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) Level 1 Precision Terrain Corrected Registered At-Sensor Radiance (AST_L1T) data contains calibrated at-sensor radiance, which corresponds with the ASTER Level 1B (AST_L1B), that has been geometrically corrected, and rotated to a north-up UTM projection. The AST_L1T is created from a single resampling of the corresponding ASTER L1A (AST_L1A) product.The precision terrain correction process incorporates GLS2000 digital elevation data with derived ground control points (GCPs) to achieve topographic accuracy for all daytime scenes where correlation statistics reach a minimum threshold. Alternate levels of correction are possible (systematic terrain, systematic, or precision) for scenes acquired at night or that otherwise represent a reduced quality ground image (e.g., cloud cover).Each AST_L1T granule is converted into three different COG files based on the sensor and spatial resolution, VNIR at 15m, SWIR at 30m and TIR at 90m. The metadata required to transform the digital numbers (DN) to radiance and reflectance values are also incorporated as metadata in the TIFF files. The filenaming convention and the organization of bands are described here.

Note that you can download quickly via CLI. (Kaggle Environment: 1min 36s, Colab: 1min) ! kaggle datasets download williamhyun/amazon-bin-image-dataset-536434-images-224x224

Amazon Bin Image Dataset

The Amazon Bin Image Dataset contains 536,434 images and metadata from bins of a pod in an operating Amazon Fulfillment Center. The bin images in this dataset are captured as robot units carry pods as part of normal Amazon Fulfillment Center operations. This dataset has many images and the corresponding medadata.

The image files have three groups according to its naming scheme.

• A file name with 1~4 digits (1,200): 1.jpg ~ 1200.jpg
• A file name with 5 digits (99,999): 00001.jpg ~ 99999.jpg
• A file name with 6 digits (435,235): 100000.jpg ~ 535234.jpg

Amazon Bin Image Dataset (536,434 images, 224x224) dataset aims to provide a resized image files and a full metadata SQLite file for Kaggle Kernel environments. You can download a single 4GB archive file via Download button on this page.

Documentation

• https://github.com/awslabs/open-data-docs/tree/main/docs/aft-vbi-pds

Download

• https://registry.opendata.aws/amazon-bin-imagery/
• https://github.com/awslabs/open-data-registry/blob/main/datasets/amazon-bin-imagery.yaml

The purpose of this resource is to demonstrate how the CUAHSI JupyterHub platform can be used to perform basic hydrologic data analysis. Temperature data was collected from the NOAA Global Historical Climatology network for two sites in the greater Seattle area. These data are organized using Python classes, and plotted in various ways to demonstrate common data analysis steps.

For more information about the GHCN data included in this resource, see; https://docs.opendata.aws/noaa-ghcn-pds/readme.html

Useful information and links for navigating this site, understanding and utilizing Open Data

🇺🇸 미국

Metadata form template for Tempe Open Data.

Documentation describing the Race and Identity-Based Data Collection Strategy data tables released as open data, including table descriptions, metadata, and glossary of terms.

Informal documentation for usage of the Groundwater Contamination Susceptibility Model (GCSM)