The bulk download facility provides the entire contents of each major API data set in a single ZIP file. A small JSON formatted manifest file lists the bulk files and the update date of each file. The manifest is generally updated daily and can be downloaded from http://api.eia.gov/bulk/manifest.txt. The manifest contains information about the bulk files, including all required common core attributes.

TRAINING DATASET: Hands-On Uploading Data (Download This File)

This dataset contains resources transformed from other datasets on HDX. They exist here only in a format modified to support visualization on HDX and may not be as up to date as the source datasets from which they are derived.

Source datasets: https://data.hdx.rwlabs.org/dataset/idps-data-by-region-in-mali

Download Service provides pre-defined data on relationship between selected territorial elements and units of territorial registration using the ATOM technology. The service is publicly available and free-of-charge (data covers the whole territory of the Czech Republic) and enables downloading of predefined data file containing data for the whole Czech Republic. Files are created during the first day of each month with data valid to the last day of previous month. The whole dataset (7 files) is compressed (ZIP) for downloading.

General information: The data sets contain information on how often materials of studies available through GESIS: Data Archive for the Social Sciences were downloaded and/or ordered through one of the archive´s plattforms/services between 2004 and 2017.

Sources and plattforms: Study materials are accessible through various GESIS plattforms and services: Data Catalogue (DBK), histat, datorium, data service (and others).

Years available: - Data Catalogue: 2012-2017 - data service: 2006-2017 - datorium: 2014-2017 - histat: 2004-2017

Data sets: Data set ZA6899_Datasets_only_all_sources contains information on how often data files such as those with dta- (Stata) or sav- (SPSS) extension have been downloaded. Identification of data files is handled semi-automatically (depending on the plattform/serice). Multiple downloads of one file by the same user (identified through IP-address or username for registered users) on the same days are only counted as one download.

Data set ZA6899_Doc_and_Data_all_sources contains information on how often study materials have been downloaded. Multiple downloads of any file of the same study by the same user (identified through IP-address or username for registered users) on the same days are only counted as one download.

Both data sets are available in three formats: csv (quoted, semicolon-separated), dta (Stata v13, labeled) and sav (SPSS, labeled). All formats contain identical information.

Variables: Variables/columns in both data sets are identical. za_nr ´Archive study number´ version ´GESIS Archiv Version´ doi ´Digital Object Identifier´ StudyNo ´Study number of respective study´ Title ´English study title´ Title_DE ´German study title´ Access ´Access category (0, A, B, C, D, E)´ PubYear ´Publication year of last version of the study´ inZACAT ´Study is currently also available via ZACAT´ inHISTAT ´Study is currently also available via HISTAT´ inDownloads ´There are currently data files available for download for this study in DBK or datorium´ Total ´All downloads combined´ downloads_2004 ´downloads/orders from all sources combined in 2004´ [up to ...] downloads_2017 ´downloads/orders from all sources combined in 2017´ d_2004_dbk ´downloads from source dbk in 2004´ [up to ...] d_2017_dbk ´downloads from source dbk in 2017´ d_2004_histat ´downloads from source histat in 2004´ [up to ...] d_2017_histat ´downloads from source histat in 2017´ d_2004_dataservice ´downloads/orders from source dataservice in 2004´ [up to ...] d_2017_dataservice ´downloads/orders from source dataservice in 2017´

More information is available within the codebook.

Automatically describing images using natural sentences is an essential task to visually impaired people's inclusion on the Internet. Although there are many datasets in the literature, most of them contain only English captions, whereas datasets with captions described in other languages are scarce.

PraCegoVer arose on the Internet, stimulating users from social media to publish images, tag #PraCegoVer and add a short description of their content. Inspired by this movement, we have proposed the #PraCegoVer, a multi-modal dataset with Portuguese captions based on posts from Instagram. It is the first large dataset for image captioning in Portuguese with freely annotated images.

PraCegoVer has 533,523 pairs with images and captions described in Portuguese collected from more than 14 thousand different profiles. Also, the average caption length in #PraCegoVer is 39.3 words and the standard deviation is 29.7.

Dataset Structure

PraCegoVer dataset is composed of the main file dataset.json and a collection of compressed files named images.tar.gz.partX

containing the images. The file dataset.json comprehends a list of json objects with the attributes:

user: anonymized user that made the post;

filename: image file name;

raw_caption: raw caption;

caption: clean caption;

date: post date.

Each instance in dataset.json is associated with exactly one image in the images directory whose filename is pointed by the attribute filename. Also, we provide a sample with five instances, so the users can download the sample to get an overview of the dataset before downloading it completely.

Download Instructions

If you just want to have an overview of the dataset structure, you can download sample.tar.gz. But, if you want to use the dataset, or any of its subsets (63k and 173k), you must download all the files and run the following commands to uncompress and join the files:

cat images.tar.gz.part* > images.tar.gz tar -xzvf images.tar.gz

Alternatively, you can download the entire dataset from the terminal using the python script download_dataset.py available in PraCegoVer repository. In this case, first, you have to download the script and create an access token here. Then, you can run the following command to download and uncompress the image files:

python download_dataset.py --access_token=

US County & Zipcode Historical Demographics

Easily lookup US historical demographics by county FIPS or zipcode in seconds with this file containing over 5,901 different columns including:

*Lat/Long *Boundaries *State FIPS *Population from 2010-2019 *Death Rate from 2010-2019 *Unemployment from 2001-2020 *Education from 1970-2019 *Gender and Age Population

Provided by bitrook.com to help Data Scientists clean data faster.

Data Sources

All Data Combined Source:

https://www.ers.usda.gov/data-products/county-level-data-sets/download-data/

Population Source:

https://www.ers.usda.gov/data-products/county-level-data-sets/download-data/

Unemployment Source:

https://www.ers.usda.gov/data-products/county-level-data-sets/download-data/

Zip FIPS Crosswalk Source:

https://data.world/niccolley/us-zipcode-to-county-state

County Boundaries Source:

https://public.opendatasoft.com/explore/dataset/us-county-boundaries/table/?disjunctive.statefp&disjunctive.countyfp&disjunctive.name&disjunctive.namelsad&disjunctive.stusab&disjunctive.state_name

Age Sex Source:

https://www2.census.gov/programs-surveys/popest/datasets/2010-2019/counties/asrh/cc-est2019-agesex-**.csv https://www2.census.gov/programs-surveys/popest/technical-documentation/file-layouts/2010-2019/cc-est2019-agesex.pdf

Races Source:

https://www2.census.gov/programs-surveys/popest/datasets/2010-2019/counties/asrh/cc-est2019-alldata.csv https://www2.census.gov/programs-surveys/popest/technical-documentation/file-layouts/2010-2019/cc-est2019-alldata.pdf

This is an auto-generated index table corresponding to a folder of files in this dataset with the same name. This table can be used to extract a subset of files based on their metadata, which can then be used for further analysis. You can view the contents of specific files by navigating to the "cells" tab and clicking on an individual file_kd.

This collection encompasses all bulk data downloads available on EIA's open data site on 5/14/2025. The manifest.txt files provides descriptions of the included datasets in a JSON format. The datasets are divided by subject.Survey forms used to collect the data are available here: https://www.eia.gov/survey/File name | SubjectAEO2025.zip | Annual Energy Outlook 2025SEDS.zip | State Energy Data SystemsELEC.zip | ElectricityNG.zip | Natural GasPET.zip | PetroleumTOTAL.zip | Total EnergyCOAL.zip | CoalSTEO.zip | Short Term Energy OutlookPET_IMPORTS.zip | Crude Oil ImportsINTL.zip | International Energy DataEBA.zip | US Electric System Operating Data (2019-present)EBA-pre2019.zip | US Electric System Operating Data (before 2019)EMISS.zip | CO2 EmissionsIEO.zip | International Energy OutlookNUC_STATUS.zip | U.S. Nuclear Outages

Download the complete MAC Address JSON database to integrate network data into your projects. Regularly updated and easy to use.

Summary

Geojson files used to visualize geospatial layers relevant to identifying and assessing trucking fleet decarbonization opportunities with the MIT Climate & Sustainability Consortium's Geospatial Trucking Industry Decarbonization Explorer (Geo-TIDE) tool.

Relevant Links

Link to the online version of the tool (requires creation of a free user account).

Link to GitHub repo with source code to produce this dataset and deploy the Geo-TIDE tool locally.

Funding

This dataset was produced with support from the MIT Climate & Sustainability Consortium.

Original Data Sources

These geojson files draw from and synthesize a number of different datasets and tools. The original data sources and tools are described below:

Filename(s) Description of Original Data Source(s) Link(s) to Download Original Data License and Attribution for Original Data Source(s)

faf5_freight_flows/*.geojson

trucking_energy_demand.geojson

highway_assignment_links_*.geojson

infrastructure_pooling_thought_experiment/*.geojson

Regional and highway-level freight flow data obtained from the Freight Analysis Framework Version 5. Shapefiles for FAF5 region boundaries and highway links are obtained from the National Transportation Atlas Database. Emissions attributes are evaluated by incorporating data from the 2002 Vehicle Inventory and Use Survey and the GREET lifecycle emissions tool maintained by Argonne National Lab.

Shapefile for FAF5 Regions

Shapefile for FAF5 Highway Network Links

FAF5 2022 Origin-Destination Freight Flow database

FAF5 2022 Highway Assignment Results

Attribution for Shapefiles: United States Department of Transportation Bureau of Transportation Statistics National Transportation Atlas Database (NTAD). Available at: https://geodata.bts.gov/search?collection=Dataset.

License for Shapefiles: This NTAD dataset is a work of the United States government as defined in 17 U.S.C. § 101 and as such are not protected by any U.S. copyrights. This work is available for unrestricted public use.

Attribution for Origin-Destination Freight Flow database: National Transportation Research Center in the Oak Ridge National Laboratory with funding from the Bureau of Transportation Statistics and the Federal Highway Administration. Freight Analysis Framework Version 5: Origin-Destination Data. Available from: https://faf.ornl.gov/faf5/Default.aspx. Obtained on Aug 5, 2024. In the public domain.

Attribution for the 2022 Vehicle Inventory and Use Survey Data: United States Department of Transportation Bureau of Transportation Statistics. Vehicle Inventory and Use Survey (VIUS) 2002 [supporting datasets]. 2024. https://doi.org/10.21949/1506070

Attribution for the GREET tool (original publication): Argonne National Laboratory Energy Systems Division Center for Transportation Research. GREET Life-cycle Model. 2014. Available from this link.

Attribution for the GREET tool (2022 updates): Wang, Michael, et al. Summary of Expansions and Updates in GREET® 2022. United States. https://doi.org/10.2172/1891644

grid_emission_intensity/*.geojson

Emission intensity data is obtained from the eGRID database maintained by the United States Environmental Protection Agency.

eGRID subregion boundaries are obtained as a shapefile from the eGRID Mapping Files database.

eGRID database

Shapefile with eGRID subregion boundaries

Attribution for eGRID data: United States Environmental Protection Agency: eGRID with 2022 data. Available from https://www.epa.gov/egrid/download-data. In the public domain.

Attribution for shapefile: United States Environmental Protection Agency: eGRID Mapping Files. Available from https://www.epa.gov/egrid/egrid-mapping-files. In the public domain.

US_elec.geojson

US_hy.geojson

US_lng.geojson

US_cng.geojson

US_lpg.geojson

Locations of direct current fast chargers and refueling stations for alternative fuels along U.S. highways. Obtained directly from the Station Data for Alternative Fuel Corridors in the Alternative Fuels Data Center maintained by the United States Department of Energy Office of Energy Efficiency and Renewable Energy.

US_elec.geojson

US_hy.geojson

US_lng.geojson

US_cng.geojson

US_lpg.geojson

Attribution: U.S. Department of Energy, Energy Efficiency and Renewable Energy. Alternative Fueling Station Corridors. 2024. Available from: https://afdc.energy.gov/corridors. In the public domain.

These data and software code ("Data") are provided by the National Renewable Energy Laboratory ("NREL"), which is operated by the Alliance for Sustainable Energy, LLC ("Alliance"), for the U.S. Department of Energy ("DOE"), and may be used for any purpose whatsoever.

daily_grid_emission_profiles/*.geojson

Hourly emission intensity data obtained from ElectricityMaps.

Original data can be downloaded as csv files from the ElectricityMaps United States of America database

Shapefile with region boundaries used by ElectricityMaps

License: Open Database License (ODbL). Details here: https://www.electricitymaps.com/data-portal

Attribution for csv files: Electricity Maps (2024). United States of America 2022-23 Hourly Carbon Intensity Data (Version January 17, 2024). Electricity Maps Data Portal. https://www.electricitymaps.com/data-portal.

Attribution for shapefile with region boundaries: ElectricityMaps contributors (2024). electricitymaps-contrib (Version v1.155.0) [Computer software]. https://github.com/electricitymaps/electricitymaps-contrib.

gen_cap_2022_state_merged.geojson

trucking_energy_demand.geojson

Grid electricity generation and net summer power capacity data is obtained from the state-level electricity database maintained by the United States Energy Information Administration.

U.S. state boundaries obtained from this United States Department of the Interior U.S. Geological Survey ScienceBase-Catalog.

Annual electricity generation by state

Net summer capacity by state

Shapefile with U.S. state boundaries

Attribution for electricity generation and capacity data: U.S. Energy Information Administration (Aug 2024). Available from: https://www.eia.gov/electricity/data/state/. In the public domain.

electricity_rates_by_state_merged.geojson

Commercial electricity prices are obtained from the Electricity database maintained by the United States Energy Information Administration.

Electricity rate by state

Attribution: U.S. Energy Information Administration (Aug 2024). Available from: https://www.eia.gov/electricity/data.php. In the public domain.

demand_charges_merged.geojson

demand_charges_by_state.geojson

Maximum historical demand charges for each state and zip code are derived from a dataset compiled by the National Renewable Energy Laboratory in this this Data Catalog.

Historical demand charge dataset

The original dataset is compiled by the National Renewable Energy Laboratory (NREL), the U.S. Department of Energy (DOE), and the Alliance for Sustainable Energy, LLC ('Alliance').

Attribution: McLaren, Joyce, Pieter Gagnon, Daniel Zimny-Schmitt, Michael DeMinco, and Eric Wilson. 2017. 'Maximum demand charge rates for commercial and industrial electricity tariffs in the United States.' NREL Data Catalog. Golden, CO: National Renewable Energy Laboratory. Last updated: July 24, 2024. DOI: 10.7799/1392982.

eastcoast.geojson

midwest.geojson

la_i710.geojson

h2la.geojson

bayarea.geojson

saltlake.geojson

northeast.geojson

Highway corridors and regions targeted for heavy duty vehicle infrastructure projects are derived from a public announcement on February 15, 2023 by the United States Department of Energy.

The shapefile with Bay area boundaries is obtained from this Berkeley Library dataset.

The shapefile with Utah county boundaries is obtained from this dataset from the Utah Geospatial Resource Center.

Shapefile for Bay Area country boundaries

Shapefile for counties in Utah

Attribution for public announcement: United States Department of Energy. Biden-Harris Administration Announces Funding for Zero-Emission Medium- and Heavy-Duty Vehicle Corridors, Expansion of EV Charging in Underserved Communities (2023). Available from https://www.energy.gov/articles/biden-harris-administration-announces-funding-zero-emission-medium-and-heavy-duty-vehicle.

Attribution for Bay area boundaries: San Francisco (Calif.). Department Of Telecommunications and Information Services. Bay Area Counties. 2006. In the public domain.

Attribution for Utah boundaries: Utah Geospatial Resource Center & Lieutenant Governor's Office. Utah County Boundaries (2023). Available from https://gis.utah.gov/products/sgid/boundaries/county/.

License for Utah boundaries: Creative Commons 4.0 International License.

incentives_and_regulations/*.geojson

State-level incentives and regulations targeting heavy duty vehicles are collected from the State Laws and Incentives database maintained by the United States Department of Energy's Alternative Fuels Data Center.

Data was collected manually from the State Laws and Incentives database.

Attribution: U.S. Department of Energy, Energy Efficiency and Renewable Energy, Alternative Fuels Data Center. State Laws and Incentives. Accessed on Aug 5, 2024 from: https://afdc.energy.gov/laws/state. In the public domain.

These data and software code ("Data") are provided by the National Renewable Energy Laboratory ("NREL"), which is operated by the Alliance for Sustainable Energy, LLC ("Alliance"), for the U.S. Department of Energy ("DOE"), and may be used for any purpose whatsoever.

costs_and_emissions/*.geojson

diesel_price_by_state.geojson

trucking_energy_demand.geojson

Lifecycle costs and emissions of electric and diesel trucking are evaluated by adapting the model developed by Moreno Sader et al., and calibrated to the Run on Less dataset for the Tesla Semi collected from the 2023 PepsiCo Semi pilot by the North American Council for Freight Efficiency.

In

This dataset contains the metadata of the datasets published in 85 Dataverse installations and information about each installation's metadata blocks. It also includes the lists of pre-defined licenses or terms of use that dataset depositors can apply to the datasets they publish in the 58 installations that were running versions of the Dataverse software that include that feature. The data is useful for reporting on the quality of dataset and file-level metadata within and across Dataverse installations and improving understandings about how certain Dataverse features and metadata fields are used. Curators and other researchers can use this dataset to explore how well Dataverse software and the repositories using the software help depositors describe data. How the metadata was downloaded The dataset metadata and metadata block JSON files were downloaded from each installation between August 22 and August 28, 2023 using a Python script kept in a GitHub repo at https://github.com/jggautier/dataverse-scripts/blob/main/other_scripts/get_dataset_metadata_of_all_installations.py. In order to get the metadata from installations that require an installation account API token to use certain Dataverse software APIs, I created a CSV file with two columns: one column named "hostname" listing each installation URL in which I was able to create an account and another column named "apikey" listing my accounts' API tokens. The Python script expects the CSV file and the listed API tokens to get metadata and other information from installations that require API tokens. How the files are organized ├── csv_files_with_metadata_from_most_known_dataverse_installations │ ├── author(citation)_2023.08.22-2023.08.28.csv │ ├── contributor(citation)_2023.08.22-2023.08.28.csv │ ├── data_source(citation)_2023.08.22-2023.08.28.csv │ ├── ... │ └── topic_classification(citation)_2023.08.22-2023.08.28.csv ├── dataverse_json_metadata_from_each_known_dataverse_installation │ ├── Abacus_2023.08.27_12.59.59.zip │ ├── dataset_pids_Abacus_2023.08.27_12.59.59.csv │ ├── Dataverse_JSON_metadata_2023.08.27_12.59.59 │ ├── hdl_11272.1_AB2_0AQZNT_v1.0(latest_version).json │ ├── ... │ ├── metadatablocks_v5.6 │ ├── astrophysics_v5.6.json │ ├── biomedical_v5.6.json │ ├── citation_v5.6.json │ ├── ... │ ├── socialscience_v5.6.json │ ├── ACSS_Dataverse_2023.08.26_22.14.04.zip │ ├── ADA_Dataverse_2023.08.27_13.16.20.zip │ ├── Arca_Dados_2023.08.27_13.34.09.zip │ ├── ... │ └── World_Agroforestry_-_Research_Data_Repository_2023.08.27_19.24.15.zip └── dataverse_installations_summary_2023.08.28.csv └── dataset_pids_from_most_known_dataverse_installations_2023.08.csv └── license_options_for_each_dataverse_installation_2023.09.05.csv └── metadatablocks_from_most_known_dataverse_installations_2023.09.05.csv This dataset contains two directories and four CSV files not in a directory. One directory, "csv_files_with_metadata_from_most_known_dataverse_installations", contains 20 CSV files that list the values of many of the metadata fields in the citation metadata block and geospatial metadata block of datasets in the 85 Dataverse installations. For example, author(citation)_2023.08.22-2023.08.28.csv contains the "Author" metadata for the latest versions of all published, non-deaccessioned datasets in the 85 installations, where there's a row for author names, affiliations, identifier types and identifiers. The other directory, "dataverse_json_metadata_from_each_known_dataverse_installation", contains 85 zipped files, one for each of the 85 Dataverse installations whose dataset metadata I was able to download. Each zip file contains a CSV file and two sub-directories: The CSV file contains the persistent IDs and URLs of each published dataset in the Dataverse installation as well as a column to indicate if the Python script was able to download the Dataverse JSON metadata for each dataset. It also includes the alias/identifier and category of the Dataverse collection that the dataset is in. One sub-directory contains a JSON file for each of the installation's published, non-deaccessioned dataset versions. The JSON files contain the metadata in the "Dataverse JSON" metadata schema. The Dataverse JSON export of the latest version of each dataset includes "(latest_version)" in the file name. This should help those who are interested in the metadata of only the latest version of each dataset. The other sub-directory contains information about the metadata models (the "metadata blocks" in JSON files) that the installation was using when the dataset metadata was downloaded. I included them so that they can be used when extracting metadata from the dataset's Dataverse JSON exports. The dataverse_installations_summary_2023.08.28.csv file contains information about each installation, including its name, URL, Dataverse software version, and counts of dataset metadata...

The Facility Registry System (FRS) identifies facilities, sites, or places subject to environmental regulation or of environmental interest to EPA programs or delegated states. Using vigorous verification and data management procedures, FRS integrates facility data from program national systems, state master facility records, tribal partners, and other federal agencies and provides the Agency with a centrally managed, single source of comprehensive and authoritative information on facilities.

This dataset contains >800K CSV files behind the GitTables 1M corpus.

For more information about the GitTables corpus, visit:

- our website for GitTables, or

- the main GitTables download page on Zenodo.

TRAINING DATASET: Hands-On Formatting Data Part 1 (Download This File)

The benchmarking datasets used for deepBlink. The npz files contain train/valid/test splits inside and can be used directly. The files belong to the following challenges / classes:- ISBI Particle tracking challenge: microtubule, vesicle, receptor- Custom synthetic (based on http://smal.ws): particle- Custom fixed cell: smfish- Custom live cell: suntagThe csv files are to determine which image in the test splits correspond to which original image, SNR, and density.

A complete list of live websites using the Better File Download technology, compiled through global website indexing conducted by WebTechSurvey.

This is an auto-generated index table corresponding to a folder of files in this dataset with the same name. This table can be used to extract a subset of files based on their metadata, which can then be used for further analysis. You can view the contents of specific files by navigating to the "cells" tab and clicking on an individual file_id.

Note that this Crossref metadata is always openly available. The difference here is that we’ve done the time-saving work of putting all of the records registered through April 2024 into one file for download. To keep this metadata current, you can access new records via our public API at: And, if you do use our API, we encourage you to read the section of the documentation on "etiquette". That is, how to use the API without making it impossible for others to use.