Context

NYC Open Data is an opportunity to engage New Yorkers in the information that is produced and used by City government. We believe that every New Yorker can benefit from Open Data, and Open Data can benefit from every New Yorker. Source: https://opendata.cityofnewyork.us/overview/

Content

Thanks to NYC Open Data, which makes public data generated by city agencies available for public use, and Citi Bike, we've incorporated over 150 GB of data in 5 open datasets into Google BigQuery Public Datasets, including:

• Over 8 million 311 service requests from 2012-2016

• More than 1 million motor vehicle collisions 2012-present

• Citi Bike stations and 30 million Citi Bike trips 2013-present

• Over 1 billion Yellow and Green Taxi rides from 2009-present

• Over 500,000 sidewalk trees surveyed decennially in 1995, 2005, and 2015

This dataset is deprecated and not being updated.

Fork this kernel to get started with this dataset.

Acknowledgements

https://opendata.cityofnewyork.us/

https://cloud.google.com/blog/big-data/2017/01/new-york-city-public-datasets-now-available-on-google-bigquery

This dataset is publicly available for anyone to use under the following terms provided by the Dataset Source - https://data.cityofnewyork.us/ - and is provided "AS IS" without any warranty, express or implied, from Google. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

By accessing datasets and feeds available through NYC Open Data, the user agrees to all of the Terms of Use of NYC.gov as well as the Privacy Policy for NYC.gov. The user also agrees to any additional terms of use defined by the agencies, bureaus, and offices providing data. Public data sets made available on NYC Open Data are provided for informational purposes. The City does not warranty the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set made available on NYC Open Data, nor are any such warranties to be implied or inferred with respect to the public data sets furnished therein.

The City is not liable for any deficiencies in the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set, or application utilizing such data set, provided by any third party.

Banner Photo by @bicadmedia from Unplash.

Inspiration

On which New York City streets are you most likely to find a loud party?

Can you find the Virginia Pines in New York City?

Where was the only collision caused by an animal that injured a cyclist?

What’s the Citi Bike record for the Longest Distance in the Shortest Time (on a route with at least 100 rides)?

https://cloud.google.com/blog/big-data/2017/01/images/148467900588042/nyc-dataset-6.png" alt="enter image description here"> https://cloud.google.com/blog/big-data/2017/01/images/148467900588042/nyc-dataset-6.png

A list of all datasets that were identified for publication on NYC Open Data and their current release status. For comprehensive information on each dataset currently on NYC Open Data, please refer to Local Law 251 of 2017: Published Data Asset Inventory.

NOTE: To review the latest plan, make sure to filter the "Report Year" column to the latest year. This dataset lists the plan entries that were removed from publishing.

This dataset contains the list of dataset nominations submitted to the NYC Open Data team.

NOTE: To review the latest plan, make sure to filter the "Report Year" column to the latest year. This dataset includes various FOIL reporting metrics as part of the NYC Open Data Compliance Plan.

Please note that this dataset is currently undergoing structural changes and will receive an update in December 2024.

This is a list of responses to public inquiries to the NYC Open Data Team via the “Contact Us” page on www.nyc.gov/opendata"> www.nyc.gov/opendata that get triaged as being “Data Questions” and “Data Errors.”

NOTE: To review the latest plan, make sure to filter the "Report Year" column to the latest year. Data on public websites maintained by or on behalf of the city agencies.

All 311 Service Requests from 2010 to present. This information is automatically updated daily.

Analysis of ‘2018 NYC Open Data Plan: FOIL Datasets’ provided by Analyst-2 (analyst-2.ai), based on source dataset retrieved from https://catalog.data.gov/dataset/2745ca6a-eab5-42b8-91f6-5eeebc86a74b on 26 January 2022.

--- Dataset description provided by original source is as follows ---

Local Law 7 of 2016 requires agencies to “review responses to freedom of information law [FOIL] requests that include the release of data to determine if such responses consist of or include public data sets that have not yet been included on the single web portal or the inclusion” on the Open Data Portal. Additionally, each City agency shall disclose “the total number, since the last update, of such agency’s freedom of information law responses that included the release of data, the total number of such responses determined to consist of or include a public data set that had not yet been included on the single web portal and the name of such public data set, where applicable, and the total number of such responses that resulted in voluntarily disclosed information being made accessible through the single web portal.”

See the agency summary statistics on data released in responses to FOIL requests here: https://data.cityofnewyork.us/City-Government/2018- Open-Data-Plan-FOIL-Report/cvse-perd

See the 2018 Open Data for All Report and Open Data Plan here: https://opendata.cityofnewyork.us/wp-content/uploads/2018/09/2018-NYC-OD4A-report.pdf

--- Original source retains full ownership of the source dataset ---

NOTE: To review the latest plan, make sure to filter the "Report Year" column to the latest year.

This dataset lists all datasets reported by agencies in their Annual Compliance plans for release on NYC Open Data. For the latest status, please refer to the NYC Open Data Release Tracker.

Analysis of ‘NYC Open Data Plan - Datasets Added Since The Publication NYC Open Data Plan ( September 2013)’ provided by Analyst-2 (analyst-2.ai), based on source dataset retrieved from https://catalog.data.gov/dataset/7bc0ef27-6a1c-4adc-94f7-b1d6e251864c on 26 January 2022.

--- Dataset description provided by original source is as follows ---

This dataset will provide the list of datasets added to the portal since the publication of the Inaugural NYC Open Data Plan ( September 2013)

--- Original source retains full ownership of the source dataset ---

NOTE: To review the latest plan, make sure to filter the "Report Year" column to the latest year. This is a list of civic engagement activities reported by city agencies for the purpose of providing context, spreading awareness, and increasing the use of public data. Details include activity descriptions, date of completion and links to further information on the reported activity. For the historical 2018-2022 data please refer to https://data.cityofnewyork.us/d/keb3-v9xt/

The following layer shows hotspot areas as delineated by NY State government. The layer shows red, orange, and yellow zones and provides activity guidance via attributes.

Community Districts are mandated by the city charter to review and monitor quality of life issues for New York City neighborhoods.This dataset is sourced from the DCP_GIS Open Data Group:Original Source: https://www.arcgis.com/home/item.html?id=436c404eb5434240841488827c9167b8

All 311 Service Requests from 2010 to present. This information is automatically updated daily.

Denied registrants in the wholesale markets

This inventory includes all items removed from the Open Data Plan. The reason for removal is documented for each item removed.

Feature layer generated from running the Enrich layer solution. NYC_City_Council_Districts were enriched

NYS Building Footprints - metadata info:The New York State building footprints service contains building footprints with address information. The footprints have address point information folded in from the Streets and Address Matching (SAM - https://gis.ny.gov/streets/) address point file. The building footprints have a field called “Address Range”, this field shows (where available) either a single address or an address range, depending on the address points that fall within the footprint. Ex: 3860 Atlantic Avenue or Ex: 32 - 34 Wheatfield Circle Building footprints in New York State are from four different sources: Microsoft, Open Data, New York State Energy Research and Development Authority (NYSERDA), and Geospatial Services. The majority of the footprints are from NYSERDA, except in NYC where the primary source was Open Data. Microsoft footprints were added where the other 2 sources were missing polygons. Field Descriptions: NYSGeo Source : tells the end user if the source is NYSERDA, Microsoft, NYC Open Data, and could expand from here in the futureAddress Point Count: the number of address points that fall within that building footprintAddress Range : If an address point falls within a footprint it lists the range of those address points. Ex: if a building is on a corner of South Pearl and Beaver Street, 40 points fall on the building, and 35 are South Pearl Street it would give the range of addresses for South Pearl. We also removed sub addresses from this range, primarily apartment related. For example, in above example, it would not list 30 South Pearl, Apartment 5A, it would list 30 South Pearl.Most Common Street : the street name of the largest number of address points. In the above example, it would list “South Pearl” as the most common street since the majority of address points list it as the street. Other Streets: the list of other streets that fall within the building footprint, if any. In the above example, “Beaver Street” would be listed since address points for Beaver Street fall on the footprint but are not in the majority.County Name : County name populated from CIESINs. If not populated from CIESINs, identified by the GSMunicipality Name : Municipality name populated from CIESINs. If not populated from CIESINs, identified by the GSSource: Source where the data came from. If NYSGeo Source = NYSERDA, the data would typically list orthoimagery, LIDAR, county data, etc.Source ID: if NYSGeo Source = NYSERDA, Source ID would typically list an orthoimage or LIDAR tileSource Date: Date the footprint was created. If the source image was from 2016 orthoimagery, 2016 would be the Source Date. Description of each footprint source:NYSERDA Building footprints that were created as part of the New York State Flood Impact Decision Support Systems https://fidss.ciesin.columbia.edu/home Footprints vary in age from county to county.Microsoft Building Footprints released 6/28/2018 - vintage unknown/varies. More info on this dataset can be found at https://blogs.bing.com/maps/2018-06/microsoft-releases-125-million-building-footprints-in-the-us-as-open-data.NYC Open Data - Building Footprints of New York City as a polygon feature class. Last updated 7/30/2018, downloaded on 8/6/2018. Feature Class of footprint outlines of buildings in New York City. Please see the following link for additional documentation- https://github.com/CityOfNewYork/nyc-geo-metadata/blob/master/Metadata/Metadata_BuildingFootprints.mdSpatial Reference of Source Data: UTM Zone 18, meters, NAD 83. Spatial Reference of Web Service: Spatial Reference of Web Service: WGS 1984 Web Mercator Auxiliary Sphere.

A brief history of water consumption in the New York City Water Supply System (Based on New York City Census population)