Dataset

This dataset was created by Vishnu Murali

Contents

This dataset contains the file of vehicle, snowmobile and boat registrations in NYS. Registrations expired more than 2 years are excluded. Records that have a scofflaw, revocation and/or suspension are included with indicators specifying those kinds of records.

You have access to two datasets: one exclusively containing car ratings, and the other containing detailed car features. These datasets provide an opportunity to work with real data, enabling you to practice various data analytics techniques such as data visualization, regression analysis for predicting prices, and classification tasks such as brand classification.

This is the EVENT data captured from the New York City CV Pilot project that was processed by the independent evaluators at Volpe. Additional data collected and data dictionary are in the attachments.

Each EVENT record documents the details of one application warning that occurred on an Aftermarket Safety Device (ASD) in an equipped host vehicle and includes CV messages from a defined recording time both before and after the warning was generated by the host ASD. Messages in the recording time window include the Basic Safety Messages (BSM) of the host vehicle that received the warning, as well as other BSMs received from the warning target equipped vehicle (for V2V applications) or other nearby equipped vehicles. Depending on the application warning type, MAP messages, Signal Phase and Timing (SPaT) messages, and Traveler Information Messages (TIM) that were heard by the host vehicle may also be included in the event record.

In Newyork City, all taxi vehicles are managed by TLC (Taxi and Limousine Commission), here is a brief description about TLC:

The New York City Taxi and Limousine Commission (TLC), created in 1971, is the agency responsible for licensing and regulating New York City's Medallion (Yellow) taxi cabs, for-hire vehicles (community-based liveries, black cars and luxury limousines), commuter vans, and paratransit vehicles. The Commission's Board consists of nine members, eight of whom are unsalaried Commissioners. The salaried Chair/ Commissioner presides over regularly scheduled public commission meetings and is the head of the agency, which maintains a staff of approximately 600 TLC employees. Over 200,000 TLC licensees complete approximately 1,000,000 trips each day. To operate for hire, drivers must first undergo a background check, have a safe driving record, and complete 24 hours of driver training. TLC-licensed vehicles are inspected for safety and emissions at TLC's Woodside Inspection Facility.

Now NYC TLC has released its Trip Record data to public for research and study purposes. There are three main taxi types in NYC: Yellow taxis are traditionally hailed by signaling to a driver who is on duty and seeking a passenger (street hail), but now they may also be hailed using an e-hail app like Curb or Arro. Yellow taxis are the only vehicles permitted to respond to a street hail from a passenger in all five boroughs. Green taxis, also known as boro taxis and street-hail liveries, were introduced in August of 2013 to improve taxi service and availability in the boroughs. Green taxis may respond to street hails, but only in the areas indicated in green on the map (i.e. above W 110 St/E 96th St in Manhattan and in the boroughs). FHV data includes trip data from high-volume for-hire vehicle bases (bases for companies dispatching 10,000+ trip per day, meaning Uber, Lyft, Via, and Juno), community livery bases, luxury limousine bases, and black car bases. Uber as one of the biggest ride-hailing services providers, its trip records are collected in High Volume For-Hire Vehicle Trip Records as well.

Based on this dataset, there are some business goals we want to achieve to improve Uber's ride-hailing service: Exploratory data analysis, research data fhvhv_tripdata_2021 and figure out underlying trip patterns in 2021. Based on fhvhv_tripdata_2021 and weather data, build predict model to predict the peak footfall. Try explore Uber's user portrait in NYC (which orders are urgent and what kind of users should be given higher priorities?)

Some useful tips about this dataset: - The trip data of the for-hire vehicles named like fhvhv_tripdata_2021-0X.parquet - Columns' description of the trip data please refer to data_dictionary_trip_records_hvfhs.pdf. - taxi_zones folder contains the geospatial data of NYC taxi zones (geopandas would be helpful). - taxi_zone_lookup.csv stores taxi zones zip code and other relevant information. - nyc 2021-01-01 to 2021-12-31.csv record the weather data of year 2021,taxi+_zone_lookup.csv stored the zone information of all taxi, data file end with .parquet could be processed by pyarrow package and convert to Pandas DataFrame.

If you find this dataset helpful, please up-vote and more high-quality datasets will be published in future!❤️

The Daily Fleet Service Report provides information on overall vehicle availability for all agencies within the City fleet, including the "DCAS Managed" fleet, which serves a number of Mayoral agencies. It also includes the Critical Fleet Summary, which tracks availability of vehicles assigned to key programs or functions, as well as availability of seasonal vehicles.

PLEASE NOTE: This dataset, which includes all TLC licensed for-hire vehicles which are in good standing and able to drive, is updated every day in the evening between 4-7pm. Please check the 'Last Update Date' field to make sure the list has updated successfully. 'Last Update Date' should show either today or yesterday's date, depending on the time of day. If the list is outdated, please download the most recent list from the link below. http://www1.nyc.gov/assets/tlc/downloads/datasets/tlc_for_hire_vehicle_active_and_inactive.csv

TLC authorized For-Hire vehicles that are active. This list is accurate to the date and time represented in the Last Date Updated and Last Time Updated fields. For inquiries about the contents of this dataset, please email licensinginquiries@tlc.nyc.gov.

This dataset contains the file of vehicle, snowmobile and boat registrations in NYS. Registrations expired more than 2 years are excluded. Records that have a scofflaw, revocation and/or suspension are included with indicators specifying those kinds of records.

This dataset contains a one-month sample of flattened EVENT data records from the New York City (NYC) Connected Vehicle (CV) Pilot that have undergone obfuscation of precise time and location details as well as other vehicle identifiers. The full unflattened event data from NYC CV pilot can be requested from the ITS DataHub Sandbox.

Each EVENT record documents the details of one application warning that occurred on an Aftermarket Safety Device (ASD) in an equipped host vehicle and includes CV messages from a defined recording time both before and after the warning was generated by the host ASD. Messages in the recording time window include the Basic Safety Messages (BSM) of the host vehicle that received the warning, as well as other BSMs received from the warning target equipped vehicle (for V2V applications) or other nearby equipped vehicles. Depending on the application warning type, MAP messages, Signal Phase and Timing (SPaT) messages, and Traveler Information Messages (TIM) that were heard by the host vehicle may also be included in the event record.

The New York City Taxi and Limousine Commission (TLC) oversees the licensing and regulation of taxi cabs and for-hire vehicles in the city. The TLC gathers data from over 200,000 license holders, including taxi drivers and limousine operators, who collectively complete around one million trips each day.

Note: The dataset used for this project was designed for educational purposes and may not accurately represent the behavior of taxi cab riders in New York City.

These records are generated from the For-Hire Vehicle (“FHV”) Trip Record submissions made by traditional livery, luxury, and black car bases. The FHV trip records include fields capturing the dispatching base license number and the pick-up date, time, and taxi zone location ID, which correspond with the NYC Taxi Zones open dataset. Each row represents a single trip in an FHV.

This dataset compiles a comprehensive database containing 90,327 street segments in New York City, covering their street design features, streetscape design, Vision Zero treatments, and neighborhood land use. It has two scales-street and street segment group (aggregation of same type of street at neighborhood). This dataset is derived based on all publicly available data, most from NYC Open Data. The detailed methods can be found in the published paper, Pedestrian and Car Occupant Crash Casualties Over a 9-Year Span of Vision Zero in New York City. To use it, please refer to the metadata file for more information and cite our work. A full list of raw data source can be found below:

Motor Vehicle Collisions – NYC Open Data: https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95

Citywide Street Centerline (CSCL) – NYC Open Data: https://data.cityofnewyork.us/City-Government/NYC-Street-Centerline-CSCL-/exjm-f27b

NYC Building Footprints – NYC Open Data: https://data.cityofnewyork.us/Housing-Development/Building-Footprints/nqwf-w8eh

Practical Canopy for New York City: https://zenodo.org/record/6547492

New York City Bike Routes – NYC Open Data: https://data.cityofnewyork.us/Transportation/New-York-City-Bike-Routes/7vsa-caz7

Sidewalk Widths NYC (originally from Sidewalk – NYC Open Data): https://www.sidewalkwidths.nyc/

LION Single Line Street Base Map - The NYC Department of City Planning (DCP): https://www.nyc.gov/site/planning/data-maps/open-data/dwn-lion.page

NYC Planimetric Database Median – NYC Open Data: https://data.cityofnewyork.us/Transportation/NYC-Planimetrics/wt4d-p43d

NYC Vision Zero Open Data (including multiple datasets including all the implementations): https://www.nyc.gov/content/visionzero/pages/open-data

NYS Traffic Data - New York State Department of Transportation Open Data: https://data.ny.gov/Transportation/NYS-Traffic-Data-Viewer/7wmy-q6mb

Smart Location Database - US Environmental Protection Agency: https://www.epa.gov/smartgrowth/smart-location-mapping

Race and ethnicity in area - American Community Survey (ACS): https://www.census.gov/programs-surveys/acs

The New York City Truck Route Network is a set of roads that commercial vehicles must use in New York City. NYC Truck Routes are also available in LION data. https://data.cityofnewyork.us/City-Government/LION/2v4z-66xt

These records are generated from the For-Hire Vehicle (“FHV”) Trip Record submissions made by traditional livery, luxury, and black car bases. The FHV trip records include fields capturing the dispatching base license number and the pick-up date, time, and taxi zone location ID, which correspond with the NYC Taxi Zones open dataset. Each row represents a single trip in an FHV.

Splitgraph serves as an HTTP API that lets you run SQL queries directly on this data to power Web applications. For example:

See the Splitgraph documentation for more information.

This dataset represents the total number of Automobiles Imported and Exported annually through maritime terminals located within Port Authority property in the Port of New York and New Jersey in vehicle units beginning 2000

This dataset contains information about motor vehicle collisions in New York City, along with associated weather conditions. Each record includes details about the crash date, location, collision ID, crash time period, contributing factors, vehicle types, the number of injuries and deaths, street name, weather description, and temperature metrics.

PLEASE NOTE: This dataset, which includes all TLC licensed medallion vehicles which are in good standing and able to drive, is updated every day in the evening between 4-7pm. Please check the 'Last Update Date' field to make sure the list has updated successfully. 'Last Update Date' should show either today or yesterday's date, depending on the time of day. If the list is outdated, please download the most recent list from the link below. http://www1.nyc.gov/assets/tlc/downloads/datasets/tlc_medallion_vehicles_authorized.csv This list contains information on the status of current medallion vehicles authorized to operate in New York City. This list is accurate to the date and time represented in the Last Date Updated and Last Time Updated fields. For inquiries about the contents of this dataset, please email licensinginquiries@tlc.nyc.gov. For a historical data up to and including 2016, please refer to https://data.cityofnewyork.us/Transportation/Historical-Medallion-Vehicles-Authorized/pvkv-25ck/

This dataset contains the file of vehicle, snowmobile and boat registrations in NYS. Expired registrations are excluded. Records that have a scofflaw, revocation and/or suspension are included with indicators specifying this.

Contains real-time traffic information from locations where NYCDOT picks up sensor feeds within the five NYC boroughs between 2018 and 2022, mostly on major arterials and highways.

Source | Usage Example

Vehicle classification counts collected by DOT for New York Metropolitan Transportation Council (NYMTC) to validate the New York Best Practice Model (NYBPM). For NYBPM screenline locations where data has been collected within the last three years are not considered for data collection for the present year. DOT collects data on at least 10% of the total NYBPM screenline locations.