Context

NYC Open Data is an opportunity to engage New Yorkers in the information that is produced and used by City government. We believe that every New Yorker can benefit from Open Data, and Open Data can benefit from every New Yorker. Source: https://opendata.cityofnewyork.us/overview/

Content

Thanks to NYC Open Data, which makes public data generated by city agencies available for public use, and Citi Bike, we've incorporated over 150 GB of data in 5 open datasets into Google BigQuery Public Datasets, including:

• Over 8 million 311 service requests from 2012-2016

• More than 1 million motor vehicle collisions 2012-present

• Citi Bike stations and 30 million Citi Bike trips 2013-present

• Over 1 billion Yellow and Green Taxi rides from 2009-present

• Over 500,000 sidewalk trees surveyed decennially in 1995, 2005, and 2015

This dataset is deprecated and not being updated.

Fork this kernel to get started with this dataset.

Acknowledgements

https://opendata.cityofnewyork.us/

https://cloud.google.com/blog/big-data/2017/01/new-york-city-public-datasets-now-available-on-google-bigquery

This dataset is publicly available for anyone to use under the following terms provided by the Dataset Source - https://data.cityofnewyork.us/ - and is provided "AS IS" without any warranty, express or implied, from Google. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

By accessing datasets and feeds available through NYC Open Data, the user agrees to all of the Terms of Use of NYC.gov as well as the Privacy Policy for NYC.gov. The user also agrees to any additional terms of use defined by the agencies, bureaus, and offices providing data. Public data sets made available on NYC Open Data are provided for informational purposes. The City does not warranty the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set made available on NYC Open Data, nor are any such warranties to be implied or inferred with respect to the public data sets furnished therein.

The City is not liable for any deficiencies in the completeness, accuracy, content, or fitness for any particular purpose or use of any public data set, or application utilizing such data set, provided by any third party.

Banner Photo by @bicadmedia from Unplash.

Inspiration

On which New York City streets are you most likely to find a loud party?

Can you find the Virginia Pines in New York City?

Where was the only collision caused by an animal that injured a cyclist?

What’s the Citi Bike record for the Longest Distance in the Shortest Time (on a route with at least 100 rides)?

https://cloud.google.com/blog/big-data/2017/01/images/148467900588042/nyc-dataset-6.png" alt="enter image description here"> https://cloud.google.com/blog/big-data/2017/01/images/148467900588042/nyc-dataset-6.png

About this dataset

Context

The dataset tabulates the New York population over the last 20 plus years. It lists the population for each year, along with the year on year change in population, as well as the change in percentage terms for each year. The dataset can be utilized to understand the population change of New York across the last two decades. For example, using this dataset, we can identify if the population is declining or increasing. If there is a change, when the population peaked, or if it is still growing and has not reached its peak. We can also compare the trend with the overall trend of United States population over the same period of time.

Key observations

In 2023, the population of New York was 8.26 million, a 0.93% decrease year-by-year from 2022. Previously, in 2022, New York population was 8.34 million, a decline of 1.49% compared to a population of 8.46 million in 2021. Over the last 20 plus years, between 2000 and 2023, population of New York increased by 242,826. In this period, the peak population was 8.74 million in the year 2020. The numbers suggest that the population has already reached its peak and is showing a trend of decline. Source: U.S. Census Bureau Population Estimates Program (PEP).

Content

When available, the data consists of estimates from the U.S. Census Bureau Population Estimates Program (PEP).

Data Coverage:

• From 2000 to 2023

Variables / Data Columns

• Year: This column displays the data year (Measured annually and for years 2000 to 2023)
• Population: The population for the specific year for the New York is shown in this column.
• Year on Year Change: This column displays the change in New York population for each year compared to the previous year.
• Change in Percent: This column displays the year on year change as a percentage. Please note that the sum of all percentages may not equal one due to rounding of values.

Good to know

Margin of Error

Data in the dataset are based on the estimates and are subject to sampling variability and thus a margin of error. Neilsberg Research recommends using caution when presening these estimates in your research.

Custom data

If you do need custom data for any of your research project, report or presentation, you can contact our research staff at research@neilsberg.com for a feasibility of a custom tabulation on a fee-for-service basis.

Inspiration

Neilsberg Research Team curates, analyze and publishes demographics and economic data from a variety of public and proprietary sources, each of which often includes multiple surveys and programs. The large majority of Neilsberg Research aggregated datasets and insights is made available for free download at https://www.neilsberg.com/research/.

Recommended for further research

This dataset is a part of the main dataset for New York Population by Year. You can refer the same here

Uber TLC FOIL Response

This directory contains data on over 4.5 million Uber pickups in New York City from April to September 2014, and 14.3 million more Uber pickups from January to June 2015. Trip-level data on 10 other for-hire vehicle (FHV) companies, as well as aggregated data for 329 FHV companies, is also included. All the files are as they were received on August 3, Sept. 15 and Sept. 22, 2015.

FiveThirtyEight obtained the data from the NYC Taxi & Limousine Commission (TLC) by submitting a Freedom of Information Law request on July 20, 2015. The TLC has sent us the data in batches as it continues to review trip data Uber and other HFV companies have submitted to it. The TLC's correspondence with FiveThirtyEight is included in the files TLC_letter.pdf, TLC_letter2.pdf and TLC_letter3.pdf. TLC records requests can be made here.

This data was used for four FiveThirtyEight stories: Uber Is Serving New York’s Outer Boroughs More Than Taxis Are, Public Transit Should Be Uber’s New Best Friend, Uber Is Taking Millions Of Manhattan Rides Away From Taxis, and Is Uber Making NYC Rush-Hour Traffic Worse?.

The Data

The dataset contains, roughly, four groups of files:

• Uber trip data from 2014 (April - September), separated by month, with detailed location information
• Uber trip data from 2015 (January - June), with less fine-grained location information
• non-Uber FHV (For-Hire Vehicle) trips. The trip information varies by company, but can include day of trip, time of trip, pickup location, driver's for-hire license number, and vehicle's for-hire license number.
• aggregate ride and vehicle statistics for all FHV companies (and, occasionally, for taxi companies)

Uber trip data from 2014

There are six files of raw data on Uber pickups in New York City from April to September 2014. The files are separated by month and each has the following columns:

• Date/Time : The date and time of the Uber pickup
• Lat : The latitude of the Uber pickup
• Lon : The longitude of the Uber pickup
• Base : The TLC base company code affiliated with the Uber pickup

These files are named:

• uber-raw-data-apr14.csv
• uber-raw-data-aug14.csv
• uber-raw-data-jul14.csv
• uber-raw-data-jun14.csv
• uber-raw-data-may14.csv
• uber-raw-data-sep14.csv

Uber trip data from 2015

Also included is the file uber-raw-data-janjune-15.csv This file has the following columns:

• Dispatching_base_num : The TLC base company code of the base that dispatched the Uber
• Pickup_date : The date and time of the Uber pickup
• Affiliated_base_num : The TLC base company code affiliated with the Uber pickup
• locationID : The pickup location ID affiliated with the Uber pickup

The Base codes are for the following Uber bases:

B02512 : Unter B02598 : Hinter B02617 : Weiter B02682 : Schmecken B02764 : Danach-NY B02765 : Grun B02835 : Dreist B02836 : Drinnen

For coarse-grained location information from these pickups, the file taxi-zone-lookup.csv shows the taxi Zone (essentially, neighborhood) and Borough for each locationID.

Non-Uber FLV trips

The dataset also contains 10 files of raw data on pickups from 10 for-hire vehicle (FHV) companies. The trip information varies by company, but can include day of trip, time of trip, pickup location, driver's for-hire license number, and vehicle's for-hire license number.

These files are named:

• American_B01362.csv
• Diplo_B01196.csv
• Highclass_B01717.csv
• Skyline_B00111.csv
• Carmel_B00256.csv
• Federal_02216.csv
• Lyft_B02510.csv
• Dial7_B00887.csv
• Firstclass_B01536.csv
• Prestige_B01338.csv

Aggregate Statistics

There is also a file other-FHV-data-jan-aug-2015.csv containing daily pickup data for 329 FHV companies from January 2015 through August 2015.

The file Uber-Jan-Feb-FOIL.csv contains aggregated daily Uber trip statistics in January and February 2015.

The New York Times Annotated Corpus contains over 1.8 million articles written and published by the New York Times between January 1, 1987 and June 19, 2007 with article metadata provided by the New York Times Newsroom, the New York Times Indexing Service and the online production staff at nytimes.com. The corpus includes:

Over 1.8 million articles (excluding wire services articles that appeared during the covered period). Over 650,000 article summaries written by library scientists. Over 1,500,000 articles manually tagged by library scientists with tags drawn from a normalized indexing vocabulary of people, organizations, locations and topic descriptors. Over 275,000 algorithmically-tagged articles that have been hand verified by the online production staff at nytimes.com. As part of the New York Times' indexing procedures, most articles are manually summarized and tagged by a staff of library scientists. This collection contains over 650,000 article-summary pairs which may prove to be useful in the development and evaluation of algorithms for automated document summarization. Also, over 1.5 million documents have at least one tag. Articles are tagged for persons, places, organizations, titles and topics using a controlled vocabulary that is applied consistently across articles. For instance if one article mentions "Bill Clinton" and another refers to "President William Jefferson Clinton", both articles will be tagged with "CLINTON, BILL".

Description in Spanish, original page The data in this dataset was collected by Properati.

Context

One of the best applications of data science and machine learning in general is the real estate business. This data set provides data for those who want to make data analysis and use of machine learning models to perform multiple tasks and generate new insights.

Content

It consists of a .csv where each row contains a publication. The .csv contains no missing data, this means that it is almost ready for use and model training. The only thing necessary is to convert the "string" type data into numerical data.

Columns

id - Notice identifier. It is not unique: if the notification is updated by the real estate agency (new version of the notification) a new record is created with the same id but different dates: registration and cancellation.

operation_type - Type of operation (these are all sales, can be removed).

l2 - Administrative level 2: usually province

l3 - Administrative level 3: usually city

lat - Latitude.

lon - Longitude.

price - Price published in the ad.

property_type - Type of property (House, Apartment, PH).

rooms - Number of rooms (useful in Argentina).

bathrooms - Number of bathrooms.

start_date - Date when the ad was created.

end_date - Date of termination of the advertisement.

created_on - Date when the first version of the notice was created.

surface_total - Total area in m².

surface_covered - Covered area in m².

title - Title of the advertisement.

description - Description of the advertisement.

ad_type - Type of ad (Property, Development/Project).

Acknowledgements

The data in this dataset was collected by Properati.

More than 22 million AAdvantage members redeem award miles each year, and changing those award tickets by phone is a smart option. ☎️+1 (855) 217-1878 Phone agents provide real-time assistance with availability and policy enforcement. ☎️+1 (855) 217-1878

To change an award ticket, call American’s reservation line at ☎️+1 (855) 217-1878 and have your AAdvantage number, confirmation code, and travel dates ready. ☎️+1 (855) 217-1878 Award tickets can usually be changed for no fee—but rules depend on fare class.

As of 2023, American Airlines eliminated change and redeposit fees for most award tickets, making modifications easier than ever. ☎️+1 (855) 217-1878 Still, fare differences apply if you book a more expensive itinerary. ☎️+1 (855) 217-1878 The agent will quote the new miles needed.

Award availability varies, so what you want may not be open. Phone agents can search multiple date and time combinations more efficiently. ☎️+1 (855) 217-1878 This is especially helpful on complex routes. ☎️+1 (855) 217-1878 Domestic awards, in particular, fill up fast.

If you’re switching to a partner airline (e.g., British Airways or Japan Airlines), award rules may differ. ☎️+1 (855) 217-1878 The phone agent will guide you through changes that comply with partner policies. ☎️+1 (855) 217-1878 Some partners charge more miles or impose blackout dates.

You can change the date, time, and routing of an award ticket by phone, but not the passenger name. ☎️+1 (855) 217-1878 Miles are non-transferable after booking. ☎️+1 (855) 217-1878 Agents can, however, redeposit miles to your account if you cancel.

If you’re within 24 hours of departure, award changes may be restricted or require rebooking under a new ticket. ☎️+1 (855) 217-1878 Some seats may no longer be available. ☎️+1 (855) 217-1878 Always call early if you need to adjust.

Canceling an award ticket by phone triggers a redeposit of miles (if eligible), often within 24–48 hours. ☎️+1 (855) 217-1878 Redeposit fees used to be $150 per ticket but were eliminated for most users. ☎️+1 (855) 217-1878 Confirm with the phone agent.

Award tickets booked with travel certificates or promotions may follow different rules. ☎️+1 (855) 217-1878 Be sure to ask about restrictions before making a change. ☎️+1 (855) 217-1878 Some promotional redemptions are non-changeable or non-refundable.

For multi-city award itineraries, changes must often be processed manually, which makes the phone line your best option. ☎️+1 (855) 217-1878 The online system struggles with complex award modifications. ☎️+1 (855) 217-1878 Phone agents also ensure taxes are recalculated correctly.

Elite members (Platinum Pro and Executive Platinum) receive priority award assistance, so mention your tier at the beginning of the call. ☎️+1 (855) 217-1878 Higher status may unlock better availability. ☎️+1 (855) 217-1878 Ask about using systemwide upgrades, too.

When your new flight requires more miles, you can top off your AAdvantage balance by purchasing additional miles. ☎️+1 (855) 217-1878 Ask the phone agent for a quote. ☎️+1 (855) 217-1878 This is useful when you're just short of the required total.

Always ask for an email confirmation after any award change. ☎️+1 (855) 217-1878 Save the new itinerary and receipt for your records in case of disputes. ☎️+1 (855) 217-1878 Also ask about any expiration dates tied to rebooked award travel.