For up to date data starting in 2018, please go to the new dataset at: https://data.sfgov.org/d/wg3w-h783

As of May 2018, the feed from the legacy mainframe CABLE was discontinued. It was extremely prone to issues and caused many delays in data accessibility. The new dataset linked above comes from the Crime Data Warehouse, a more reliable data system maintained by the Police Department.

This data will undergo a minor update to conform more closely to the schema of the new dataset. We will post a change notice when that work is planned. This change will not include adding new fields or backfilling data. It is provided as is. We are keeping data from the two systems separate to make it transparent to data users that there were fundamental changes.

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data. We will migrate to the new system for DataSF in the upcoming months.

1) Data introduction • San-francisco-crime dataset provides nearly 12 years of crime reports across San Francisco, from Sunset to SOMA and Marina to Excelsior.

2) Data utilization (1) San-francisco-crime data has characteristics that: • Data range is from January 1, 2003 to May 13, 2015. The goal is to predict and visualize crime categories based on 9 variables such as date, category, and description. (2) San-francisco-crime data can be used to: • Public Safety and Policing: Law enforcement agencies can use this data to enhance public safety by identifying crime hot spots, optimizing patrol routes, and allocating resources more effectively. • Urban Planning: By analyzing crime patterns, urban planners and policy makers can make informed decisions about urban development, zoning, and community services to improve safety and quality of life.

SF Crime Classification Dataset

Raw and processed files used for a Kaggle project on crime prediction in San Francisco. Full project and explanation available on GitHub:🔗 https://github.com/JulienDesdo/AI-Projects

As of November 2023, this map has been updated to use a new format. For details, please see here.
For up to date data starting in 2018, please go to the new dataset at: https://data.sfgov.org/d/wg3w-h783

As of May 2018, the feed from the legacy mainframe CABLE was discontinued. It was extremely prone to issues and caused many delays in data accessibility. The new dataset linked above comes from the Crime Data Warehouse, a more reliable data system maintained by the Police Department.

This data will undergo a minor update to conform more closely to the schema of the new dataset. We will post a change notice when that work is planned. This change will not include adding new fields or backfilling data. It is provided as is. We are keeping data from the two systems separate to make it transparent to data users that there were fundamental changes.

A log of dataset alerts open, monitored or resolved on the open data portal. Alerts can include issues as well as deprecation or discontinuation notices.

A. SUMMARY These data represent hate crimes reported by the SFPD to the California Department of Justice. Read the detailed overview of this dataset here.

What is a Hate Crime? A hate crime is a crime against a person, group, or property motivated by the victim's real or perceived protected social group. An individual may be the victim of a hate crime if they have been targeted because of their actual or perceived: (1) disability, (2) gender, (3) nationality, (4) race or ethnicity, (5) religion, (6) sexual orientation, and/or (7) association with a person or group with one or more of these actual or perceived characteristics. Hate crimes are serious crimes that may result in imprisonment or jail time.

B. HOW THE DATASET IS CREATED How is a Hate Crime Processed?
Not all prejudice incidents including the utterance of hate speech rise to the level of a hate crime. The U.S. Constitution allows hate speech if it does not interfere with the civil rights of others. While these acts are certainly hurtful, they do not rise to the level of criminal violations and thus may not be prosecuted. When a prejudice incident is reported, the reporting officer conducts a preliminary investigation and writes a crime or incident report. Bigotry must be the central motivation for an incident to be determined to be a hate crime. In that report, all facts such as verbatims or statements that occurred before or after the incident and characteristics such as the race, ethnicity, sex, religion, or sexual orientations of the victim and suspect (if known) are included. To classify a prejudice incident, the San Francisco Police Department’s Hate Crimes Unit of the Special Investigations Division conducts an analysis of the incident report to determine if the incident falls under the definition of a “hate crime” as defined by state law. California Penal Code 422.55 - Hate Crime Definition

C. UPDATE PROCESS These data are updated monthly.

D. HOW TO USE THIS DATASET This dataset includes the following information about each incident: the hate crime offense, bias type, location/time, and the number of hate crime victims and suspects. The data presented mirrors data published by the California Department of Justice, albeit at a higher frequency. The publishing of these data meet requirements set forth in PC 13023.

E. RELATED DATASETS California Department of Justice - Hate Crimes Info California Department of Justice - Hate Crimes Data

Context

DataSF seeks to transform the way that the City of San Francisco works -- through the use of data.

https://datasf.org/about/

Content

This dataset contains the following tables: ['311_service_requests', 'bikeshare_stations', 'bikeshare_status', 'bikeshare_trips', 'film_locations', 'sffd_service_calls', 'sfpd_incidents', 'street_trees']

• This data includes all San Francisco 311 service requests from July 2008 to the present, and is updated daily. 311 is a non-emergency number that provides access to non-emergency municipal services.
• This data includes fire unit responses to calls from April 2000 to present and is updated daily. Data contains the call number, incident number, address, unit identifier, call type, and disposition. Relevant time intervals are also included. Because this dataset is based on responses, and most calls involved multiple fire units, there are multiple records for each call number. Addresses are associated with a block number, intersection or call box.
• This data includes incidents from the San Francisco Police Department (SFPD) Crime Incident Reporting system, from January 2003 until the present (2 weeks ago from current date). The dataset is updated daily. Please note: the SFPD has implemented a new system for tracking crime. This dataset is still sourced from the old system, which is in the process of being retired (a multi-year process).
• This data includes a list of San Francisco Department of Public Works maintained street trees including: planting date, species, and location. Data includes 1955 to present.

This dataset is deprecated and not being updated.

Fork this kernel to get started with this dataset.

Acknowledgements

http://datasf.org/

• https://cloud.google.com/bigquery/public-data/sfo-311
• https://cloud.google.com/bigquery/public-data/sffd-service-calls
• https://cloud.google.com/bigquery/public-data/sfpd-reports
• https://cloud.google.com/bigquery/public-data/sfo-trees

Dataset Source: SF OpenData. This dataset is publicly available for anyone to use under the following terms provided by the Dataset Source - http://sfgov.org/ - and is provided "AS IS" without any warranty, express or implied, from Google. Google disclaims all liability for any damages, direct or indirect, resulting from the use of the dataset.

Banner Photo by @meric from Unplash.

Inspiration

Which neighborhoods have the highest proportion of offensive graffiti?

Which complaint is most likely to be made using Twitter and in which neighborhood?

What are the most complained about Muni stops in San Francisco?

What are the top 10 incident types that the San Francisco Fire Department responds to?

How many medical incidents and structure fires are there in each neighborhood?

What’s the average response time for each type of dispatched vehicle?

Which category of police incidents have historically been the most common in San Francisco?

What were the most common police incidents in the category of LARCENY/THEFT in 2016?

Which non-criminal incidents saw the biggest reporting change from 2015 to 2016?

What is the average tree diameter?

What is the highest number of a particular species of tree planted in a single year?

Which San Francisco locations feature the largest number of trees?

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data. We will migrate to the new system for DataSF in the upcoming months.

1) Data Introduction • The San Francisco Crime Classification Dataset is designed to classify various types of crimes reported in San Francisco. It includes detailed information such as date, category, description, day of the week, police district, resolution, and location coordinates, making it suitable for crime analysis and predictive modeling.

2) Data Utilization (1) Crime Classification data has characteristics that: • It contains detailed records of criminal activities, including the type of crime, date and time of occurrence, location, and police action taken. This allows for comprehensive analysis of crime patterns and trends in different areas of San Francisco. (2) Crime Classification data can be used to: • Law Enforcement: Assists police departments in understanding crime distribution and effectiveness of crime prevention strategies, enhancing patrol planning and resource allocation. • Urban Planning and Safety: Supports city planners and public safety officials in identifying high-crime areas, enabling the development of targeted interventions and safety measures.

This data includes incidents from the San Francisco Police Department (SFPD) Crime Incident Reporting system, from January 2003 until the present (2 weeks ago from current date). The dataset is updated daily. Please note: the SFPD has implemented a new system for tracking crime. This dataset is still sourced from the old system, which is in the process of being retired (a multi-year process). This public dataset is hosted in Google BigQuery and is included in BigQuery's 1TB/mo of free tier processing. This means that each user receives 1TB of free BigQuery processing every month, which can be used to run queries on this public dataset. Watch this short video to learn how to get started quickly using BigQuery to access public datasets. What is BigQuery .

[Update 04/16/2018]: We are still developing the automation for the new dataset. We do not have an updated publishing date at the moment. We have a target schema and have provided a crosswalk document attached to the existing dataset in advance of the changes. We will update this document if there's new information to share.

[Change Notice 03/13/2018]: By the end of this month, this dataset will become historical and a new one will be created starting with incident data in 2018. This one will remain here, but no longer be updated. The new one will have data coming from a new system, will not have a 2 week lag, and have updated districts among other quality improvements. We will attach a guide here with more detailed change updates as soon as we have them.

+++++++++

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data.

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data. We will migrate to the new system for DataSF in the upcoming months.

Violent and property crime rates per 100,000 population for San Mateo County and the State of California. The total crimes used to calculate the rates for San Mateo County include data from: Sheriff's Department Unincorporated, Atherton, Belmont, Brisbane, Broadmoor, Burlingame, Colma, Daly City, East Palo Alto, Foster City, Half Moon Bay, Hillsborough, Menlo Park, Millbrae, Pacifica, Redwood City, San Bruno, San Carlos, San Mateo, South San Francisco, Bay Area DPR, BART, Union Pacific Railroad, and CA Highway Patrol.

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data. We will migrate to the new system for DataSF in the upcoming months.

This survey was conducted by the Center for Urban Affairs and Policy Research at Northwestern University to gather information for two projects that analyzed the impact of crime on the lives of city dwellers. These projects were the Reactions to Crime (RTC) Project, which was supported by the United States Department of Justice's National Institute of Justice as part of its Research Agreements Program, and the Rape Project, supported by the National Center for the Prevention and Control of Rape, a subdivision of the National Institute of Mental Health. Both investigations were concerned with individual behavior and collective reactions to crime. The Rape Project was specifically concerned with sexual assault and its consequences for the lives of women. The three cities selected for study were Chicago, Philadelphia, and San Francisco. A total of ten neighborhoods were chosen from these cities along a number of dimensions -- ethnicity, class, crime, and levels of organizational activity. In addition, a small city-wide sample was drawn from each city. Reactions to crime topics covered how individuals band together to deal with crime problems, individual responses to crime such as property marking or the installation of locks and bars, and the impact of fear of crime on day-to-day behavior -- for example, shopping and recreational patterns. Respondents were asked several questions that called for self-reports of behavior, including events and conditions in their home areas, their relationship to their neighbors, who they knew and visited around their homes, and what they watched on TV and read in the newspapers. Also included were a number of questions measuring respondents' perceptions of the extent of crime in their communities, whether they knew someone who had been a victim, and what they had done to reduce their own chances of being victimized. Questions on sexual assault/rape included whether the respondent thought this was a neighborhood problem, if the number of rapes in the neighborhood were increasing or decreasing, how many women they thought had been sexually assaulted or raped in the neighborhood in the previous year, and how they felt about various rape prevention measures, such as increasing home security, women not going out alone at night, women dressing more modestly, learning self-defense techniques, carrying weapons, increasing men's respect of women, and newspapers publishing the names of known rapists. Female respondents were asked whether they thought it likely that they would be sexually assaulted in the next year, how much they feared sexual assault when going out alone after dark in the neighborhood, whether they knew a sexual assault victim, whether they had reported any sexual assaults to police, and where and when sexual assaults took place that they were aware of. Demographic information collected on respondents includes age, race, ethnicity, education, occupation, income, and whether the respondent owned or rented their home.

Dataset

This dataset was created by PJIANG

Contents

As of July 19, 2015, the PD District boundaries have been updated through a redistricting process. These new boundaries are not reflected in the dataset yet so you cannot compare data from July 19, 2015 onward to official reports from PD with the Police District column. We are working on an update to the dataset to reflect the updated boundaries starting with data entered July 19 onward.

Incidents derived from SFPD Crime Incident Reporting system Updated daily, showing data from 1/1/2003 up until two weeks ago from current date. Please note: San Francisco police have implemented a new system for tracking crime. The dataset included here is still coming from the old system, which is in the process of being retired (a multi-year process). Data included here is no longer the official SFPD data. We will migrate to the new system for DataSF in the upcoming months.

San Francisco Police Department Crime Reporting Plots. These have historically been used for reporting various stats. Derived from shapefile sent by SFPD in May 2003.

Context

This dataset was put together from multiple sources of open data I found online, which I used in a first iteration to predict the probability of crime in a given neighborhood and date in San Fransisco. The dataset is perhaps a bit unfocused in itself, rather, the focus is the kernel for which the data is used.

Content

This dataset combines multiple data sources I found online when working on this kernel.

Acknowledgements

The sources of each file of the dataset are listed under the description of each file. Please refer to additional informatioon at these sources before using the data.

Inspiration

I'm exploring predicting crime. And want to learn. And help others learn.