!!!WARNING~~~This dataset has a large number of flaws and is unable to properly answer many questions that people generally use it to answer, such as whether national hate crimes are changing (or at least they use the data so improperly that they get the wrong answer). A large number of people using this data (academics, advocates, reporting, US Congress) do so inappropriately and get the wrong answer to their questions as a result. Indeed, many published papers using this data should be retracted. Before using this data I highly recommend that you thoroughly read my book on UCR data, particularly the chapter on hate crimes (https://ucrbook.com/hate-crimes.html) as well as the FBI's own manual on this data. The questions you could potentially answer well are relatively narrow and generally exclude any causal relationships. ~~~WARNING!!!For a comprehensive guide to this data and other UCR data, please see my book at ucrbook.comVersion 10 release notes:Adds 2022 dataVersion 9 release notes:Adds 2021 data.Version 8 release notes:Adds 2019 and 2020 data. Please note that the FBI has retired UCR data ending in 2020 data so this will be the last UCR hate crime data they release. Changes .rda file to .rds.Version 7 release notes:Changes release notes description, does not change data.Version 6 release notes:Adds 2018 dataVersion 5 release notes:Adds data in the following formats: SPSS, SAS, and Excel.Changes project name to avoid confusing this data for the ones done by NACJD.Adds data for 1991.Fixes bug where bias motivation "anti-lesbian, gay, bisexual, or transgender, mixed group (lgbt)" was labeled "anti-homosexual (gay and lesbian)" prior to 2013 causing there to be two columns and zero values for years with the wrong label.All data is now directly from the FBI, not NACJD. The data initially comes as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. Version 4 release notes: Adds data for 2017.Adds rows that submitted a zero-report (i.e. that agency reported no hate crimes in the year). This is for all years 1992-2017. Made changes to categorical variables (e.g. bias motivation columns) to make categories consistent over time. Different years had slightly different names (e.g. 'anti-am indian' and 'anti-american indian') which I made consistent. Made the 'population' column which is the total population in that agency. Version 3 release notes: Adds data for 2016.Order rows by year (descending) and ORI.Version 2 release notes: Fix bug where Philadelphia Police Department had incorrect FIPS county code. The Hate Crime data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains information about hate crimes reported in the United States. Please note that the files are quite large and may take some time to open.Each row indicates a hate crime incident for an agency in a given year. I have made a unique ID column ("unique_id") by combining the year, agency ORI9 (the 9 character Originating Identifier code), and incident number columns together. Each column is a variable related to that incident or to the reporting agency. Some of the important columns are the incident date, what crime occurred (up to 10 crimes), the number of victims for each of these crimes, the bias motivation for each of these crimes, and the location of each crime. It also includes the total number of victims, total number of offenders, and race of offenders (as a group). Finally, it has a number of columns indicating if the victim for each offense was a certain type of victim or not (e.g. individual victim, business victim religious victim, etc.). The only changes I made to the data are the following. Minor changes to column names to make all column names 32 characters or fewer (so it can be saved in a Stata format), made all character values lower case, reordered columns. I also generated incident month, weekday, and month-day variables from the incident date variable included in the original data.

Context

The Uniform Crime Reporting (UCR) Program has been the starting place for law enforcement executives, students of criminal justice, researchers, members of the media, and the public at large seeking information on crime in the nation. The program was conceived in 1929 by the International Association of Chiefs of Police to meet the need for reliable uniform crime statistics for the nation. In 1930, the FBI was tasked with collecting, publishing, and archiving those statistics.

Today, four annual publications, Crime in the United States, National Incident-Based Reporting System, Law Enforcement Officers Killed and Assaulted, and Hate Crime Statistics are produced from data received from over 18,000 city, university/college, county, state, tribal, and federal law enforcement agencies voluntarily participating in the program. The crime data are submitted either through a state UCR Program or directly to the FBI’s UCR Program.

This dataset focuses on the crime rates and law enforcement employment data in the state of California.

Content

Crime and law enforcement employment rates are separated into individual files, focusing on offenses by enforcement agency, college/university campus, county, and city. Categories of crimes reported include violent crime, murder and nonnegligent manslaughter, rape, robbery, aggravated assault, property crime, burglary, larceny-theft, motor vehicle damage, and arson. In the case of rape, data is collected for both revised and legacy definitions. In some cases, a small number of enforcement agencies switched definition collection sometime within the same year.

Acknowledgements

This dataset originates from the FBI UCR project, and the complete dataset for all 2015 crime reports can be found here.

Inspiration

• What are the most common types of crimes in California? Are there certain crimes that are more common in a particular place category, such as a college/university campus, compared to the rest of the state?
• How does the number of law enforcement officers compare to the crime rates of a particular area? Is the ratio similar throughout the state, or do certain campuses, counties, or cities have a differing rate?
• How does the legacy vs. refined definition of rape differ, and how do the rape counts compare? If you pulled the same data from FBI datasets for previous years, can you see a difference in rape rates over time?

The FBI NIBRS (National Incident-Based Reporting System) data is the way the FBI is currently asking police agencies across the US to report crime data in their jurisdictions. This is coming to replace the traditional Summary Reporting System (SRS) in which the data from crimes was aggregated and so many details of crimes were not recorded. NIBRS includes details on each single crime incident—as well as on separate offenses within the same incident—including information on victims, known offenders, relationships between victims and offenders, arrestees, and property involved in crimes. It is important to note that not all agencies in every state have contributed to the NIBRS, therefore as you can see in the image below, not all states have data covering all their population. https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F6432833%2F589a07f0116dcb6fab8892d2fc74e966%2Fnibrs_pop_coverage_map_2021.png?generation=1672211210548630&alt=media" alt=""> All the data is available here

***Starting on March 7th, 2024, the Los Angeles Police Department (LAPD) will adopt a new Records Management System for reporting crimes and arrests. This new system is being implemented to comply with the FBI's mandate to collect NIBRS-only data (NIBRS — FBI - https://www.fbi.gov/how-we-can-help-you/more-fbi-services-and-information/ucr/nibrs). During this transition, users will temporarily see only incidents reported in the retiring system. However, the LAPD is actively working on generating new NIBRS datasets to ensure a smoother and more efficient reporting system. *** **Update 1/18/2024 - LAPD is facing issues with posting the Crime data, but we are taking immediate action to resolve the problem. We understand the importance of providing reliable and up-to-date information and are committed to delivering it. As we work through the issues, we have temporarily reduced our updates from weekly to bi-weekly to ensure that we provide accurate information. Our team is actively working to identify and resolve these issues promptly. We apologize for any inconvenience this may cause and appreciate your understanding. Rest assured, we are doing everything we can to fix the problem and get back to providing weekly updates as soon as possible. ** This dataset reflects incidents of crime in the City of Los Angeles dating back to 2020. This data is transcribed from original crime reports that are typed on paper and therefore there may be some inaccuracies within the data. Some _location fields with missing data are noted as (0°, 0°). Address fields are only provided to the nearest hundred block in order to maintain privacy. This data is as accurate as the data in the database. Please note questions or concerns in the comments.

Content

Hate Crimes

This folder contains data behind the story Higher Rates Of Hate Crimes Are Tied To Income Inequality.

Header	Definition
state	State name
median_household_income	Median household income, 2016
share_unemployed_seasonal	Share of the population that is unemployed (seasonally adjusted), Sept. 2016
share_population_in_metro_areas	Share of the population that lives in metropolitan areas, 2015
share_population_with_high_school_degree	Share of adults 25 and older with a high-school degree, 2009
share_non_citizen	Share of the population that are not U.S. citizens, 2015
share_white_poverty	Share of white residents who are living in poverty, 2015
gini_index	Gini Index, 2015
share_non_white	Share of the population that is not white, 2015
share_voters_voted_trump	Share of 2016 U.S. presidential voters who voted for Donald Trump
hate_crimes_per_100k_splc	Hate crimes per 100,000 population, Southern Poverty Law Center, Nov. 9-18, 2016
avg_hatecrimes_per_100k_fbi	Average annual hate crimes per 100,000 population, FBI, 2010-2015

Sources: Kaiser Family Foundation Kaiser Family Foundation Kaiser Family Foundation Census Bureau Kaiser Family Foundation Kaiser Family Foundation Census Bureau Kaiser Family Foundation United States Elections Project Southern Poverty Law Center FBI

Correction

Please see the following commit: https://github.com/fivethirtyeight/data/commit/fbc884a5c8d45a0636e1d6b000021632a0861986

Context

This is a dataset from FiveThirtyEight hosted on their GitHub. Explore FiveThirtyEight data using Kaggle and all of the data sources available through the FiveThirtyEight organization page!

• Update Frequency: This dataset is updated daily.

Acknowledgements

This dataset is maintained using GitHub's API and Kaggle's API.

This dataset is distributed under the Attribution 4.0 International (CC BY 4.0) license.

It is important for the community to understand what is – and is not – a hate crime. First and foremost, the incident must be a crime. Although that may seem obvious, most speech is not a hate crime, regardless of how offensive it may be. In addition, a hate crime is not a crime, but a possible motive for a crime.It can be difficult to establish a motive for a crime. Therefore, the classification as a hate crime is subject to change as an investigation proceeds – even as prosecutors continue an investigation. If a person is found guilty of a hate crime, the court may fine the offender up to 1½ times the maximum fine and imprison him or her for up to 1½ times the maximum term authorized for the underlying crime.While the District strives to reduce crime for all residents of and visitors to the city, hate crimes can make a particular community feel vulnerable and more fearful. This is unacceptable, and is the reason everyone must work together not just to address allegations of hate crimes, but also to proactively educate the public about hate crimes.The figures in this data align with DC Official Code 22-3700. Because the DC statute differs from the FBI Uniform Crime Reporting (UCR) and National Incident-Based Reporting System (NIBRS) definitions, these figures may be higher than those reported to the FBI.Each month, an MPD team reviews crimes that have been identified as potentially motivated by hate/bias to determine whether there is sufficient information to support that designation. The data in this document is current through the end of the most recent month.The hate crimes dataset is not an official MPD database of record and may not match details in records pulled from the official Records Management System (RMS).Unknown or blank values in the Targeted Group field may be present prior to 2016 data. As of January 2022, an offense with multiple bias categories would be reflected as such.Data is updated on the 15th of every month.

Alaska crime data from 2000 to present from the FBI Uniform Crime Reporting (UCR) program. Information includes data on both violent and property crime.The UCR Program's primary objective is to generate reliable information for use in law enforcement administration, operation, and management; over the years, however, the data have become one of the country’s leading social indicators. The program has been the starting place for law enforcement executives, students of criminal justice, researchers, members of the media, and the public at large seeking information on crime in the nation. The program was conceived in 1929 by the International Association of Chiefs of Police to meet the need for reliable uniform crime statistics for the nation. In 1930, the FBI was tasked with collecting, publishing, and archiving those statistics.Source: US Federal Bureau of Investigation (FBI)This data has been visualized in a Geographic Information Systems (GIS) format and is provided as a service in the DCRA Information Portal by the Alaska Department of Commerce, Community, and Economic Development Division of Community and Regional Affairs (SOA DCCED DCRA), Research and Analysis section. SOA DCCED DCRA Research and Analysis is not the authoritative source for this data. For more information and for questions about this data, see: FBI UCR ProgramOffenses Known to Law Enforcement, by State by City, 2017 The FBI collects these data through the Uniform Crime Reporting (UCR) Program. Important note about rape data In 2013, the FBI’s UCR Program initiated the collection of rape data under a revised definition within the Summary Based Reporting System. The term “forcible” was removed from the offense name, and the definition was changed to “penetration, no matter how slight, of the vagina or anus with any body part or object, or oral penetration by a sex organ of another person, without the consent of the victim.” In 2016, the FBI Director approved the recommendation to discontinue the reporting of rape data using the UCR legacy definition beginning in 2017. General comment This table provides the volume of violent crime (murder and nonnegligent manslaughter, rape, robbery, and aggravated assault) and property crime (burglary, larceny-theft, and motor vehicle theft) as reported by city and town law enforcement agencies (listed alphabetically by state) that contributed data to the UCR Program. (Note: Arson is not included in the property crime total in this table; however, if complete arson data were provided, it will appear in the arson column.) Caution against ranking Readers should take into consideration relevant factors in addition to an area’s crime statistics when making any valid comparisons of crime among different locales. UCR Statistics: Their Proper Use provides more details. Methodology The data used in creating this table were from all city and town law enforcement agencies submitting 12 months of complete offense data for 2017. Rape figures, and violent crime, which rape is a part, will not be published in this table for agencies submitting rape using the UCR legacy rape definition. The rape figures, and violent crime, which rape is a part, published in this table are from only those agencies using the UCR revised rape definition as well as converted data from agencies that reported data for rape, sodomy, and sexual assault with an object via NIBRS. The FBI does not publish arson data unless it receives data from either the agency or the state for all 12 months of the calendar year. When the FBI determines that an agency’s data collection methodology does not comply with national UCR guidelines, the figure(s) for that agency’s offense(s) will not be included in the table, and the discrepancy will be explained in a footnote. Population estimation For the 2017 population estimates used in this table, the FBI computed individual rates of growth from one year to the next for every city/town and county using 2010 decennial population counts and 2011 through 2016 population estimates from the U.S. Census Bureau. Each agency’s rates of growth were averaged; that average was then applied and added to its 2016 Census population estimate to derive the agency’s 2017 population estimate.

Context

Introduction: The dataset used for this experiment is real and authentic. The dataset is acquired from UCI machine learning repository website [13]. The title of the dataset is ‘Crime and Communities’. It is prepared using real data from socio-economic data from 1990 US Census, law enforcement data from the 1990 US LEMAS survey, and crimedata from the 1995 FBI UCR [13]. This dataset contains a total number of 147 attributes and 2216 instances.

The per capita crimes variables were calculated using population values included in the 1995 FBI data (which differ from the 1990 Census values).

Content

The variables included in the dataset involve the community, such as the percent of the population considered urban, and the median family income, and involving law enforcement, such as per capita number of police officers, and percent of officers assigned to drug units. The crime attributes (N=18) that could be predicted are the 8 crimes considered 'Index Crimes' by the FBI)(Murders, Rape, Robbery, .... ), per capita (actually per 100,000 population) versions of each, and Per Capita Violent Crimes and Per Capita Nonviolent Crimes)

predictive variables : 125 non-predictive variables : 4 potential goal/response variables : 18

Acknowledgements

http://archive.ics.uci.edu/ml/datasets/Communities%20and%20Crime%20Unnormalized

U. S. Department of Commerce, Bureau of the Census, Census Of Population And Housing 1990 United States: Summary Tape File 1a & 3a (Computer Files),

U.S. Department Of Commerce, Bureau Of The Census Producer, Washington, DC and Inter-university Consortium for Political and Social Research Ann Arbor, Michigan. (1992)

U.S. Department of Justice, Bureau of Justice Statistics, Law Enforcement Management And Administrative Statistics (Computer File) U.S. Department Of Commerce, Bureau Of The Census Producer, Washington, DC and Inter-university Consortium for Political and Social Research Ann Arbor, Michigan. (1992)

U.S. Department of Justice, Federal Bureau of Investigation, Crime in the United States (Computer File) (1995)

Inspiration

Your data will be in front of the world's largest data science community. What questions do you want to see answered?

Data available in the dataset may not act as a complete source of information for identifying factors that contribute to more violent and non-violent crimes as many relevant factors may still be missing.

However, I would like to try and answer the following questions answered.

1. Analyze if number of vacant and occupied houses and the period of time the houses were vacant had contributed to any significant change in violent and non-violent crime rates in communities

2. How has unemployment changed crime rate(violent and non-violent) in the communities?

3. Were people from a particular age group more vulnerable to crime?

4. Does ethnicity play a role in crime rate?

5. Has education played a role in bringing down the crime rate?

THIS DATASET WAS LAST UPDATED AT 2:11 AM EASTERN ON JULY 12

OVERVIEW

2019 had the most mass killings since at least the 1970s, according to the Associated Press/USA TODAY/Northeastern University Mass Killings Database.

In all, there were 45 mass killings, defined as when four or more people are killed excluding the perpetrator. Of those, 33 were mass shootings . This summer was especially violent, with three high-profile public mass shootings occurring in the span of just four weeks, leaving 38 killed and 66 injured.

A total of 229 people died in mass killings in 2019.

The AP's analysis found that more than 50% of the incidents were family annihilations, which is similar to prior years. Although they are far less common, the 9 public mass shootings during the year were the most deadly type of mass murder, resulting in 73 people's deaths, not including the assailants.

One-third of the offenders died at the scene of the killing or soon after, half from suicides.

About this Dataset

The Associated Press/USA TODAY/Northeastern University Mass Killings database tracks all U.S. homicides since 2006 involving four or more people killed (not including the offender) over a short period of time (24 hours) regardless of weapon, location, victim-offender relationship or motive. The database includes information on these and other characteristics concerning the incidents, offenders, and victims.

The AP/USA TODAY/Northeastern database represents the most complete tracking of mass murders by the above definition currently available. Other efforts, such as the Gun Violence Archive or Everytown for Gun Safety may include events that do not meet our criteria, but a review of these sites and others indicates that this database contains every event that matches the definition, including some not tracked by other organizations.

This data will be updated periodically and can be used as an ongoing resource to help cover these events.

Using this Dataset

To get basic counts of incidents of mass killings and mass shootings by year nationwide, use these queries:

Mass killings by year

Mass shootings by year

To get these counts just for your state:

Filter killings by state

Definition of "mass murder"

Mass murder is defined as the intentional killing of four or more victims by any means within a 24-hour period, excluding the deaths of unborn children and the offender(s). The standard of four or more dead was initially set by the FBI.

This definition does not exclude cases based on method (e.g., shootings only), type or motivation (e.g., public only), victim-offender relationship (e.g., strangers only), or number of locations (e.g., one). The time frame of 24 hours was chosen to eliminate conflation with spree killers, who kill multiple victims in quick succession in different locations or incidents, and to satisfy the traditional requirement of occurring in a “single incident.”

Offenders who commit mass murder during a spree (before or after committing additional homicides) are included in the database, and all victims within seven days of the mass murder are included in the victim count. Negligent homicides related to driving under the influence or accidental fires are excluded due to the lack of offender intent. Only incidents occurring within the 50 states and Washington D.C. are considered.

Methodology

Project researchers first identified potential incidents using the Federal Bureau of Investigation’s Supplementary Homicide Reports (SHR). Homicide incidents in the SHR were flagged as potential mass murder cases if four or more victims were reported on the same record, and the type of death was murder or non-negligent manslaughter.

Cases were subsequently verified utilizing media accounts, court documents, academic journal articles, books, and local law enforcement records obtained through Freedom of Information Act (FOIA) requests. Each data point was corroborated by multiple sources, which were compiled into a single document to assess the quality of information.

In case(s) of contradiction among sources, official law enforcement or court records were used, when available, followed by the most recent media or academic source.

Case information was subsequently compared with every other known mass murder database to ensure reliability and validity. Incidents listed in the SHR that could not be independently verified were excluded from the database.

Project researchers also conducted extensive searches for incidents not reported in the SHR during the time period, utilizing internet search engines, Lexis-Nexis, and Newspapers.com. Search terms include: [number] dead, [number] killed, [number] slain, [number] murdered, [number] homicide, mass murder, mass shooting, massacre, rampage, family killing, familicide, and arson murder. Offender, victim, and location names were also directly searched when available.

This project started at USA TODAY in 2012.

Contacts

Contact AP Data Editor Justin Myers with questions, suggestions or comments about this dataset at jmyers@ap.org. The Northeastern University researcher working with AP and USA TODAY is Professor James Alan Fox, who can be reached at j.fox@northeastern.edu or 617-416-4400.

Investigator(s): Federal Bureau of Investigation Since 1930, the Federal Bureau of Investigation (FBI) has compiled the Uniform Crime Reports (UCR) to serve as periodic nationwide assessments of reported crimes not available elsewhere in the criminal justice system. With the 1977 data, the title was expanded to Uniform Crime Reporting Program Data. Each year, participating law enforcement agencies contribute reports to the FBI either directly or through their state reporting programs. ICPSR archives the UCR data as five separate components: (1) summary data, (2) county-level data, (3) incident-level data (National Incident-Based Reporting System [NIBRS]), (4) hate crime data, and (5) various, mostly nonrecurring, data collections. Summary data are reported in four types of files: (a) Offenses Known and Clearances by Arrest, (b) Property Stolen and Recovered, (c) Supplementary Homicide Reports (SHR), and (d) Police Employee (LEOKA) Data (Law Enforcement Officers Killed or Assaulted). The county-level data provide counts of arrests and offenses aggregated to the county level. County populations are also reported. In the late 1970s, new ways to look at crime were studied. The UCR program was subsequently expanded to capture incident-level data with the implementation of the National Incident-Based Reporting System. The NIBRS data focus on various aspects of a crime incident. The gathering of hate crime data by the UCR program was begun in 1990. Hate crimes are defined as crimes that manifest evidence of prejudice based on race, religion, sexual orientation, or ethnicity. In September 1994, disabilities, both physical and mental, were added to the list. The fifth component of ICPSR's UCR holdings is comprised of various collections, many of which are nonrecurring and prepared by individual researchers. These collections go beyond the scope of the standard UCR collections provided by the FBI, either by including data for a range of years or by focusing on other aspects of analysis. NACJD has produced resource guides on UCR and on NIBRS data.

For any questions about this data please email me at jacob@crimedatatool.com. If you use this data, please cite it.

Version 5 release notes:
Adds data in the following formats: SPSS, SAS, and Excel.Changes project name to avoid confusing this data for the ones done by NACJD.Adds data for 1991.Fixes bug where bias motivation "anti-lesbian, gay, bisexual, or transgender, mixed group (lgbt)" was labeled "anti-homosexual (gay and lesbian)" prior to 2013 causing there to be two columns and zero values for years with the wrong label.All data is now directly from the FBI, not NACJD. The data initially comes as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. For the R code used to clean this data, see here. https://github.com/jacobkap/crime_data. Version 4 release notes:
Adds data for 2017.Adds rows that submitted a zero-report (i.e. that agency reported no hate crimes in the year). This is for all years 1992-2017. Made changes to categorical variables (e.g. bias motivation columns) to make categories consistent over time. Different years had slightly different names (e.g. 'anti-am indian' and 'anti-american indian') which I made consistent.
Made the 'population' column which is the total population in that agency.

Version 3 release notes:
Adds data for 2016.Order rows by year (descending) and ORI.Version 2 release notes:
Fix bug where Philadelphia Police Department had incorrect FIPS county code. The Hate Crime data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains information about hate crimes reported in the United States. Please note that the files are quite large and may take some time to open.

Each row indicates a hate crime incident for an agency in a given year. I have made a unique ID column ("unique_id") by combining the year, agency ORI9 (the 9 character Originating Identifier code), and incident number columns together. Each column is a variable related to that incident or to the reporting agency.
Some of the important columns are the incident date, what crime occurred (up to 10 crimes), the number of victims for each of these crimes, the bias motivation for each of these crimes, and the location of each crime. It also includes the total number of victims, total number of offenders, and race of offenders (as a group). Finally, it has a number of columns indicating if the victim for each offense was a certain type of victim or not (e.g. individual victim, business victim religious victim, etc.).

The only changes I made to the data are the following. Minor changes to column names to make all column names 32 characters or fewer (so it can be saved in a Stata format), changed the name of some UCR offense codes (e.g. from "agg asslt" to "aggravated assault"), made all character values lower case, reordered columns. I also added state, county, and place FIPS code from the LEAIC (crosswalk) and generated incident month, weekday, and month-day variables from the incident date variable included in the original data.

The UNIFORM CRIME REPORTING PROGRAM DATA: OFFENSES KNOWN AND CLEARANCES BY ARREST, 2012 dataset is a compilation of offenses reported to law enforcement agencies in the United States. Due to the vast number of categories of crime committed in the United States, the FBI has limited the type of crimes included in this compilation to those crimes which people are most likely to report to police and those crimes which occur frequently enough to be analyzed across time. Crimes included are criminal homicide, forcible rape, robbery, aggravated assault, burglary, larceny-theft, and motor vehicle theft. Much information about these crimes is provided in this dataset. The number of times an offense has been reported, the number of reported offenses that have been cleared by arrests, and the number of cleared offenses which involved offenders under the age of 18 are the major items of information collected.

Effective March 7, 2024, the Los Angeles Police Department (LAPD) implemented a new Records Management System aligning with the FBI's National Incident-Based Reporting System (NIBRS) requirements. This switch, part of a nationwide mandate, enhances the granularity and specificity of crime data. You can learn more about NIBRS on the FBI's website here: https://www.fbi.gov/how-we-can-help-you/more-fbi-services-and-information/ucr/nibrs NIBRS is more comprehensive than the previous Summary Reporting System (SRS) used in the Uniform Crime Reporting (UCR) program. Unlike SRS, which grouped crimes into general categories, NIBRS collects detailed information for each incident, including multiple offenses, offenders, and victims when applicable. This detail-rich format may give the impression of increased crime levels due to its broader capture of criminal activity, but it actually provides a more accurate and nuanced view of crime in our community. This change sets a new baseline for crime reporting, reflecting incidents in the City of Los Angeles starting from March 7, 2024. NIBRS collects detailed information about each victim per incident, including victim- demographics information and specific crime details, providing more insight into affected individuals within each reported crime.

Version 5 release notes:

• Removes support for SPSS and Excel data.
• Changes the crimes that are stored in each file. There are more files now with fewer crimes per file. The files and their included crimes have been updated below.
  
• Adds in agencies that report 0 months of the year.
• Adds a column that indicates the number of months reported. This is generated summing up the number of unique months an agency reports data for. Note that this indicates the number of months an agency reported arrests for ANY crime. They may not necessarily report every crime every month. Agencies that did not report a crime with have a value of NA for every arrest column for that crime.
• Removes data on runaways.
  

Version 4 release notes:

• Changes column names from "poss_coke" and "sale_coke" to "poss_heroin_coke" and "sale_heroin_coke" to clearly indicate that these column includes the sale of heroin as well as similar opiates such as morphine, codeine, and opium. Also changes column names for the narcotic columns to indicate that they are only for synthetic narcotics.
  

Version 3 release notes:

• Add data for 2016.
• Order rows by year (descending) and ORI.

Version 2 release notes:

• Fix bug where Philadelphia Police Department had incorrect FIPS county code.

The Arrests by Age, Sex, and Race data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains highly granular data on the number of people arrested for a variety of crimes (see below for a full list of included crimes). The data sets here combine data from the years 1980-2015 into a single file. These files are quite large and may take some time to load.

All the data was downloaded from NACJD as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. For the R code used to clean this data, see here. https://github.com/jacobkap/crime_data. If you have any questions, comments, or suggestions please contact me at jkkaplan6@gmail.com.

I did not make any changes to the data other than the following. When an arrest column has a value of "None/not reported", I change that value to zero. This makes the (possible incorrect) assumption that these values represent zero crimes reported. The original data does not have a value when the agency reports zero arrests other than "None/not reported." In other words, this data does not differentiate between real zeros and missing values. Some agencies also incorrectly report the following numbers of arrests which I change to NA: 10000, 20000, 30000, 40000, 50000, 60000, 70000, 80000, 90000, 100000, 99999, 99998.

To reduce file size and make the data more manageable, all of the data is aggregated yearly. All of the data is in agency-year units such that every row indicates an agency in a given year. Columns are crime-arrest category units. For example, If you choose the data set that includes murder, you would have rows for each agency-year and columns with the number of people arrests for murder. The ASR data breaks down arrests by age and gender (e.g. Male aged 15, Male aged 18). They also provide the number of adults or juveniles arrested by race. Because most agencies and years do not report the arrestee's ethnicity (Hispanic or not Hispanic) or juvenile outcomes (e.g. referred to adult court, referred to welfare agency), I do not include these columns.

To make it easier to merge with other data, I merged this data with the Law Enforcement Agency Identifiers Crosswalk (LEAIC) data. The data from the LEAIC add FIPS (state, county, and place) and agency type/subtype. Please note that some of the FIPS codes have leading zeros and if you open it in Excel it will automatically delete those leading zeros.

I created 9 arrest categories myself. The categories are:

• Total Male Juvenile
• Total Female Juvenile
• Total Male Adult
• Total Female Adult
• Total Ma

All BPD data on Open Baltimore is preliminary data and subject to change. The information presented through Open Baltimore represents Part I victim based crime data. The data do not represent statistics submitted to the FBI's Uniform Crime Report (UCR); therefore any comparisons are strictly prohibited. For further clarification of UCR data, please visit http://www.fbi.gov/about-us/cjis/ucr/ucr. Please note that this data is preliminary and subject to change. Prior month data is likely to show changes when it is refreshed on a monthly basis. All data is geocoded to the approximate latitude/longitude location of the incident and excludes those records for which an address could not be geocoded. Any attempt to match the approximate location of the incident to an exact address is strictly prohibited.

Acknowledgements

This dataset was kindly made available by the City of Baltimore. You can find the original dataset, which is updated regularly, here.

Version 3 release notes: Adds data for 2016.Order rows by year (descending) and ORI.Version 2 release notes: Fix bug where Philadelphia Police Department had incorrect FIPS county code. The Hate Crime data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains information about hate crimes reported in the United States. The data sets here combine all data from the years 1992-2015 into a single file. Please note that the files are quite large and may take some time to open.Each row indicates a hate crime incident for an agency in a given year. I have made a unique ID column ("unique_id") by combining the year, agency ORI9 (the 9 character Originating Identifier code), and incident number columns together. Each column is a variable related to that incident or to the reporting agency. Some of the important columns are the incident date, what crime occurred (up to 10 crimes), the number of victims for each of these crimes, the bias motivation for each of these crimes, and the location of each crime. It also includes the total number of victims, total number of offenders, and race of offenders (as a group). Finally, it has a number of columns indicating if the victim for each offense was a certain type of victim or not (e.g. individual victim, business victim religious victim, etc.). All the data was downloaded from NACJD as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. For the R code used to clean this data, see here. https://github.com/jacobkap/crime_data. The only changes I made to the data are the following. Minor changes to column names to make all column names 32 characters or fewer (so it can be saved in a Stata format), changed the name of some UCR offense codes (e.g. from "agg asslt" to "aggravated assault"), made all character values lower case, reordered columns. I also added state, county, and place FIPS code from the LEAIC (crosswalk) and generated incident month, weekday, and month-day variables from the incident date variable included in the original data. The zip file contains the data in the following formats and a codebook: .csv - Microsoft Excel.dta - Stata.sav - SPSS.rda - RIf you have any questions, comments, or suggestions please contact me at jkkaplan6@gmail.com.

The National Incident-Based Reporting System (NIBRS) is a part of the Uniform Crime Reporting Program (UCR), administered by the Federal Bureau of Investigation (FBI). In the late 1970s, the law enforcement community called for a thorough evaluative study of the UCR with the objective of recommending an expanded and enhanced UCR program to meet law enforcement needs into the 21st century. The FBI fully concurred with the need for an updated program to meet contemporary needs and provided its support, formulating a comprehensive redesign effort. Following a multiyear study, a "Blueprint for the Future of the Uniform Crime Reporting Program" was developed. Using the "Blueprint" and in consultation with local and state law enforcement executives, the FBI formulated new guidelines for the Uniform Crime Reports. The National Incident-Based Reporting System (NIBRS) is being implemented to meet these guidelines. NIBRS data are archived at ICPSR as 13 separate data files, which may be merged by using linkage variables. The data focus on a variety of aspects of a crime incident. Part 4, Administrative Segment, offers data on the incident itself (date and time). Each crime incident is delineated by one administrative segment record. Also provided are Part 5, Offense Segment (offense type, location, weapon use, and bias motivation), Part 6, Property Segment (type of property loss, property description, property value, drug type and quantity), Part 7, Victim Segment (age, sex, race, ethnicity, and injuries), Part 8, Offender Segment (age, sex, and race), and Part 9, Arrestee Segment (arrest date, age, sex, race, and weapon use). The Batch Header Segment (Parts 1-3) separates and identifies individual police agencies by Originating Agency Identifier (ORI). Batch Header information, which is contained on three records for each ORI, includes agency name, geographic location, and population of the area. Part 10, Group B Arrest Report Segment, includes arrestee data for Group B crimes. Window Segments files (Parts 11-13) pertain to incidents for which the complete Group A Incident Report was not submitted to the FBI. In general, a Window Segment record will be generated if the incident occurred prior to January 1 of the previous year or if the incident occurred prior to when the agency started NIBRS reporting. As with UCR, participation in NIBRS is voluntary on the part of law enforcement agencies. The data are not a representative sample of crime in the United States. For 1999, 18 states, fully or partially participating in NIBRS, were included in the dataset.

The National Incident-Based Reporting System (NIBRS) is a part of the Uniform Crime Reporting Program (UCR), administered by the Federal Bureau of Investigation (FBI). In the late 1970s, the law enforcement community called for a thorough evaluative study of the UCR with the objective of recommending an expanded and enhanced UCR program to meet law enforcement needs into the 21st century. The FBI fully concurred with the need for an updated program to meet contemporary needs and provided its support, formulating a comprehensive redesign effort. Following a multiyear study, a "Blueprint for the Future of the Uniform Crime Reporting Program" was developed. Using the "Blueprint," and in consultation with local and state law enforcement executives, the FBI formulated new guidelines for the Uniform Crime Reports. The National Incident-Based Reporting System (NIBRS) was implemented to meet these guidelines. NIBRS data as formatted by the FBI are stored in a single file. These data are organized by various segment levels (record types). There are six main segment levels: administrative, offense, property, victim, offender, and arrestee. Each segment level has a different length and layout. There are other segment levels which occur with less frequency than the six main levels. Significant computing resources are necessary to work with the data in its single-file format. In addition, the user must be sophisticated in working with data in complex file types. While it is convenient to think of NIBRS as a hierarchical file, its structure is more similar to a relational database in that there are key variables that link the different segment levels together. NIBRS data are archived at ICPSR as 11 separate data files per year, which may be merged by using linkage variables. Prior to 2013 the data were archived and distributed as 13 separate data files, including three separate batch header record files. Starting with the 2013 data, the FBI combined the three batch header files into one file. Consequently, ICPSR instituted new file numbering for the data. NIBRS data focus on a variety of aspects of a crime incident. Part 2 (formerly Part 4), Administrative Segment, offers data on the incident itself (date and time). Each crime incident is delineated by one administrative segment record. Also provided are Part 3 (formerly Part 5), Offense Segment (offense type, location, weapon use, and bias motivation), Part 4 (formerly Part 6), Property Segment (type of property loss, property description, property value, drug type and quantity), Part 5 (formerly Part 7), Victim Segment (age, sex, race, ethnicity, and injuries), Part 6 (formerly Part 8), Offender Segment (age, sex, and race), and Part 7 (formerly Part 9), Arrestee Segment (arrest date, age, sex, race, and weapon use). The Batch Header Segment (Part 1, formerly Parts 1-3) separates and identifies individual police agencies by Originating Agency Identifier (ORI). Batch Header information, which is contained on three records for each ORI, includes agency name, geographic location, and population of the area. Part 8 (formerly Part 10), Group B Arrest Report Segment, includes arrestee data for Group B crimes. Window Segments files (Parts 9-11, formerly Parts 11-13) pertain to incidents for which the complete Group A Incident Report was not submitted to the FBI. In general, a Window Segment record will be generated if the incident occurred prior to January 1 of the previous year or if the incident occurred prior to when the agency started NIBRS reporting. As with the UCR, participation in NIBRS is voluntary on the part of law enforcement agencies. The data are not a representative sample of crime in the United States. Recognizing many differences in computing resources and that many users will be interested in only one or two segment levels, ICPSR has decided to make the data available as multiple files. Each NIBRS segment level in the FBI's single-file format has been made into a separate rectangular ASCII data file. Linkage (key) variables are used to perform analyses that involve two or more segment levels. If the user is interested in variables contained in one segment level, then the data are easy to work with since each segment level file is simply a rectangular ASCII data file. Setup files are available to read each segment level. Also, with only one segment level, the issue of

This dataset contains aggregate data on violent index victimizations at the quarter level of each year (i.e., January – March, April – June, July – September, October – December), from 2001 to the present (1991 to present for Homicides), with a focus on those related to gun violence. Index crimes are 10 crime types selected by the FBI (codes 1-4) for special focus due to their seriousness and frequency. This dataset includes only those index crimes that involve bodily harm or the threat of bodily harm and are reported to the Chicago Police Department (CPD). Each row is aggregated up to victimization type, age group, sex, race, and whether the victimization was domestic-related. Aggregating at the quarter level provides large enough blocks of incidents to protect anonymity while allowing the end user to observe inter-year and intra-year variation. Any row where there were fewer than three incidents during a given quarter has been deleted to help prevent re-identification of victims. For example, if there were three domestic criminal sexual assaults during January to March 2020, all victims associated with those incidents have been removed from this dataset. Human trafficking victimizations have been aggregated separately due to the extremely small number of victimizations.

This dataset includes a " GUNSHOT_INJURY_I " column to indicate whether the victimization involved a shooting, showing either Yes ("Y"), No ("N"), or Unknown ("UKNOWN.") For homicides, injury descriptions are available dating back to 1991, so the "shooting" column will read either "Y" or "N" to indicate whether the homicide was a fatal shooting or not. For non-fatal shootings, data is only available as of 2010. As a result, for any non-fatal shootings that occurred from 2010 to the present, the shooting column will read as “Y.” Non-fatal shooting victims will not be included in this dataset prior to 2010; they will be included in the authorized dataset, but with "UNKNOWN" in the shooting column.

The dataset is refreshed daily, but excludes the most recent complete day to allow CPD time to gather the best available information. Each time the dataset is refreshed, records can change as CPD learns more about each victimization, especially those victimizations that are most recent. The data on the Mayor's Office Violence Reduction Dashboard is updated daily with an approximately 48-hour lag. As cases are passed from the initial reporting officer to the investigating detectives, some recorded data about incidents and victimizations may change once additional information arises. Regularly updated datasets on the City's public portal may change to reflect new or corrected information.

How does this dataset classify victims?

The methodology by which this dataset classifies victims of violent crime differs by victimization type:

Homicide and non-fatal shooting victims: A victimization is considered a homicide victimization or non-fatal shooting victimization depending on its presence in CPD's homicide victims data table or its shooting victims data table. A victimization is considered a homicide only if it is present in CPD's homicide data table, while a victimization is considered a non-fatal shooting only if it is present in CPD's shooting data tables and absent from CPD's homicide data table.

To determine the IUCR code of homicide and non-fatal shooting victimizations, we defer to the incident IUCR code available in CPD's Crimes, 2001-present dataset (available on the City's open data portal). If the IUCR code in CPD's Crimes dataset is inconsistent with the homicide/non-fatal shooting categorization, we defer to CPD's Victims dataset.

For a criminal homicide, the only sensible IUCR codes are 0110 (first-degree murder) or 0130 (second-degree murder). For a non-fatal shooting, a sensible IUCR code must signify a criminal sexual assault, a robbery, or, most commonly, an aggravated battery. In rare instances, the IUCR code in CPD's Crimes and Victims dataset do not align with the homicide/non-fatal shooting categorization:

1. In instances where a homicide victimization does not correspond to an IUCR code 0110 or 0130, we set the IUCR code to "01XX" to indicate that the victimization was a homicide but we do not know whether it was a first-degree murder (IUCR code = 0110) or a second-degree murder (IUCR code = 0130).
2. When a non-fatal shooting victimization does not correspond to an IUCR code that signifies a criminal sexual assault, robbery, or aggravated battery, we enter “UNK” in the IUCR column, “YES” in the GUNSHOT_I column, and “NON-FATAL” in the PRIMARY column to indicate that the victim was non-fatally shot, but the precise IUCR code is unknown.

Other violent crime victims: For other violent crime types, we refer to the IUCR classification that exists in CPD's victim table, with only one exception:

1. When there is an incident that is associated with no victim with a matching IUCR code, we assume that this is an error. Every crime should have at least 1 victim with a matching IUCR code. In these cases, we change the IUCR code to reflect the incident IUCR code because CPD's incident table is considered to be more reliable than the victim table.

Note: All businesses identified as victims in CPD data have been removed from this dataset.

Note: The definition of “homicide” (shooting or otherwise) does not include justifiable homicide or involuntary manslaughter. This dataset also excludes any cases that CPD considers to be “unfounded” or “noncriminal.”

Note: In some instances, the police department's raw incident-level data and victim-level data that were inputs into this dataset do not align on the type of crime that occurred. In those instances, this dataset attempts to correct mismatches between incident and victim specific crime types. When it is not possible to determine which victims are associated with the most recent crime determination, the dataset will show empty cells in the respective demographic fields (age, sex, race, etc.).

Note: The initial reporting officer usually asks victims to report demographic data. If victims are unable to recall, the reporting officer will use their best judgment. “Unknown” can be reported if it is truly unknown.

The National Incident-Based Reporting System (NIBRS) is a part of the Uniform Crime Reporting Program (UCR), administered by the Federal Bureau of Investigation (FBI). In the late 1970s, the law enforcement community called for a thorough evaluative study of the UCR with the objective of recommending an expanded and enhanced UCR program to meet law enforcement needs into the 21st century. The FBI fully concurred with the need for an updated program to meet contemporary needs and provided its support, formulating a comprehensive redesign effort. Following a multiyear study, a "Blueprint for the Future of the Uniform Crime Reporting Program" was developed. Using the "Blueprint" and in consultation with local and state law enforcement executives, the FBI formulated new guidelines for the Uniform Crime Reports. The National Incident-Based Reporting System (NIBRS) is being implemented to meet these guidelines. NIBRS data are archived at ICPSR as 13 separate data files, which may be merged by using linkage variables. The data focus on a variety of aspects of a crime incident. Part 4, Administrative Segment, offers data on the incident itself (date and time). Each crime incident is delineated by one administrative segment record. Also provided are Part 5, Offense Segment (offense type, location, weapon use, and bias motivation), Part 6, Property Segment (type of property loss, property description, property value, drug type and quantity), Part 7, Victim Segment (age, sex, race, ethnicity, and injuries), Part 8, Offender Segment (age, sex, and race), and Part 9, Arrestee Segment (arrest date, age, sex, race, and weapon use). The Batch Header Segment (Parts 1-3) separates and identifies individual police agencies by Originating Agency Identifier (ORI). Batch Header information, which is contained on three records for each ORI, includes agency name, geographic location, and population of the area. Part 10, Group B Arrest Report Segment, includes arrestee data for Group B crimes. Window Segments files (Parts 11-13) pertain to incidents for which the complete Group A Incident Report was not submitted to the FBI. In general, a Window Segment record will be generated if the incident occurred prior to January 1 of the previous year or if the incident occurred prior to when the agency started NIBRS reporting. As with UCR, participation in NIBRS is voluntary on the part of law enforcement agencies. The data are not a representative sample of crime in the United States. For 2000, 18 states, fully or partially participating in NIBRS, were included in the dataset.