!!!WARNING~~~This dataset has a large number of flaws and is unable to properly answer many questions that people generally use it to answer, such as whether national hate crimes are changing (or at least they use the data so improperly that they get the wrong answer). A large number of people using this data (academics, advocates, reporting, US Congress) do so inappropriately and get the wrong answer to their questions as a result. Indeed, many published papers using this data should be retracted. Before using this data I highly recommend that you thoroughly read my book on UCR data, particularly the chapter on hate crimes (https://ucrbook.com/hate-crimes.html) as well as the FBI's own manual on this data. The questions you could potentially answer well are relatively narrow and generally exclude any causal relationships. ~~~WARNING!!!For a comprehensive guide to this data and other UCR data, please see my book at ucrbook.comVersion 10 release notes:Adds 2022 dataVersion 9 release notes:Adds 2021 data.Version 8 release notes:Adds 2019 and 2020 data. Please note that the FBI has retired UCR data ending in 2020 data so this will be the last UCR hate crime data they release. Changes .rda file to .rds.Version 7 release notes:Changes release notes description, does not change data.Version 6 release notes:Adds 2018 dataVersion 5 release notes:Adds data in the following formats: SPSS, SAS, and Excel.Changes project name to avoid confusing this data for the ones done by NACJD.Adds data for 1991.Fixes bug where bias motivation "anti-lesbian, gay, bisexual, or transgender, mixed group (lgbt)" was labeled "anti-homosexual (gay and lesbian)" prior to 2013 causing there to be two columns and zero values for years with the wrong label.All data is now directly from the FBI, not NACJD. The data initially comes as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. Version 4 release notes: Adds data for 2017.Adds rows that submitted a zero-report (i.e. that agency reported no hate crimes in the year). This is for all years 1992-2017. Made changes to categorical variables (e.g. bias motivation columns) to make categories consistent over time. Different years had slightly different names (e.g. 'anti-am indian' and 'anti-american indian') which I made consistent. Made the 'population' column which is the total population in that agency. Version 3 release notes: Adds data for 2016.Order rows by year (descending) and ORI.Version 2 release notes: Fix bug where Philadelphia Police Department had incorrect FIPS county code. The Hate Crime data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains information about hate crimes reported in the United States. Please note that the files are quite large and may take some time to open.Each row indicates a hate crime incident for an agency in a given year. I have made a unique ID column ("unique_id") by combining the year, agency ORI9 (the 9 character Originating Identifier code), and incident number columns together. Each column is a variable related to that incident or to the reporting agency. Some of the important columns are the incident date, what crime occurred (up to 10 crimes), the number of victims for each of these crimes, the bias motivation for each of these crimes, and the location of each crime. It also includes the total number of victims, total number of offenders, and race of offenders (as a group). Finally, it has a number of columns indicating if the victim for each offense was a certain type of victim or not (e.g. individual victim, business victim religious victim, etc.). The only changes I made to the data are the following. Minor changes to column names to make all column names 32 characters or fewer (so it can be saved in a Stata format), made all character values lower case, reordered columns. I also generated incident month, weekday, and month-day variables from the incident date variable included in the original data.

Effective March 7, 2024, the Los Angeles Police Department (LAPD) implemented a new Records Management System aligning with the FBI's National Incident-Based Reporting System (NIBRS) requirements. This switch, part of a nationwide mandate, enhances the granularity and specificity of crime data. You can learn more about NIBRS on the FBI's website here: https://www.fbi.gov/how-we-can-help-you/more-fbi-services-and-information/ucr/nibrs NIBRS is more comprehensive than the previous Summary Reporting System (SRS) used in the Uniform Crime Reporting (UCR) program. Unlike SRS, which grouped crimes into general categories, NIBRS collects detailed information for each incident, including multiple offenses, offenders, and victims when applicable. This detail-rich format may give the impression of increased crime levels due to its broader capture of criminal activity, but it actually provides a more accurate and nuanced view of crime in our community. This change sets a new baseline for crime reporting, reflecting incidents in the City of Los Angeles starting from March 7, 2024. NIBRS collects detailed information about each victim per incident, including victim- demographics information and specific crime details, providing more insight into affected individuals within each reported crime.

Alaska crime data from 2000 to present from the FBI Uniform Crime Reporting (UCR) program. Information includes data on both violent and property crime.The UCR Program's primary objective is to generate reliable information for use in law enforcement administration, operation, and management; over the years, however, the data have become one of the country’s leading social indicators. The program has been the starting place for law enforcement executives, students of criminal justice, researchers, members of the media, and the public at large seeking information on crime in the nation. The program was conceived in 1929 by the International Association of Chiefs of Police to meet the need for reliable uniform crime statistics for the nation. In 1930, the FBI was tasked with collecting, publishing, and archiving those statistics.Source: US Federal Bureau of Investigation (FBI)This data has been visualized in a Geographic Information Systems (GIS) format and is provided as a service in the DCRA Information Portal by the Alaska Department of Commerce, Community, and Economic Development Division of Community and Regional Affairs (SOA DCCED DCRA), Research and Analysis section. SOA DCCED DCRA Research and Analysis is not the authoritative source for this data. For more information and for questions about this data, see: FBI UCR ProgramOffenses Known to Law Enforcement, by State by City, 2017 The FBI collects these data through the Uniform Crime Reporting (UCR) Program. Important note about rape data In 2013, the FBI’s UCR Program initiated the collection of rape data under a revised definition within the Summary Based Reporting System. The term “forcible” was removed from the offense name, and the definition was changed to “penetration, no matter how slight, of the vagina or anus with any body part or object, or oral penetration by a sex organ of another person, without the consent of the victim.” In 2016, the FBI Director approved the recommendation to discontinue the reporting of rape data using the UCR legacy definition beginning in 2017. General comment This table provides the volume of violent crime (murder and nonnegligent manslaughter, rape, robbery, and aggravated assault) and property crime (burglary, larceny-theft, and motor vehicle theft) as reported by city and town law enforcement agencies (listed alphabetically by state) that contributed data to the UCR Program. (Note: Arson is not included in the property crime total in this table; however, if complete arson data were provided, it will appear in the arson column.) Caution against ranking Readers should take into consideration relevant factors in addition to an area’s crime statistics when making any valid comparisons of crime among different locales. UCR Statistics: Their Proper Use provides more details. Methodology The data used in creating this table were from all city and town law enforcement agencies submitting 12 months of complete offense data for 2017. Rape figures, and violent crime, which rape is a part, will not be published in this table for agencies submitting rape using the UCR legacy rape definition. The rape figures, and violent crime, which rape is a part, published in this table are from only those agencies using the UCR revised rape definition as well as converted data from agencies that reported data for rape, sodomy, and sexual assault with an object via NIBRS. The FBI does not publish arson data unless it receives data from either the agency or the state for all 12 months of the calendar year. When the FBI determines that an agency’s data collection methodology does not comply with national UCR guidelines, the figure(s) for that agency’s offense(s) will not be included in the table, and the discrepancy will be explained in a footnote. Population estimation For the 2017 population estimates used in this table, the FBI computed individual rates of growth from one year to the next for every city/town and county using 2010 decennial population counts and 2011 through 2016 population estimates from the U.S. Census Bureau. Each agency’s rates of growth were averaged; that average was then applied and added to its 2016 Census population estimate to derive the agency’s 2017 population estimate.

Context

The Uniform Crime Reporting (UCR) Program has been the starting place for law enforcement executives, students of criminal justice, researchers, members of the media, and the public at large seeking information on crime in the nation. The program was conceived in 1929 by the International Association of Chiefs of Police to meet the need for reliable uniform crime statistics for the nation. In 1930, the FBI was tasked with collecting, publishing, and archiving those statistics.

Today, four annual publications, Crime in the United States, National Incident-Based Reporting System, Law Enforcement Officers Killed and Assaulted, and Hate Crime Statistics are produced from data received from over 18,000 city, university/college, county, state, tribal, and federal law enforcement agencies voluntarily participating in the program. The crime data are submitted either through a state UCR Program or directly to the FBI’s UCR Program.

This dataset focuses on the crime rates and law enforcement employment data in the state of California.

Content

Crime and law enforcement employment rates are separated into individual files, focusing on offenses by enforcement agency, college/university campus, county, and city. Categories of crimes reported include violent crime, murder and nonnegligent manslaughter, rape, robbery, aggravated assault, property crime, burglary, larceny-theft, motor vehicle damage, and arson. In the case of rape, data is collected for both revised and legacy definitions. In some cases, a small number of enforcement agencies switched definition collection sometime within the same year.

Acknowledgements

This dataset originates from the FBI UCR project, and the complete dataset for all 2015 crime reports can be found here.

Inspiration

• What are the most common types of crimes in California? Are there certain crimes that are more common in a particular place category, such as a college/university campus, compared to the rest of the state?
• How does the number of law enforcement officers compare to the crime rates of a particular area? Is the ratio similar throughout the state, or do certain campuses, counties, or cities have a differing rate?
• How does the legacy vs. refined definition of rape differ, and how do the rape counts compare? If you pulled the same data from FBI datasets for previous years, can you see a difference in rape rates over time?

The UNIFORM CRIME REPORTING PROGRAM DATA: OFFENSES KNOWN AND CLEARANCES BY ARREST, 2010 dataset is a compilation of offenses reported to law enforcement agencies in the United States. Due to the vast number of categories of crime committed in the United States, the FBI has limited the type of crimes included in this compilation to those crimes which people are most likely to report to police and those crimes which occur frequently enough to be analyzed across time. Crimes included are criminal homicide, forcible rape, robbery, aggravated assault, burglary, larceny-theft, and motor vehicle theft. Much information about these crimes is provided in this dataset. The number of times an offense has been reported, the number of reported offenses that have been cleared by arrests, and the number of cleared offenses which involved offenders under the age of 18 are the major items of information collected.

THIS DATASET WAS LAST UPDATED AT 2:11 AM EASTERN ON JULY 12

OVERVIEW

2019 had the most mass killings since at least the 1970s, according to the Associated Press/USA TODAY/Northeastern University Mass Killings Database.

In all, there were 45 mass killings, defined as when four or more people are killed excluding the perpetrator. Of those, 33 were mass shootings . This summer was especially violent, with three high-profile public mass shootings occurring in the span of just four weeks, leaving 38 killed and 66 injured.

A total of 229 people died in mass killings in 2019.

The AP's analysis found that more than 50% of the incidents were family annihilations, which is similar to prior years. Although they are far less common, the 9 public mass shootings during the year were the most deadly type of mass murder, resulting in 73 people's deaths, not including the assailants.

One-third of the offenders died at the scene of the killing or soon after, half from suicides.

About this Dataset

The Associated Press/USA TODAY/Northeastern University Mass Killings database tracks all U.S. homicides since 2006 involving four or more people killed (not including the offender) over a short period of time (24 hours) regardless of weapon, location, victim-offender relationship or motive. The database includes information on these and other characteristics concerning the incidents, offenders, and victims.

The AP/USA TODAY/Northeastern database represents the most complete tracking of mass murders by the above definition currently available. Other efforts, such as the Gun Violence Archive or Everytown for Gun Safety may include events that do not meet our criteria, but a review of these sites and others indicates that this database contains every event that matches the definition, including some not tracked by other organizations.

This data will be updated periodically and can be used as an ongoing resource to help cover these events.

Using this Dataset

To get basic counts of incidents of mass killings and mass shootings by year nationwide, use these queries:

Mass killings by year

Mass shootings by year

To get these counts just for your state:

Filter killings by state

Definition of "mass murder"

Mass murder is defined as the intentional killing of four or more victims by any means within a 24-hour period, excluding the deaths of unborn children and the offender(s). The standard of four or more dead was initially set by the FBI.

This definition does not exclude cases based on method (e.g., shootings only), type or motivation (e.g., public only), victim-offender relationship (e.g., strangers only), or number of locations (e.g., one). The time frame of 24 hours was chosen to eliminate conflation with spree killers, who kill multiple victims in quick succession in different locations or incidents, and to satisfy the traditional requirement of occurring in a “single incident.”

Offenders who commit mass murder during a spree (before or after committing additional homicides) are included in the database, and all victims within seven days of the mass murder are included in the victim count. Negligent homicides related to driving under the influence or accidental fires are excluded due to the lack of offender intent. Only incidents occurring within the 50 states and Washington D.C. are considered.

Methodology

Project researchers first identified potential incidents using the Federal Bureau of Investigation’s Supplementary Homicide Reports (SHR). Homicide incidents in the SHR were flagged as potential mass murder cases if four or more victims were reported on the same record, and the type of death was murder or non-negligent manslaughter.

Cases were subsequently verified utilizing media accounts, court documents, academic journal articles, books, and local law enforcement records obtained through Freedom of Information Act (FOIA) requests. Each data point was corroborated by multiple sources, which were compiled into a single document to assess the quality of information.

In case(s) of contradiction among sources, official law enforcement or court records were used, when available, followed by the most recent media or academic source.

Case information was subsequently compared with every other known mass murder database to ensure reliability and validity. Incidents listed in the SHR that could not be independently verified were excluded from the database.

Project researchers also conducted extensive searches for incidents not reported in the SHR during the time period, utilizing internet search engines, Lexis-Nexis, and Newspapers.com. Search terms include: [number] dead, [number] killed, [number] slain, [number] murdered, [number] homicide, mass murder, mass shooting, massacre, rampage, family killing, familicide, and arson murder. Offender, victim, and location names were also directly searched when available.

This project started at USA TODAY in 2012.

Contacts

Contact AP Data Editor Justin Myers with questions, suggestions or comments about this dataset at jmyers@ap.org. The Northeastern University researcher working with AP and USA TODAY is Professor James Alan Fox, who can be reached at j.fox@northeastern.edu or 617-416-4400.

Context

Introduction: The dataset used for this experiment is real and authentic. The dataset is acquired from UCI machine learning repository website [13]. The title of the dataset is ‘Crime and Communities’. It is prepared using real data from socio-economic data from 1990 US Census, law enforcement data from the 1990 US LEMAS survey, and crimedata from the 1995 FBI UCR [13]. This dataset contains a total number of 147 attributes and 2216 instances.

The per capita crimes variables were calculated using population values included in the 1995 FBI data (which differ from the 1990 Census values).

Content

The variables included in the dataset involve the community, such as the percent of the population considered urban, and the median family income, and involving law enforcement, such as per capita number of police officers, and percent of officers assigned to drug units. The crime attributes (N=18) that could be predicted are the 8 crimes considered 'Index Crimes' by the FBI)(Murders, Rape, Robbery, .... ), per capita (actually per 100,000 population) versions of each, and Per Capita Violent Crimes and Per Capita Nonviolent Crimes)

predictive variables : 125 non-predictive variables : 4 potential goal/response variables : 18

Acknowledgements

http://archive.ics.uci.edu/ml/datasets/Communities%20and%20Crime%20Unnormalized

U. S. Department of Commerce, Bureau of the Census, Census Of Population And Housing 1990 United States: Summary Tape File 1a & 3a (Computer Files),

U.S. Department Of Commerce, Bureau Of The Census Producer, Washington, DC and Inter-university Consortium for Political and Social Research Ann Arbor, Michigan. (1992)

U.S. Department of Justice, Bureau of Justice Statistics, Law Enforcement Management And Administrative Statistics (Computer File) U.S. Department Of Commerce, Bureau Of The Census Producer, Washington, DC and Inter-university Consortium for Political and Social Research Ann Arbor, Michigan. (1992)

U.S. Department of Justice, Federal Bureau of Investigation, Crime in the United States (Computer File) (1995)

Inspiration

Your data will be in front of the world's largest data science community. What questions do you want to see answered?

Data available in the dataset may not act as a complete source of information for identifying factors that contribute to more violent and non-violent crimes as many relevant factors may still be missing.

However, I would like to try and answer the following questions answered.

1. Analyze if number of vacant and occupied houses and the period of time the houses were vacant had contributed to any significant change in violent and non-violent crime rates in communities

2. How has unemployment changed crime rate(violent and non-violent) in the communities?

3. Were people from a particular age group more vulnerable to crime?

4. Does ethnicity play a role in crime rate?

5. Has education played a role in bringing down the crime rate?

Version 5 release notes:

• Removes support for SPSS and Excel data.
• Changes the crimes that are stored in each file. There are more files now with fewer crimes per file. The files and their included crimes have been updated below.
  
• Adds in agencies that report 0 months of the year.
• Adds a column that indicates the number of months reported. This is generated summing up the number of unique months an agency reports data for. Note that this indicates the number of months an agency reported arrests for ANY crime. They may not necessarily report every crime every month. Agencies that did not report a crime with have a value of NA for every arrest column for that crime.
• Removes data on runaways.
  

Version 4 release notes:

• Changes column names from "poss_coke" and "sale_coke" to "poss_heroin_coke" and "sale_heroin_coke" to clearly indicate that these column includes the sale of heroin as well as similar opiates such as morphine, codeine, and opium. Also changes column names for the narcotic columns to indicate that they are only for synthetic narcotics.
  

Version 3 release notes:

• Add data for 2016.
• Order rows by year (descending) and ORI.

Version 2 release notes:

• Fix bug where Philadelphia Police Department had incorrect FIPS county code.

The Arrests by Age, Sex, and Race data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains highly granular data on the number of people arrested for a variety of crimes (see below for a full list of included crimes). The data sets here combine data from the years 1980-2015 into a single file. These files are quite large and may take some time to load.

All the data was downloaded from NACJD as ASCII+SPSS Setup files and read into R using the package asciiSetupReader. All work to clean the data and save it in various file formats was also done in R. For the R code used to clean this data, see here. https://github.com/jacobkap/crime_data. If you have any questions, comments, or suggestions please contact me at jkkaplan6@gmail.com.

I did not make any changes to the data other than the following. When an arrest column has a value of "None/not reported", I change that value to zero. This makes the (possible incorrect) assumption that these values represent zero crimes reported. The original data does not have a value when the agency reports zero arrests other than "None/not reported." In other words, this data does not differentiate between real zeros and missing values. Some agencies also incorrectly report the following numbers of arrests which I change to NA: 10000, 20000, 30000, 40000, 50000, 60000, 70000, 80000, 90000, 100000, 99999, 99998.

To reduce file size and make the data more manageable, all of the data is aggregated yearly. All of the data is in agency-year units such that every row indicates an agency in a given year. Columns are crime-arrest category units. For example, If you choose the data set that includes murder, you would have rows for each agency-year and columns with the number of people arrests for murder. The ASR data breaks down arrests by age and gender (e.g. Male aged 15, Male aged 18). They also provide the number of adults or juveniles arrested by race. Because most agencies and years do not report the arrestee's ethnicity (Hispanic or not Hispanic) or juvenile outcomes (e.g. referred to adult court, referred to welfare agency), I do not include these columns.

To make it easier to merge with other data, I merged this data with the Law Enforcement Agency Identifiers Crosswalk (LEAIC) data. The data from the LEAIC add FIPS (state, county, and place) and agency type/subtype. Please note that some of the FIPS codes have leading zeros and if you open it in Excel it will automatically delete those leading zeros.

I created 9 arrest categories myself. The categories are:

• Total Male Juvenile
• Total Female Juvenile
• Total Male Adult
• Total Female Adult
• Total Ma

Content

Hate Crimes

This folder contains data behind the story Higher Rates Of Hate Crimes Are Tied To Income Inequality.

Header	Definition
state	State name
median_household_income	Median household income, 2016
share_unemployed_seasonal	Share of the population that is unemployed (seasonally adjusted), Sept. 2016
share_population_in_metro_areas	Share of the population that lives in metropolitan areas, 2015
share_population_with_high_school_degree	Share of adults 25 and older with a high-school degree, 2009
share_non_citizen	Share of the population that are not U.S. citizens, 2015
share_white_poverty	Share of white residents who are living in poverty, 2015
gini_index	Gini Index, 2015
share_non_white	Share of the population that is not white, 2015
share_voters_voted_trump	Share of 2016 U.S. presidential voters who voted for Donald Trump
hate_crimes_per_100k_splc	Hate crimes per 100,000 population, Southern Poverty Law Center, Nov. 9-18, 2016
avg_hatecrimes_per_100k_fbi	Average annual hate crimes per 100,000 population, FBI, 2010-2015

Sources: Kaiser Family Foundation Kaiser Family Foundation Kaiser Family Foundation Census Bureau Kaiser Family Foundation Kaiser Family Foundation Census Bureau Kaiser Family Foundation United States Elections Project Southern Poverty Law Center FBI

Correction

Please see the following commit: https://github.com/fivethirtyeight/data/commit/fbc884a5c8d45a0636e1d6b000021632a0861986

Context

This is a dataset from FiveThirtyEight hosted on their GitHub. Explore FiveThirtyEight data using Kaggle and all of the data sources available through the FiveThirtyEight organization page!

• Update Frequency: This dataset is updated daily.

Acknowledgements

This dataset is maintained using GitHub's API and Kaggle's API.

This dataset is distributed under the Attribution 4.0 International (CC BY 4.0) license.

Anti-Jewish attacks were the most common form of anti-religious group hate crimes in the United States in 2023, with ***** cases. Anti-Islamic hate crimes were the second most common anti-religious hate crimes in that year, with *** incidents.

For a comprehensive guide to this data and other UCR data, please see my book at ucrbook.comVersion 18 release notes:Adds data for 2021.Version 17 release notes:Adds data for 2020.Please note that the FBI has retired UCR data ending in 2020 data so this will be the last Offenses Known and Clearances by Arrest data they release. Changes .rda files to .rds. Please note that in 2020 the card_actual_pt variable always returns that the month was reported. This causes 2020 to report that all months are reported for all agencies because I use the card_actual_pt variable to measure how many months were reported. This variable is almost certainly incorrect since it is extremely unlikely that all agencies suddenly always report. However, I am keeping this incorrect value to maintain a consistent definition of how many months are missing (measuring missing months through card_actual_type, for example, gives different results for previous years so I don't want to change this). Version 16 release notes:Changes release notes description, does not change data.Version 15 release notes:Adds data for 2019.Please note that in 2019 the card_actual_pt variable always returns that the month was reported. This causes 2019 to report that all months are reported for all agencies because I use the card_actual_pt variable to measure how many months were reported. This variable is almost certainly incorrect since it is extremely unlikely that all agencies suddenly always report. However, I am keeping this incorrect value to maintain a consistent definition of how many months are missing (measuring missing months through card_actual_type, for example, gives different results for previous years so I don't want to change this). Version 14 release notes:Adds arson data from the UCR's Arson dataset. This adds just the arson variables about the number of arson incidents, not the complete set of variables in that dataset (which include damages from arson and whether structures were occupied or not during the arson.As arson is an index crime, both the total index and the index property columns now include arson offenses. The "all_crimes" variables also now include arson.Adds a arson_number_of_months_missing column indicating how many months were not reporting (i.e. missing from the annual data) in the arson data. In most cases, this is the same as the normal number_of_months_missing but not always so please check if you intend to use arson data.Please note that in 2018 the card_actual_pt variable always returns that the month was reported. This causes 2018 to report that all months are reported for all agencies because I use the card_actual_pt variable to measure how many months were reported. This variable is almost certainly incorrect since it is extremely unlikely that all agencies suddenly always report. However, I am keeping this incorrect value to maintain a consistent definition of how many months are missing (measuring missing months through card_actual_type, for example, gives different results for previous years so I don't want to change this).For some reason, a small number of agencies (primarily federal agencies) had the same ORI number in 2018 and I removed these duplicate agencies. Version 13 release notes: Adds 2018 dataNew Orleans (ORI = LANPD00) data had more unfounded crimes than actual crimes in 2018 so unfounded columns for 2018 are all NA. Version 12 release notes: Adds population 1-3 columns - if an agency is in multiple counties, these variables show the population in the county with the most people in that agency in it (population_1), second largest county (population_2), and third largest county (population_3). Also adds county 1-3 columns which identify which counties the agency is in. The population column is the sum of the three population columns. Thanks to Mike Maltz for the suggestion!Fixes bug in the crosswalk data that is merged to this file that had the incorrect FIPS code for Clinton, Tennessee (ORI = TN00101). Thanks for Brooke Watson for catching this bug!Adds a last_month_reported column which says which month was reported last. This is actually how the FBI defines number_of_months_reported so is a more accurate representation of that. Removes the number_of_months_reported variable as the name is misleading. You should use the last_month_reported or the number_of_months_missing (see below) var

This dataset contains aggregate data on violent index victimizations at the quarter level of each year (i.e., January – March, April – June, July – September, October – December), from 2001 to the present (1991 to present for Homicides), with a focus on those related to gun violence. Index crimes are 10 crime types selected by the FBI (codes 1-4) for special focus due to their seriousness and frequency. This dataset includes only those index crimes that involve bodily harm or the threat of bodily harm and are reported to the Chicago Police Department (CPD). Each row is aggregated up to victimization type, age group, sex, race, and whether the victimization was domestic-related. Aggregating at the quarter level provides large enough blocks of incidents to protect anonymity while allowing the end user to observe inter-year and intra-year variation. Any row where there were fewer than three incidents during a given quarter has been deleted to help prevent re-identification of victims. For example, if there were three domestic criminal sexual assaults during January to March 2020, all victims associated with those incidents have been removed from this dataset. Human trafficking victimizations have been aggregated separately due to the extremely small number of victimizations.

This dataset includes a " GUNSHOT_INJURY_I " column to indicate whether the victimization involved a shooting, showing either Yes ("Y"), No ("N"), or Unknown ("UKNOWN.") For homicides, injury descriptions are available dating back to 1991, so the "shooting" column will read either "Y" or "N" to indicate whether the homicide was a fatal shooting or not. For non-fatal shootings, data is only available as of 2010. As a result, for any non-fatal shootings that occurred from 2010 to the present, the shooting column will read as “Y.” Non-fatal shooting victims will not be included in this dataset prior to 2010; they will be included in the authorized dataset, but with "UNKNOWN" in the shooting column.

The dataset is refreshed daily, but excludes the most recent complete day to allow CPD time to gather the best available information. Each time the dataset is refreshed, records can change as CPD learns more about each victimization, especially those victimizations that are most recent. The data on the Mayor's Office Violence Reduction Dashboard is updated daily with an approximately 48-hour lag. As cases are passed from the initial reporting officer to the investigating detectives, some recorded data about incidents and victimizations may change once additional information arises. Regularly updated datasets on the City's public portal may change to reflect new or corrected information.

How does this dataset classify victims?

The methodology by which this dataset classifies victims of violent crime differs by victimization type:

Homicide and non-fatal shooting victims: A victimization is considered a homicide victimization or non-fatal shooting victimization depending on its presence in CPD's homicide victims data table or its shooting victims data table. A victimization is considered a homicide only if it is present in CPD's homicide data table, while a victimization is considered a non-fatal shooting only if it is present in CPD's shooting data tables and absent from CPD's homicide data table.

To determine the IUCR code of homicide and non-fatal shooting victimizations, we defer to the incident IUCR code available in CPD's Crimes, 2001-present dataset (available on the City's open data portal). If the IUCR code in CPD's Crimes dataset is inconsistent with the homicide/non-fatal shooting categorization, we defer to CPD's Victims dataset.

For a criminal homicide, the only sensible IUCR codes are 0110 (first-degree murder) or 0130 (second-degree murder). For a non-fatal shooting, a sensible IUCR code must signify a criminal sexual assault, a robbery, or, most commonly, an aggravated battery. In rare instances, the IUCR code in CPD's Crimes and Victims dataset do not align with the homicide/non-fatal shooting categorization:

1. In instances where a homicide victimization does not correspond to an IUCR code 0110 or 0130, we set the IUCR code to "01XX" to indicate that the victimization was a homicide but we do not know whether it was a first-degree murder (IUCR code = 0110) or a second-degree murder (IUCR code = 0130).
2. When a non-fatal shooting victimization does not correspond to an IUCR code that signifies a criminal sexual assault, robbery, or aggravated battery, we enter “UNK” in the IUCR column, “YES” in the GUNSHOT_I column, and “NON-FATAL” in the PRIMARY column to indicate that the victim was non-fatally shot, but the precise IUCR code is unknown.

Other violent crime victims: For other violent crime types, we refer to the IUCR classification that exists in CPD's victim table, with only one exception:

1. When there is an incident that is associated with no victim with a matching IUCR code, we assume that this is an error. Every crime should have at least 1 victim with a matching IUCR code. In these cases, we change the IUCR code to reflect the incident IUCR code because CPD's incident table is considered to be more reliable than the victim table.

Note: All businesses identified as victims in CPD data have been removed from this dataset.

Note: The definition of “homicide” (shooting or otherwise) does not include justifiable homicide or involuntary manslaughter. This dataset also excludes any cases that CPD considers to be “unfounded” or “noncriminal.”

Note: In some instances, the police department's raw incident-level data and victim-level data that were inputs into this dataset do not align on the type of crime that occurred. In those instances, this dataset attempts to correct mismatches between incident and victim specific crime types. When it is not possible to determine which victims are associated with the most recent crime determination, the dataset will show empty cells in the respective demographic fields (age, sex, race, etc.).

Note: The initial reporting officer usually asks victims to report demographic data. If victims are unable to recall, the reporting officer will use their best judgment. “Unknown” can be reported if it is truly unknown.

This study was conducted in response to a lack of existing data collections relating specifically to acts of American terrorism. A primary goal of the study was to create an empirical database from which criminological theories and governmental policies could be effectively evaluated. The American Terrorism Study began in 1989 when the Federal Bureau of Investigation's (FBI) Terrorist Research and Analytical Center released a list of persons indicted as a result of investigation under the FBI's Counterterrorism Program. Since that time, FBI has released additional lists to the principal investigators. After receiving a list of persons indicted in federal criminal court as a result of an official terrorism investigation, the researchers reviewed the cases at either the federal district court where the cases were tried or at the federal regional records center where the cases were archived. The researchers divided the dataset into five distinct datasets. Part 1, Counts Data, provides data on every count for each indictee in each indictment. This is the basic dataset. There were 7,306 counts from 1980 to 2002. Part 2, Indictees Data, provides data on each of the 574 indictees from 1980-2002. Part 3, Persons Data, provides data on the 510 individuals who were indicted by the federal government as a result of a terrorism investigation. Part 4, Cases Data, provides one line of data for each of the 172 criminal terrorism cases that resulted from a federal terrorism investigation. Part 5, Group Data, provides one line of case data for each of the 85 groups that were tried in federal court for terrorism-related activity. Each of the five datasets includes information on approximately 80 variables divided into four major categories: (1) demographic information, (2) information about the terrorist group to which the individual belongs, (3) prosecution and defense data, and (4) count/case outcome and sentencing data.

All BPD data on Open Baltimore is preliminary data and subject to change. The information presented through Open Baltimore represents Part I victim based crime data. The data do not represent statistics submitted to the FBI's Uniform Crime Report (UCR); therefore any comparisons are strictly prohibited. For further clarification of UCR data, please visit http://www.fbi.gov/about-us/cjis/ucr/ucr. Please note that this data is preliminary and subject to change. Prior month data is likely to show changes when it is refreshed on a monthly basis. All data is geocoded to the approximate latitude/longitude location of the incident and excludes those records for which an address could not be geocoded. Any attempt to match the approximate location of the incident to an exact address is strictly prohibited.

Acknowledgements

This dataset was kindly made available by the City of Baltimore. You can find the original dataset, which is updated regularly, here.

The Data

Following the implementation of the Brady act in 1994, the Federal Bureau of Investigation (FBI) developed a system to conduct background checks on individuals wanting to obtain a firearm. The system known as the National Instant Criminal Background Check System (NICS) was created in collaboration with the Buereu of Alcohol, Tabacco and Firearms and local law enforcement agencies. Since it's inception in November 1998, the FBI has released monthly data from each state and U.S territory. The FBI claims that over 300 million requests have been aprroved, and 1.5 million have been denied.

Source

The FBI releases the monthly data in pdf format. Thanks to Buzzfeed's Jeremy Singer Vine, a public repository on resides on GitHub containing the pdf data parsed into a csv file. The data csv file can be accessed here: https://raw.githubusercontent.com/BuzzFeedNews/nics-firearm-background-checks/master/data/nics-firearm-background-checks.csv The pdf version of the data can be found here: https://www.fbi.gov/file-repository/nics_firearm_checks_-_month_year_by_state_type.pdf/view

Important Considerations About the Data

The data simply collects the quantity of background checks conducted. The FBI advices agaisnt the use of this data to analyze gun sales, as conducting a background check does not implictly mean that a firearm was purchased. For example, some states require monthly background checks on all their current conceal carry permit holders. Additionally, some states participate in the program more agressively than others. A map displaying the level of compliance by state can be found here: https://www.fbi.gov/file-repository/nics-participation-map.pdf/view

This version (V3) fixes a bug in Version 2 where 1993 data did not properly deal with missing values, leading to enormous counts of crime being reported. This is a collection of Offenses Known and Clearances By Arrest data from 1960 to 2016. The monthly zip files contain one data file per year(57 total, 1960-2016) as well as a codebook for each year. These files have been read into R using the ASCII and setup files from ICPSR (or from the FBI for 2016 data) using the package asciiSetupReader. The end of the zip folder's name says what data type (R, SPSS, SAS, Microsoft Excel CSV, feather, Stata) the data is in. Due to file size limits on open ICPSR, not all file types were included for all the data. The files are lightly cleaned. What this means specifically is that column names and value labels are standardized. In the original data column names were different between years (e.g. the December burglaries cleared column is "DEC_TOT_CLR_BRGLRY_TOT" in 1975 and "DEC_TOT_CLR_BURG_TOTAL" in 1977). The data here have standardized columns so you can compare between years and combine years together. The same thing is done for values inside of columns. For example, the state column gave state names in some years, abbreviations in others. For the code uses to clean and read the data, please see my GitHub file here. https://github.com/jacobkap/crime_data/blob/master/R_code/offenses_known.RThe zip files labeled "yearly" contain yearly data rather than monthly. These also contain far fewer descriptive columns about the agencies in an attempt to decrease file size. Each zip folder contains two files: a data file in whatever format you choose and a codebook. The data file is aggregated yearly and has already combined every year 1960-2016. For the code I used to do this, see here https://github.com/jacobkap/crime_data/blob/master/R_code/yearly_offenses_known.R.If you find any mistakes in the data or have any suggestions, please email me at jkkaplan6@gmail.comAs a description of what UCR Offenses Known and Clearances By Arrest data contains, the following is copied from ICPSR's 2015 page for the data.The Uniform Crime Reporting Program Data: Offenses Known and Clearances By Arrest dataset is a compilation of offenses reported to law enforcement agencies in the United States. Due to the vast number of categories of crime committed in the United States, the FBI has limited the type of crimes included in this compilation to those crimes which people are most likely to report to police and those crimes which occur frequently enough to be analyzed across time. Crimes included are criminal homicide, forcible rape, robbery, aggravated assault, burglary, larceny-theft, and motor vehicle theft. Much information about these crimes is provided in this dataset. The number of times an offense has been reported, the number of reported offenses that have been cleared by arrests, and the number of cleared offenses which involved offenders under the age of 18 are the major items of information collected.

In September 2017, the Intelligence Advanced Research Projects Activity (IARPA) held a data collection as part of its Nail to Nail (N2N) Fingerprint Challenge. Participating Challengers deployed devices designed to collect an image of the full nail to nail surface area of a fingerprint equivalent to a rolled fingerprint from an unacclimated user, without assistance from a trained operator. Traditional operator-assisted live-scan rolled fingerprints were also captured, along with assorted other friction ridge live-scan and latent captures.In this data collection, study participants needed to have their fingerprints captured using traditional operator-assisted techniques in order to quantify the performance of the Challenger devices. IARPA invited members of the Federal Bureau of Investigation (FBI) Biometric Training Team to the data collection to perform this task. Each study participant had N2N fingerprint images captured twice, each by a different FBI expert, resulting in two N2N baseline datasets.To ensure the veracity of recorded N2N finger positions in the baseline datasets, Challenge test staff also captured plain fingerprint impressions in a 4-4-2 slap configuration. This capture method refers to simultaneously imaging the index, middle, ring, and little fingers on the right hand, then repeating the process on the left hand, and finishing with the simultaneous capture of the left and right thumbs. This technique is a best practice to ensure finger sequence order, since it is physically challenging for a study participant to change the ordering of fingers when imaging them simultaneously. There were four baseline (two rolled and two slap), eight challenger and ten auxiliary fingerprint sensors deployed during the data collection, amassing a series of rolled and plain images. It was required that the baseline devices achieve 100% acquisition rate, in order to verify the recorded friction ridge generalized positions (FRGPs) and study participant identifiers for other devices. There were no such requirements for Challenger devices. Not all devices were able to achieve 100% acquisition rate.Plain, rolled, and touch-free impression fingerprints were captured from a multitude of devices, as well as sets of plain palm impressions. NIST also partnered with the FBI and Schwarz Forensic Enterprises (SFE) to design activity scenarios in which subjects would likely leave fingerprints on different objects. The activities and associated objects were chosen in order to use a number of latent print development techniques and simulate the types of objects often found in real law enforcement case work.

Version 11 release notes:Changes release notes description, does not change data.Version 10 release notes:The data now has the following age categories (which were previously aggregated into larger groups to reduce file size): under 10, 10-12, 13-14, 40-44, 45-49, 50-54, 55-59, 60-64, over 64. These categories are available for female, male, and total (female+male) arrests. The previous aggregated categories (under 15, 40-49, and over 49 have been removed from the data). Version 9 release notes:For each offense, adds a variable indicating the number of months that offense was reported - these variables are labeled as "num_months_[crime]" where [crime] is the offense name. These variables are generated by the number of times one or more arrests were reported per month for that crime. For example, if there was at least one arrest for assault in January, February, March, and August (and no other months), there would be four months reported for assault. Please note that this does not differentiate between an agency not reporting that month and actually having zero arrests. The variable "number_of_months_reported" is still in the data and is the number of months that any offense was reported. So if any agency reports murder arrests every month but no other crimes, the murder number of months variable and the "number_of_months_reported" variable will both be 12 while every other offense number of month variable will be 0. Adds data for 2017 and 2018.Version 8 release notes:Adds annual data in R format.Changes project name to avoid confusing this data for the ones done by NACJD.Fixes bug where bookmaking was excluded as an arrest category. Changed the number of categories to include more offenses per category to have fewer total files. Added a "total_race" file for each category - this file has total arrests by race for each crime and a breakdown of juvenile/adult by race. Version 7 release notes: Adds 1974-1979 dataAdds monthly data (only totals by sex and race, not by age-categories). All data now from FBI, not NACJD. Changes some column names so all columns are <=32 characters to be usable in Stata.Changes how number of months reported is calculated. Now it is the number of unique months with arrest data reported - months of data from the monthly header file (i.e. juvenile disposition data) are not considered in this calculation. Version 6 release notes: Fix bug where juvenile female columns had the same value as juvenile male columns.Version 5 release notes: Removes support for SPSS and Excel data.Changes the crimes that are stored in each file. There are more files now with fewer crimes per file. The files and their included crimes have been updated below.Adds in agencies that report 0 months of the year.Adds a column that indicates the number of months reported. This is generated summing up the number of unique months an agency reports data for. Note that this indicates the number of months an agency reported arrests for ANY crime. They may not necessarily report every crime every month. Agencies that did not report a crime with have a value of NA for every arrest column for that crime.Removes data on runaways.Version 4 release notes: Changes column names from "poss_coke" and "sale_coke" to "poss_heroin_coke" and "sale_heroin_coke" to clearly indicate that these column includes the sale of heroin as well as similar opiates such as morphine, codeine, and opium. Also changes column names for the narcotic columns to indicate that they are only for synthetic narcotics. Version 3 release notes: Add data for 2016.Order rows by year (descending) and ORI.Version 2 release notes: Fix bug where Philadelphia Police Department had incorrect FIPS county code. The Arrests by Age, Sex, and Race (ASR) data is an FBI data set that is part of the annual Uniform Crime Reporting (UCR) Program data. This data contains highly granular data on the number of people arrested for a variety of crimes (see below for a full list of included crimes). The data sets here combine data from the years 1974-2018 into a single file for each group of crimes. Each monthly file is only a single year as my laptop can't handle combining all the years together. These files are quite large and may take some time to load. Columns are crime-arrest category units. For example, If you choose the data set that includes murder, you would have rows for each age

Version 4 release notes:I am retiring this dataset - please do not use it. The reason that I made this dataset is that I had seen a lot of recent articles using the NACJD version of the data and had several requests that I make a concatenated version myself. This data is heavily flawed as noted in the excellent Maltz & Targonski's (2002) paper (see PDF available to download here and important paragraph from that article below) and I was worried that people were using the data without considering these flaws. So the data available here had the warning below this section (originally at the top of these notes so it was the most prominent thing) and had the Maltz & Targonski PDF included in the zip file so people were aware of it. There are two reasons that I am retiring it. First, I see papers and other non-peer reviewed reports still published using this data without addressing the main flaws noted by Maltz and Targonski. I don't want to have my work contribute to research that I think is fundamentally flawed. Second, this data is actually more flawed that I originally understood. The imputation process to replace missing data is based off of a bad design, and Maltz and Targonski talk about this in detail so I won't discuss it too much. The additional problem is that the variable that determines whether an agency has missing data is fatally flawed. That variable is the "number_of_months_reported" variable which is actually just the last month reported. So if you only report in December it'll have 12 months reported instead of 1. So even a good imputation process will be based on such a flawed measure of missingness that it will be wrong. How big of an issue is this? At the moment I haven't looked into it in enough detail to be sure but it's enough of a problem that I no longer want to release this kind of data (within the UCR data there are variables that you can use to try to determine the actual number of months reported but that stopped being useful due to a change in the data in 2018 by the FBI. And even that measure is not always accurate for years before 2018.).!!! Important Note: There are a number of flaws in the imputation process to make these county-level files. Included as one of the files to download (and also in every zip file) is Maltz & Targonski's 2002 paper on these flaws and why they are such an issue. I very strongly recommend that you read this paper in its entirety before working on this data. I am only publishing this data because people do use county-level data anyways and I want them to know of the risks. Important Note !!!The following paragraph is the abstract to Maltz & Targonski's paper: County-level crime data have major gaps, and the imputation schemes for filling in the gaps are inadequate and inconsistent. Such data were used in a recent study of guns and crime without considering the errors resulting from imputation. This note describes the errors and how they may have affected this study. Until improved methods of imputing county-level crime data are developed, tested, and implemented, they should not be used, especially in policy studies.Version 3 release notes: Adds a variable to all data sets indicating the "coverage" which is the proportion of the agencies in that county-year that report complete data (i.e. that aren't imputed, 100 = no imputation, 0 = all agencies imputed for all months in that year.). Thanks to Dr. Monica Deza for the suggestion. The following is directly from NACJD's codebook for county data and is an excellent explainer of this variable.The Coverage Indicator variable represents the proportion of county data that is reported for a given year. The indicator ranges from 0 to 100. A value of 0 indicates that no data for the county were reported and all data have been imputed. A value of 100 indicates that all ORIs in the county reported for all 12 months in the year. Coverage Indicator is calculated as follows: CI_x = 100 * ( 1 - SUM_i { [ORIPOP_i/COUNTYPOP] * [ (12 - MONTHSREPORTED_i)/12 ] } ) where CI = Coverage Indicator x = county i = ORI within countyReorders data so it's sorted by year then county rather than vice versa as before.Version 2 release notes: Fixes bug where Butler University (ORI = IN04940) had wrong FIPS state and FIPS state+county codes from the LEAIC crosswa

The Uniform Crime Reporting (UCR) Program has been the starting place for law enforcement executives, students of criminal justice, researchers, members of the media, and the public at large seeking information on crime in the nation. Part I categorizes incidents in two categories: violent and property crimes. Aggravated assault, forcible rape, murder, and robbery are classified as violent crime, while burglary, larceny-theft, and motor vehicle theft are classified as property crimes. This dataset contains FBI Uniform Crime Reporting (UCR) Part I crime data for the last 40 years in Greensboro, North Carolina. The crime rate or index is calculated on a per 100,000 resident basis.A crime rate describes the number of crimes reported to law enforcement agencies per 100,000 residents. A crime rate is calculated by dividing the number of reported crimes by the total population; the result is multiplied by 100,000. For example, in 2013 there were 496 robberies in Greensboro and the population was 268,176 according to the SBI estimate. This equals a robbery crime rate of 185 per 100,000 general population.496/268,176 = 0.00184953165085615 x 100,000 = 184.95The Greensboro Police Department is comprised of 787 sworn and non-sworn employees dedicated to the mission of partnering to fight crime for a safer Greensboro. We believe that effectively fighting crime requires everyone's effort. With your assistance, we can make our city safer. Wondering what you can do?Take reasonable steps to prevent being victimized. Lock your car and home doors. Be aware of your surroundings. If something or someonefeels out of the ordinary, go to a safe place.Be additional eyes and ears for us. Report suspicious or unusual activity, and provide tips through Crime Stoppers that can help solve crime.Look out for your neighbors. Strong communities with active Neighborhood Watch programs are not attractive to criminals. By taking care of the people around you, you can create safe places to live and work.Get involved! If you have children, teach them how to react to bullying, what the dangers of texting and driving are, and how to safely use the Internet. Talk with your older relatives about scams that target senior citizens.Learn more about GPD. Ride along with us. Participate in the Police Citizens' Academy. Volunteer, apply for an internship, or better yet join us.You may have heard about our philosophy of neighborhood-oriented policing. This is practice in policing that combines data-driven crime analysis with police/citizen partnerships to solve problems.In the spirit of partnership with the community, our goal is to make the Greensboro Police Department as accessible as possible to the people we serve. Policies and procedures, referred to as directives, are rules that all Greensboro Police Department employees must follow in carrying out the mission of the department. We will update the public copy of the directives in a timely manner to remain consistent with new policy and procedure updates.