Exploring the Relationship Between Policing and Income in Oakland

Cory McCubbrey, Tanay Gupta, Whitney Huff, and Mina Mohebbi

1 Introduction

For our project we analyzed data that impacted the Bay Area. We used a dataset from Oakland Police Department called the OPD Racial Impact Data for 2016 to 2018. This dataset had a previous report done by the Oakland Police Department that provided a statistical overview of discretionary stop data that we expanded upon. In addition to the main dataset we are utilizing, we also have located maps concerning niches of poverty in Oakland, as well as data on gentrification in Oakland. We combined this data with census tracts that included police beats in order to merge with another dataset that gave us income by location. This gave us further information and data to experiment with maps and graphs. We looked at the median income of beats in terms of the number of arrests, citations, and stops overall in those areas. We then imported additional census data and features such as education, housing and spoken languages to explore if machine learning models could predict the frequency of stops in a police beat.

1.1 Why Does This Matter?

Despite Oakland’s recent prominence and changing dynamic as a center for jobs and entrepreneurship, the city experiences great wealth inequality as a result of growing gentrification. There is a burgeoning need to address its disproportionate impact on existing communities of color. The influx of wealthier individuals is highest in Oakland’s “flatlands” neighborhoods, where people of color have long been the majority, and many are low-income (Montojo 2018). Gentrification causes greater criminalization of these previously lower-income neighborhoods- behaviors once considered not out of the ordinary for a particular area goes increasingly reported as crime as wealthier individuals flood these neighborhoods (Fayyad 2018). More reports of quality of life crimes occur, which entangles long-term residents in a greater degree of policing they had not experienced prior (Fayyad 2018). The concern that comes with this change in policing as a result of gentrification is that a greater degree of misconduct and disproportionate, negative action will be taken against low-income individuals in these neighborhoods (Fayyad 2018). As a result, we find it pertinent to explore the relationship between income and policing in Oakland, and to determine whether income is a predictor/correlative with particular kinds of policing- this includes stops, types of encounters, and results of encounters.

1.2 Hypothesis

The median income of beats influences policing, namely in terms of outcomes and number of arrests, citations, and stops overall. We believe higher income police beats may lead to more lenient outcomes, more justified stops, and less felony and misdemeanor arrests. Conversely, we believe that lower income police beats experience higher frequencies of stops, and more harsh outcomes as a result of these stops. We imported additional census data to explore if machine learning models can predict the frequency of stops in a police beat based on demographic information such as education, housing, and spoken languages.

1.3 Importing Packages and Data

	ContactDate	Beat	Race	Gender	EncounterType	ReasonForEncounter	Search	TypeOfSearch	ResultOfSearch	ResultOfEncounter	IntelligenceLedFactor
0	01/01/2016 12:00:00 AM	31X	White	Male	Vehicle	Traffic Violation	False	No	No	FI Report	NaN
1	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN
2	01/01/2016 12:00:00 AM	31Y	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN
3	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN
4	01/01/2016 12:00:00 AM	31X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Citation	NaN
...	...	...	...	...	...	...	...	...	...	...	...
83992	12/31/2017 12:00:00 AM	35X	Afr American	Male	Vehicle	Probation/Parole	True	Cursory	None	No Action	Yes
83993	12/31/2017 12:00:00 AM	35X	Afr American	Male	Vehicle	Probation/Parole	True	Cursory	None	No Action	Yes
83994	12/31/2017 12:00:00 AM	07X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	Citation	No
83995	12/31/2017 12:00:00 AM	30X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Warning	No
83996	12/31/2017 12:00:00 AM	21X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	No Action	Yes

83997 rows × 11 columns

This dataset includes a range of data across eleven columns. The ContactDate column has the dates of the incidents, and the Beat column lists the code for the area patrolled within Oakland. In order to understand what these codes mean, we also were able to access a map that displayed all of the beat boundaries in the city labeled with their individual codes. The dataset also includes columns on race and gender. The EncounterType column describes whether the stop was for a vehicle, bicycle, or pedestrian. ReasonForEncounter describes if the stop was conducted due to traffic violation, reasonable suspicion, or another reason. Search contains Boolean True/False values for if a search was conducted. TypeOfSearch describes whether there was no search or probable cause for the search. ResultOfSearch describes what was found, and ResultOfEncounter includes data on whether a citation, report, misdemeanor, or other charge was given. IntelligenceLedFactor is the final column, and describes whether or not intelligence-led policing was involved in the initial stop.

<folium.features.GeoJson at 0x7f78d0d34be0>

2 Exploratory Data Analysis

png

The bar chart above depicts the races with the highest rates of poverty in Oakland in the year 2013. Hispanic people and black people experience the greatest rates of poverty, both around 25% of those impoverished in Oakland. White people are the third highest at just under 20%. Asian, other, multiracial, Native American, and Pacific Islanders experience a lower share of poverty in comparison to the top three.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78d08b83c8>

png

For the year 2016, African Americans experience the highest count of individuals affected by poverty, at around 20,000 people. Hispanic people are second, but have a lower count at around 7000 individuals. White, Asian, and other races have under 5000 individuals living in poverty, and the three categories together, when added up, are still far lower than the amount of African Americans affected by poverty.

	ContactDate	Beat	Race	Gender	EncounterType	ReasonForEncounter	Search	TypeOfSearch	ResultOfSearch	ResultOfEncounter	IntelligenceLedFactor	Year
0	01/01/2016 12:00:00 AM	31X	White	Male	Vehicle	Traffic Violation	False	No	No	FI Report	NaN	2016
1	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016
2	01/01/2016 12:00:00 AM	31Y	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016
3	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016
4	01/01/2016 12:00:00 AM	31X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016
...	...	...	...	...	...	...	...	...	...	...	...	...
83992	12/31/2017 12:00:00 AM	35X	Afr American	Male	Vehicle	Probation/Parole	True	Cursory	None	No Action	Yes	2017
83993	12/31/2017 12:00:00 AM	35X	Afr American	Male	Vehicle	Probation/Parole	True	Cursory	None	No Action	Yes	2017
83994	12/31/2017 12:00:00 AM	07X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	Citation	No	2017
83995	12/31/2017 12:00:00 AM	30X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Warning	No	2017
83996	12/31/2017 12:00:00 AM	21X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	No Action	Yes	2017

83997 rows × 12 columns

True

2.1 Merging Census Data with Police Data

1.1 We want to explore the relationship between our Oakland Police dataset with the financial and racial information found in our census dataset. Our Oakland Police dataset is broken into police beats, whereas our financial dataset is broken into census tracts. Based on the visualization below, we found that census tracts mostly lie within the boundaries of police beats.

The map above shows census tracts in red. We overlaid beats (black borders) over the census tracts to visualize how census tracts were encompassed by police beats.

1.2 Considering that most census tracts fall within police beats, we decided to represent each census tract by its geomentric centroid. This way, we can merge the census data with police beats based on which beat each centroid falls into. Our end goal is to pair the income of each census tract to its corresponding police beat.

1.3 Now we begin the process of conducting a spatial join between the census tracts and police beats.

/usr/local/lib/python3.6/dist-packages/pyproj/crs/crs.py:53: FutureWarning: '+init=<authority>:<code>' syntax is deprecated. '<authority>:<code>' is the preferred initialization method. When making the change, be mindful of axis order changes: https://pyproj4.github.io/pyproj/stable/gotchas.html#axis-order-changes-in-proj-6
  return _prepare_from_string(" ".join(pjargs))

	ContactDate	Beat	Race	Gender	EncounterType	ReasonForEncounter	Search	TypeOfSearch	ResultOfSearch	ResultOfEncounter	IntelligenceLedFactor	Year	geometry	Geography
0	01/01/2016 12:00:00 AM	31X	White	Male	Vehicle	Traffic Violation	False	No	No	FI Report	NaN	2016	MULTIPOLYGON (((-122.19533 37.75215, -122.1960...	4090
1	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016	MULTIPOLYGON (((-122.19533 37.75215, -122.1960...	4090
2	01/01/2016 12:00:00 AM	31Y	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016	MULTIPOLYGON (((-122.19234 37.74976, -122.1917...	4091
3	01/01/2016 12:00:00 AM	31X	Hispanic	Male	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016	MULTIPOLYGON (((-122.19533 37.75215, -122.1960...	4090
4	01/01/2016 12:00:00 AM	31X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Citation	NaN	2016	MULTIPOLYGON (((-122.19533 37.75215, -122.1960...	4090
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
139811	12/31/2017 12:00:00 AM	30X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Warning	No	2017	MULTIPOLYGON (((-122.18200 37.76962, -122.1818...	4085
139812	12/31/2017 12:00:00 AM	30X	Afr American	Female	Vehicle	Traffic Violation	False	No	No	Warning	No	2017	MULTIPOLYGON (((-122.18200 37.76962, -122.1818...	4086
139813	12/31/2017 12:00:00 AM	21X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	No Action	Yes	2017	MULTIPOLYGON (((-122.21853 37.79858, -122.2186...	4063
139814	12/31/2017 12:00:00 AM	21X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	No Action	Yes	2017	MULTIPOLYGON (((-122.21853 37.79858, -122.2186...	4064
139815	12/31/2017 12:00:00 AM	21X	Afr American	Male	Vehicle	Traffic Violation	False	No	No	No Action	Yes	2017	MULTIPOLYGON (((-122.21853 37.79858, -122.2186...	4062.01

139816 rows × 14 columns

The table above is the original Oakland police data, with a column added that includes the corresponding census tract for the beat which the stop took place in.

/usr/local/lib/python3.6/dist-packages/pandas/core/frame.py:3997: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  errors=errors,

	Beat	Geography	Household Income by Race
16	17X	4054.02	33555
17	17X	4054.01	39712

# 3 Choose Features for Modeling

3.1 Education Data per Tract

I chose education as a feature because it is often correlated with wealth and crime. The more a child is educated, the more money they can earn, and therefore the less likely they are to perform a crime. As well, when kids are less educated, they are more likely to take part in crime to help their families or for a sense of security. Additionally, poorer areas are much more likely to have high crime rates, which directly correlates with the median education of that area.

I cleaned the original table by first naming the original table. Second, I dropped the columns that I did not need such as “geoid, Male, Female, and the error columns”. Next, I combined all of the columns that fell under the category of high school or less for male and female into a new column called “High School or less”. Then, I did the same for by adding new columns for those that fell under “Associates/Bachelor degrees” and “Masters/Professional/Doctorate degrees”. Finally, I dropped all of the columns except for three new ones.

	Total Degrees	High School or less	Associates/Bachelor Degrees	Masters/Professional/Doctorate Degrees	Census Tracts
0	2455	136	1097	1222	4001
1	1579	83	699	797	4002
2	3978	612	1899	1467	4003
3	3000	292	1384	1324	4004
4	2892	421	1526	945	4005
...	...	...	...	...	...
356	3235	603	1812	820	4517.04
357	60	7	36	17	9819
358	49	20	25	4	9820
359	585	79	295	211	9832
360	0	0	0	0	9900

361 rows × 5 columns

The final table above contains the number of particular degrees for each census tract, which is the column on the far right.

3.2 Housing Units per Tract

We chose to use housing units per tract (by type as well) for our next feature, to help us predict the amount of stops in a police beat. The first assumption that this particular set of data as a feature may improve our model is because crime can be attributed to neighborhood type. For instance, neighborhoods with a higher density of 2+ unit housing may be more prone to stops and crime due to the increased population and likely increased patrols. Areas with more density of units themselves may be more prone to crime and patrols due to an increased population. In Oakland, at least, detached housing and single units tend to be in neighborhoods with that type of housing and a lesser population, versus other parts of the city more urban or multi-unit housing units are common. Therefore, the more units and thus occupants and population in a particular beat, the more patrols may go there and conduct more stops. Hopefully, the model will show that this factor helps to predict whether or not more stops are conducted.

We determined in order to reduce the amount of features while retaining the valuable information this data offered us, we decided to combine the values for 2.0 units, 3 or 4, 5 to 9, 10 to 19, 20 to 49, and 50 or more into 2.0+, just as a column for multiunit housing. We retained the 1, attached and 1, detached descriptions, as well as Mobile home and Boat, RV, van, etc.

	geoid	Geography	Total Housing Units	1, detached	1, attached	Mobile home	Boat, RV, van, etc	2.0+
3	14000US06001400400	4004	1875	749	93	0	0	1033
4	14000US06001400500	4005	1719	630	197	0	0	892
5	14000US06001400600	4006	732	346	110	0	0	276
6	14000US06001400700	4007	2036	848	80	18	0	1090
7	14000US06001400800	4008	1752	607	187	0	0	958
...	...	...	...	...	...	...	...	...
356	14000US06001451704	451704	1722	1437	38	0	0	247
357	14000US06001981900	9819	31	16	0	0	0	15
358	14000US06001982000	9820	32	21	0	0	0	11
359	14000US06001983200	9832	370	18	8	3	0	341
360	14000US06001990000	9900	0	0	0	0	0	0

358 rows × 8 columns

Here we began merging the education and housing units table with the beats and income table. While the housing and education tables contained information for all census tracts in Alameda County, the merge simply kept the tracts that corresponded to the beats data we had- the tracts in Oakland.

3.3 Spoken Languages Per Tract

This data from the census measured the number of people from each census tract that spoke different languages at home. We wanted to explore whether the amount of non-English speakers in an area correlated with the amount of police presence. The original dataset was divided into multiple age groups, and then further divided by language regions: ‘English’, ‘Spanish’, ‘Indo-European’, ‘Asian and Pacific Island’, and ‘Other’. This represented far too many features for the purpose of our analysis, so we combined all ages together. We then combined the non-English categories together in order to arrive at a single statistic we could use as a feature. The percentage of people who speak a language other than English for each census tract is calculated using the code below. This is being used mainly to see if foreign languages are related to police behavior, which would be revealing if police behavior is unfairly effecting people in this way.

	Geography	Speak Non-English language Cnt	Languages Total
0	4001	853	2920
1	4002	376	1912
2	4003	809	4704
3	4004	600	3624
4	4005	640	3503
...	...	...	...
356	451704	826	4140
357	9819	4	60
358	9820	15	63
359	9832	129	609
360	9900	0	0

361 rows × 3 columns

The table above contains the count for speakers of non-English languages, and the total for individuals speaking languages. This information is accompanied by the Geography column, which contains the respective census tract.

3.4 Marrital status per tract

	Geography	Total Married	Total
0	4001	1622	2705
1	4002	859	1690
2	4003	1936	4312
3	4004	1315	3285
4	4005	1244	3277
...	...	...	...
356	4517.04	2277	3657
357	9819	22	60
358	9820	22	49
359	9832	252	596
360	9900	0	0

361 rows × 3 columns

To begin with, the data regarding the marital status for people in oakland was very specific and broken down into elements like “never married”, “divorced” etc. The data was further broken down into male and female. From all this, we first chose only people who are currently married. This is because the goal behind using this data was to check if being married had any impact on the outcome variable. After selecting this, we had to further clean the data by eliminating all the columns that calculated the errors. Then we added up the total number of married males and females. Finally we were left with columns that represented each tract and two rows: one for the total number of people that were married and one row for the total number of people in the tract. The two loops in the code, iterate through each column to create a series of data. The loop for tracts, splits the string in each value to only pick out the tract number and the other two loops simply pick out the value for the number of married people and total. Finally, we had to transpose the data and add the series that we had gathered to the final marriage table from where it was merged with the features_final table for further analysis.

3.5 Merging Features

Below we merged the language and marriage tables to to the housing, education, beats, and income table from the previous merge.

3.6 Final Features Table

Each beat can contain multiple census tracts. We added the values for each census tract together if they existed inside the same police beat. We then converted the totals for each beat to percentages, which represent the ratio of people in a specific category to the total population of all census tracts that lie within a certain police beat.

	Beat	Household Income by Race	1, detached	1, attached	Mobile home	Boat, RV, van, etc	2.0+	High School or less	Associates/Bachelor Degrees	Masters/Professional/Doctorate Degrees	Speak Non-English language Cnt	Total Married	Count
0	01X	183496	3.8	3.3	2.5	0.0	90.4	31.6	49.5	18.9	44.0	38.8	679
1	02X	206706	10.0	9.5	0.0	0.0	80.5	36.8	57.1	6.0	23.3	21.5	2142
2	02Y	179299	27.7	10.8	0.0	0.0	61.6	31.3	54.7	14.0	26.5	30.5	1617
3	03X	19490	2.1	1.0	1.2	0.0	95.7	58.4	32.9	8.7	84.3	51.4	1936
4	03Y	74139	6.0	0.9	0.0	0.3	92.8	46.8	38.7	14.6	53.8	40.8	780
5	04X	46217	1.0	0.6	0.3	0.1	98.0	25.1	55.1	19.8	39.2	27.0	2951
6	05X	78516	26.8	7.7	0.3	0.0	65.2	26.4	58.5	15.1	27.4	27.9	1466
7	05Y	66308	24.2	11.8	0.0	0.0	64.0	14.9	70.2	14.9	25.9	38.2	353
8	06X	25985	29.1	7.4	0.0	0.0	63.5	31.4	57.3	11.3	36.3	28.3	3257
9	07X	85447	36.9	5.7	0.0	0.0	57.4	28.0	60.0	12.0	24.6	19.3	2893
10	08X	141827	5.1	3.5	0.0	0.0	91.4	23.3	58.0	18.6	26.0	30.3	2482
11	09X	241003	12.0	2.3	0.0	0.0	85.8	10.2	54.2	35.6	28.4	40.1	1207
12	10X	116802	39.0	9.4	0.0	0.0	51.7	23.9	57.4	18.7	26.8	35.7	994
13	10Y	97497	42.1	5.2	0.4	0.0	52.3	28.5	55.9	15.5	23.5	29.2	1619
14	11X	145247	39.8	12.5	0.0	0.0	47.7	12.6	54.4	33.0	17.8	37.9	755
15	12X	154413	28.8	4.8	0.0	0.0	66.4	12.5	58.3	29.2	26.1	31.1	882
16	12Y	337977	39.9	5.0	0.0	0.0	55.1	11.5	46.5	41.9	17.4	44.3	550
17	13X	363042	80.7	1.4	0.3	0.0	17.6	6.2	50.1	43.7	21.3	61.3	257
18	13Y	327491	77.9	0.5	0.0	0.0	21.6	5.2	53.1	41.7	21.8	57.4	110
19	13Z	477874	96.2	0.0	0.0	0.0	3.8	6.9	56.1	37.0	13.9	65.8	321
20	14X	214952	3.4	2.5	1.9	0.0	92.2	18.0	54.0	28.0	30.2	29.6	1102
21	14Y	158438	20.3	3.6	0.9	0.0	75.3	9.7	55.4	34.8	27.8	38.6	142
22	15X	136345	26.3	2.3	0.0	0.0	71.4	18.3	55.1	26.6	36.9	40.4	638
23	16X	304390	78.3	2.7	0.0	0.0	19.0	5.3	51.8	42.9	15.6	61.3	176
24	16Y	84375	61.3	2.9	0.5	0.0	35.2	11.4	56.6	32.0	15.3	52.5	272
25	17Y	152139	26.9	8.0	0.4	0.0	64.7	32.7	52.6	14.7	48.9	37.2	413
26	18Y	44375	48.8	7.3	0.0	0.6	43.3	54.4	38.1	7.4	61.3	38.8	419
27	19X	66547	11.9	5.3	1.4	0.6	80.8	37.3	51.5	11.2	62.8	38.2	4715
28	20X	78068	20.5	12.1	1.4	0.0	66.0	53.4	40.9	5.7	71.8	35.7	3229
29	21X	140143	51.6	3.8	1.1	0.0	43.5	50.7	42.4	6.9	48.9	36.6	1353
30	21Y	142975	44.0	3.9	1.3	0.0	50.8	50.2	41.5	8.3	70.0	41.5	1266
31	22X	220982	67.3	0.8	0.0	0.0	31.9	10.1	58.9	31.0	26.8	54.8	330
32	22Y	326251	77.0	4.7	0.6	0.0	17.6	20.3	50.9	28.8	26.2	48.4	237
33	23X	37319	30.0	10.0	0.8	0.0	59.2	68.0	27.6	4.5	76.6	40.6	4137
34	24Y	43125	44.4	8.1	0.6	0.0	46.9	51.4	36.5	12.1	55.7	42.4	701
35	25X	166483	69.5	4.4	0.0	0.0	26.1	19.7	53.9	26.4	30.6	48.6	801
36	25Y	103670	63.5	14.5	0.0	0.0	22.0	14.5	54.3	31.2	25.3	51.0	67
37	26X	53214	45.0	5.0	1.0	0.0	49.0	55.2	39.5	5.3	68.8	40.3	1984
38	26Y	49977	40.2	4.1	0.3	0.0	55.4	67.6	31.5	0.9	59.6	34.2	2532
39	27X	53300	53.6	5.0	0.0	2.5	38.9	38.0	54.1	8.0	36.9	40.1	2331
40	27Y	116174	49.1	4.6	0.6	0.0	45.7	59.7	34.7	5.6	57.3	37.3	2476
41	28X	151306	80.6	5.0	0.2	0.0	14.2	22.8	57.4	19.8	23.8	38.8	659
42	29X	51135	50.4	4.5	0.7	0.0	44.4	27.4	60.5	12.1	18.6	35.1	1960
43	30X	78537	64.4	3.6	0.2	0.0	31.8	59.5	36.5	4.0	54.6	39.0	4364
44	30Y	111908	68.0	7.0	0.0	0.0	25.0	35.5	50.3	14.2	37.5	38.6	1980
45	31X	44125	77.0	0.5	1.9	0.0	20.5	64.0	34.5	1.5	53.5	44.8	821
46	31Y	40000	89.4	0.0	0.0	0.0	10.6	62.6	34.7	2.8	54.2	33.4	1065
47	31Z	46250	74.4	4.2	0.0	0.0	21.5	69.6	29.2	1.2	60.2	41.0	376
48	32X	101385	64.5	11.0	1.2	0.0	23.3	60.3	36.1	3.7	55.8	38.7	2122
49	32Y	108439	65.1	5.4	1.4	0.0	28.1	42.0	52.4	5.6	25.5	29.0	1063
50	33X	70871	56.0	9.6	0.4	0.0	34.0	70.8	26.8	2.4	68.1	40.8	2314
51	34X	76873	54.5	5.5	0.2	0.0	39.7	68.8	28.7	2.5	69.1	36.4	4803
52	35X	101955	62.4	2.7	0.0	0.0	35.0	49.3	42.5	8.2	42.4	37.6	2656
53	35Y	204772	87.3	7.2	0.0	0.0	5.5	18.9	53.2	28.0	13.9	55.0	122

Here we have the table that lists the total amount of stops within that beat and the combined median income for each beat based on the income of its contained census tracts. We will use this to determine if there is any significant correlation between our police data and the incomes within each police beat.

	Beat	Count	Avg Median Household Income
0	01X	679	91748.000000
1	02X	2142	68902.000000
2	02Y	1617	59766.333333
3	03X	1936	19490.000000
4	03Y	780	37069.500000
5	04X	2951	23108.500000
6	05X	1466	39258.000000
7	05Y	353	66308.000000
8	06X	3257	25985.000000
9	07X	2893	42723.500000
10	08X	2482	47275.666667
11	09X	1207	80334.333333
12	10X	994	58401.000000
13	10Y	1619	48748.500000
14	11X	755	72623.500000
15	12X	882	77206.500000
16	12Y	550	112659.000000
17	13X	257	181521.000000
18	13Y	110	163745.500000
19	13Z	321	159291.333333
20	14X	1102	71650.666667
21	14Y	142	79219.000000
22	15X	638	68172.500000
23	16X	176	152195.000000
24	16Y	272	84375.000000
25	17X	489	36633.500000
26	17Y	413	50713.000000
27	18X	1228	32220.000000
28	18Y	419	44375.000000
29	19X	4715	33273.500000
30	20X	3229	39034.000000
31	21X	1353	46714.333333
32	21Y	1266	47658.333333
33	22X	330	110491.000000
34	22Y	237	108750.333333
35	23X	4137	37319.000000
36	24X	1403	49675.500000
37	24Y	701	43125.000000
38	25X	801	83241.500000
39	25Y	67	103670.000000
40	26X	1984	53214.000000
41	26Y	2532	24988.500000
42	27X	2331	53300.000000
43	27Y	2476	38724.666667
44	28X	659	75653.000000
45	29X	1960	51135.000000
46	30X	4364	39268.500000
47	30Y	1980	55954.000000
48	31X	821	44125.000000
49	31Y	1065	40000.000000
50	31Z	376	46250.000000
51	32X	2122	50692.500000
52	32Y	1063	54219.500000
53	33X	2314	35435.500000
54	34X	4803	38436.500000
55	35X	2656	50977.500000
56	35Y	122	102386.000000

	Count	Avg Median Household Income
count	57.000000	57.000000
mean	1473.631579	64095.289474
std	1223.295617	35963.574680
min	67.000000	19490.000000
25%	489.000000	39268.500000
50%	1102.000000	50977.500000
75%	2142.000000	77206.500000
max	4803.000000	181521.000000

Exploration of police searches by police beat.

Beats with a higher ratio of searches leading to arrests do not necessarily have higher household incomes, or higher numbers of searches in general.

	Beat	Count	Avg Median Household Income	#Searches	#Searches Leading to Arrest	geometry	Search Arrest Ratio
18	13Y	110	163745.500000	3	0.0	MULTIPOLYGON (((-122.24383 37.88399, -122.2425...	0.000000
21	14Y	142	79219.000000	8	0.0	MULTIPOLYGON (((-122.24348 37.81879, -122.2433...	0.000000
19	13Z	321	159291.333333	10	0.0	MULTIPOLYGON (((-122.20105 37.84320, -122.2009...	0.000000
39	25Y	67	103670.000000	7	0.0	MULTIPOLYGON (((-122.15411 37.79936, -122.1527...	0.000000
26	17Y	413	50713.000000	63	1.0	MULTIPOLYGON (((-122.22647 37.80065, -122.2265...	0.015873
45	29X	1960	51135.000000	297	23.0	MULTIPOLYGON (((-122.16873 37.77837, -122.1678...	0.077441
31	21X	1353	46714.333333	233	20.0	MULTIPOLYGON (((-122.21853 37.79858, -122.2186...	0.085837
11	09X	1207	80334.333333	58	5.0	MULTIPOLYGON (((-122.24202 37.83012, -122.2427...	0.086207
50	31Z	376	46250.000000	92	8.0	MULTIPOLYGON (((-122.17057 37.73202, -122.1705...	0.086957
52	32Y	1063	54219.500000	125	11.0	MULTIPOLYGON (((-122.15083 37.75347, -122.1508...	0.088000
43	27Y	2476	38724.666667	370	33.0	MULTIPOLYGON (((-122.19727 37.77164, -122.1960...	0.089189
25	17X	489	36633.500000	98	9.0	MULTIPOLYGON (((-122.24000 37.79385, -122.2400...	0.091837
32	21Y	1266	47658.333333	197	19.0	MULTIPOLYGON (((-122.21074 37.79651, -122.2106...	0.096447
13	10Y	1619	48748.500000	224	22.0	MULTIPOLYGON (((-122.26809 37.83685, -122.2678...	0.098214
56	35Y	122	102386.000000	10	1.0	MULTIPOLYGON (((-122.14127 37.71420, -122.1421...	0.100000
4	03Y	780	37069.500000	98	10.0	MULTIPOLYGON (((-122.27500 37.79821, -122.2750...	0.102041
41	26Y	2532	24988.500000	419	43.0	MULTIPOLYGON (((-122.19374 37.76237, -122.1932...	0.102625
22	15X	638	68172.500000	29	3.0	MULTIPOLYGON (((-122.25540 37.80039, -122.2550...	0.103448
55	35X	2656	50977.500000	379	40.0	MULTIPOLYGON (((-122.15083 37.75347, -122.1508...	0.105541
28	18Y	419	44375.000000	93	10.0	MULTIPOLYGON (((-122.24009 37.79378, -122.2400...	0.107527
20	14X	1102	71650.666667	54	6.0	MULTIPOLYGON (((-122.26236 37.81092, -122.2621...	0.111111
27	18X	1228	32220.000000	257	29.0	MULTIPOLYGON (((-122.23163 37.78872, -122.2316...	0.112840
47	30Y	1980	55954.000000	367	43.0	MULTIPOLYGON (((-122.15686 37.77149, -122.1565...	0.117166
36	24X	1403	49675.500000	170	22.0	MULTIPOLYGON (((-122.20418 37.78251, -122.2042...	0.129412
35	23X	4137	37319.000000	437	58.0	MULTIPOLYGON (((-122.22450 37.76494, -122.2252...	0.132723
40	26X	1984	53214.000000	278	37.0	MULTIPOLYGON (((-122.21159 37.77082, -122.2110...	0.133094
54	34X	4803	38436.500000	875	120.0	MULTIPOLYGON (((-122.16195 37.74836, -122.1625...	0.137143
33	22X	330	110491.000000	51	7.0	MULTIPOLYGON (((-122.20239 37.81810, -122.2022...	0.137255
46	30X	4364	39268.500000	728	100.0	MULTIPOLYGON (((-122.18200 37.76962, -122.1818...	0.137363
30	20X	3229	39034.000000	393	55.0	MULTIPOLYGON (((-122.22363 37.78201, -122.2239...	0.139949
2	02Y	1617	59766.333333	185	26.0	MULTIPOLYGON (((-122.29151 37.81080, -122.2915...	0.140541
51	32X	2122	50692.500000	349	51.0	MULTIPOLYGON (((-122.17974 37.73917, -122.1785...	0.146132
14	11X	755	72623.500000	47	7.0	MULTIPOLYGON (((-122.26230 37.83726, -122.2623...	0.148936
42	27X	2331	53300.000000	346	56.0	MULTIPOLYGON (((-122.19457 37.77353, -122.1949...	0.161850
37	24Y	701	43125.000000	115	19.0	MULTIPOLYGON (((-122.20324 37.79224, -122.2030...	0.165217
16	12Y	550	112659.000000	29	5.0	MULTIPOLYGON (((-122.25357 37.85082, -122.2536...	0.172414
3	03X	1936	19490.000000	132	23.0	MULTIPOLYGON (((-122.26220 37.79220, -122.2623...	0.174242
15	12X	882	77206.500000	63	11.0	MULTIPOLYGON (((-122.25663 37.82762, -122.2566...	0.174603
10	08X	2482	47275.666667	183	32.0	MULTIPOLYGON (((-122.25668 37.82753, -122.2575...	0.174863
12	10X	994	58401.000000	181	32.0	MULTIPOLYGON (((-122.28363 37.85058, -122.2835...	0.176796
48	31X	821	44125.000000	89	16.0	MULTIPOLYGON (((-122.19533 37.75215, -122.1960...	0.179775
6	05X	1466	39258.000000	183	33.0	MULTIPOLYGON (((-122.27833 37.80777, -122.2786...	0.180328
1	02X	2142	68902.000000	360	66.0	MULTIPOLYGON (((-122.28215 37.79626, -122.2838...	0.183333
44	28X	659	75653.000000	63	12.0	MULTIPOLYGON (((-122.17379 37.78043, -122.1750...	0.190476
29	19X	4715	33273.500000	723	140.0	MULTIPOLYGON (((-122.25418 37.79900, -122.2537...	0.193638
53	33X	2314	35435.500000	366	73.0	MULTIPOLYGON (((-122.19234 37.74976, -122.1920...	0.199454
38	25X	801	83241.500000	73	15.0	MULTIPOLYGON (((-122.17717 37.79429, -122.1770...	0.205479
8	06X	3257	25985.000000	594	138.0	MULTIPOLYGON (((-122.27771 37.83124, -122.2775...	0.232323
24	16Y	272	84375.000000	16	4.0	MULTIPOLYGON (((-122.21548 37.81233, -122.2151...	0.250000
5	04X	2951	23108.500000	175	44.0	MULTIPOLYGON (((-122.26160 37.81068, -122.2615...	0.251429
49	31Y	1065	40000.000000	222	59.0	MULTIPOLYGON (((-122.19234 37.74976, -122.1917...	0.265766
9	07X	2893	42723.500000	615	179.0	MULTIPOLYGON (((-122.27836 37.82637, -122.2780...	0.291057
7	05Y	353	66308.000000	43	13.0	MULTIPOLYGON (((-122.28990 37.81514, -122.2903...	0.302326
34	22Y	237	108750.333333	28	9.0	MULTIPOLYGON (((-122.17065 37.80008, -122.1706...	0.321429
17	13X	257	181521.000000	3	1.0	MULTIPOLYGON (((-122.23484 37.84816, -122.2342...	0.333333
23	16X	176	152195.000000	12	4.0	MULTIPOLYGON (((-122.23215 37.81619, -122.2316...	0.333333
0	01X	679	91748.000000	50	17.0	MULTIPOLYGON (((-122.27506 37.79811, -122.2746...	0.340000

The map above displays the density of searches by police beat. The darker the coloration in the beat, the higher the amount of searches that are conducted. The two areas with black coloration are not police beats and are out of OPD jurisdiction, the larger one being the town of Piedmont that is surrounded by Oakland.

4 Exploratory Data Analysis: Results of Encounter

The original police data from OPD’s Racial Impact Report contained a column called ‘Result of Encounter’, which detailed the result of the stop conducted by the officer involved. These included citations, warnings, field reports, felony arrest, misdemeanor arrests, and reports taken with no action. We found it pertinent to analyze this as a factor that may be influenced by income in a particular beat, with the logic that neighborhoods with higher incomes may experience higher occurrences of lower level results of encounter, like citations or warnings, rather than felony arrests. The following plots are labeled with the average median household income on the x-axis, and count of the respective encounter on the y-axis. Each point represents a particular beat.

Citation                  12142
Warning                    8376
FI Report                  6572
Felony Arrest              2152
Misdemeanor Arrest         1867
Report Taken-No Action     1460
Name: ResultOfEncounter, dtype: int64

Citations are the most heavily issued result of encounter, at 31,437 from all beats. Report Taken- No Action is the lowest issued result of encounter.

Beat  ResultOfEncounter     
01X   Citation                  126
      FI Report                  39
      Warning                    35
      Misdemeanor Arrest         18
      Felony Arrest               9
      Report Taken-No Action      4
02X   Warning                   287
      FI Report                 286
      Citation                  282
      Felony Arrest              89
      Misdemeanor Arrest         37
      Report Taken-No Action     27
02Y   Citation                  159
      Warning                   151
      FI Report                 125
      Felony Arrest              40
      Report Taken-No Action     36
      Misdemeanor Arrest         13
03X   Citation                  407
      Warning                   165
Name: ResultOfEncounter, dtype: int64

		Count
Beat	ResultOfEncounter
01X	Citation	126
	FI Report	39
	Warning	35
	Misdemeanor Arrest	18
	Felony Arrest	9
...	...	...
35Y	FI Report	9
	Report Taken-No Action	4
	Warning	2
	Felony Arrest	1
	Misdemeanor Arrest	1

331 rows × 1 columns

The results of encounter we are choosing to focus on are Citations, Warnings, No Action, Felony Arrests, and Misdemeanor Arrests. The reason behind this is that these categories are what we believe to possibly be more closely related to policing in neighborhoods based on income. In the following sections, we juxtaposed the counts per beat for each respective result with the median income for that beat, as determined by our census tract data work above. Then, for each result of encounter, we plotted the average median income against these counts to visualize the relationship between them. After doing so, we took the natural log of the counts for each result and created a linear plot for each result. This was done to solidify if there was any statistical correlation between the median income and number of that result of encounter; it allowed us to calculate the Pearson’s correlation coefficient.

4.1 Citations and Income

	Beat	ResultOfEncounter	Count
201	23X	Citation	910
313	34X	Citation	745
265	30X	Citation	536
241	27X	Citation	484
30	04X	Citation	476

	Beat	ResultOfEncounter	Count_x	Avg Median Household Income
0	23X	Citation	910	37319.0
1	34X	Citation	745	38436.5
2	30X	Citation	536	39268.5

	Beat	ResultOfEncounter	Count_x	Avg Median Household Income
56	13Y	Citation	12	163745.500000
55	25Y	Citation	13	103670.000000
54	22Y	Citation	14	108750.333333

png

The plot above has a point for each police beat. The x-axis has the average median household income, and the y-axis has the count of citations issued. There appears to be a negative relationship between income and citations given; as income increases, the count of citations issued decreases.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78cc9de7f0>

png

Pearsons correlation: -0.632

	Count	Avg Median Household Income
count	57.000000	57.000000
mean	213.017544	64095.289474
std	187.032607	35963.574680
min	12.000000	19490.000000
25%	72.000000	39268.500000
50%	159.000000	50977.500000
75%	294.000000	77206.500000
max	910.000000	181521.000000

In the data above, we can see that the top three beats (23X, 34X, 04X) by number of citations have a range of median income of [23108.50, 38436.50]. The bottom three beats (25Y, 35Y, 13Y) have a range of median income of [102386.0, 163745.50]. From this alone, it appears that the counts increase among lower income neighborhoods and decrease among higher income neighborhoods. However, there could be conflicting factors, and these will be discussed at the end of the section. The correlation coefficient calculated from the linearization of the plot is -0.632, which is low and shows moderately weak correlation between median income and number of citations issued.

4.2 Warnings and Income

	Beat	ResultOfEncounter	Count	Avg Median Household Income
0	34X	Warning	492	38436.5
1	19X	Warning	438	33273.5
2	30X	Warning	412	39268.5

	Beat	ResultOfEncounter	Count	Avg Median Household Income
56	35Y	Warning	2	102386.0
55	13Y	Warning	3	163745.5
54	25Y	Warning	4	103670.0

png

The plot above has a point for each police beat. The x-axis has the average median household income, and the y-axis has the count of warnings issued. There appears to be a negative relationship between income and warnings issued; as income increases, the count of warnings issued decreases.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78ca120c18>

png

Pearsons correlation: -0.731

	Count	Avg Median Household Income
count	57.000000	57.000000
mean	146.947368	64095.289474
std	126.692460	35963.574680
min	2.000000	19490.000000
25%	41.000000	39268.500000
50%	124.000000	50977.500000
75%	205.000000	77206.500000
max	492.000000	181521.000000

In the data above, we can see that the top three beats (19X, 34X, 30X) by number of warnings issued have a range of median income of [33273.50, 39268.50]. The bottom three beats (25Y, 35Y, 16X) have a range of median income of [102386.0, 152195.0]. Again, it appears that the counts increase among lower income neighborhoods and decrease among higher income neighborhoods. The correlation coefficient calculated from the linearization of the plot is -0.731, which is a higher and stronger correlation between median income and number of warnings issued than for citations.

4.3 No Action Taken and Income

	Beat	ResultOfEncounter	Count	Avg Median Household Income
40	30X	Report Taken-No Action	109.0	39268.5
48	34X	Report Taken-No Action	94.0	38436.5
8	06X	Report Taken-No Action	84.0	25985.0

	Beat	ResultOfEncounter	Count	Avg Median Household Income
29	22Y	Report Taken-No Action	1.0	108750.333333
19	14Y	Report Taken-No Action	1.0	79219.000000
17	13X	Report Taken-No Action	1.0	181521.000000

png

The plot above has a point for each police beat. The x-axis has the average median household income, and the y-axis has the count of no-action taken as a result of the encounter. There appears to be a negative relationship between income and no action taken; as income increases, the count of no-action taken results decreases.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78cd1ffac8>

png

Pearsons correlation: -0.671

	Count	Avg Median Household Income
count	51.000000	57.000000
mean	28.627451	64095.289474
std	25.698218	35963.574680
min	1.000000	19490.000000
25%	8.000000	39268.500000
50%	22.000000	50977.500000
75%	41.000000	77206.500000
max	109.000000	181521.000000

In the data above, we can see that the top three beats by no action taken have a range of median income of [33273.50, 39268.50]. Note that this range, and the three beats in which this range matches are the same ones as for warnings- 30X, 34X, and 19X. The bottom three beats (22Y, 13X, 14Y) have a range of median income of [79219.0, 181521.0]. Again, it appears that the counts increase among lower income neighborhoods and decrease among higher income neighborhoods. The correlation coefficient calculated from the linearization of the plot is -0.671, which is a moderately weak correlation between the result of encounter and median income.

4.4 Misdemeanor Arrests and Income

	Beat	ResultOfEncounter	Count	Avg Median Household Income
27	19X	Misdemeanor Arrest	294.0	33273.5
9	07X	Misdemeanor Arrest	157.0	42723.5
28	20X	Misdemeanor Arrest	124.0	39034.0

	Beat	ResultOfEncounter	Count	Avg Median Household Income
54	35Y	Misdemeanor Arrest	1.0	102386.0
24	17Y	Misdemeanor Arrest	1.0	50713.0
19	14Y	Misdemeanor Arrest	1.0	79219.0

png

The plot above has a point for each police beat. The x-axis has the average median household income, and the y-axis has the count of misdemeanor arrests. There appears to be a negative relationship between income and misdemeanor arrests; as income increases, the count of misdemeanor arrests decreases.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78ca026b38>

png

Pearsons correlation: -0.659

	Count	Avg Median Household Income
count	55.000000	57.000000
mean	33.945455	64095.289474
std	48.158849	35963.574680
min	1.000000	19490.000000
25%	7.000000	39268.500000
50%	18.000000	50977.500000
75%	39.500000	77206.500000
max	294.000000	181521.000000

In the data above, we can see that the top three beats (19X, 07X, 20X) by misdemeanor arrests have a range of median income of [33273.50, 42723.50]. The bottom three beats (25Y, 13Z, 13X) have a range of median income of [103670.0, 181521.0]. Again, it appears that the counts increase among lower income neighborhoods and decrease among higher income neighborhoods. The correlation coefficient calculated from the linearization of the plot is -0.659, which is a moderate correlation between misdemeanor arrests and income.

4.5 Felony Arrests and Income

	Beat	ResultOfEncounter	Count	Avg Median Household Income
51	34X	Felony Arrest	154.0	38436.5
43	30X	Felony Arrest	144.0	39268.5
8	06X	Felony Arrest	140.0	25985.0

	Beat	ResultOfEncounter	Count	Avg Median Household Income
53	35Y	Felony Arrest	1.0	102386.0
19	14Y	Felony Arrest	2.0	79219.0
17	13Y	Felony Arrest	2.0	163745.5

png

The plot above has a point for each police beat. The x-axis has the average median household income, and the y-axis has the count of felony arrests. There appears to be a negative relationship between income and felony arrests; as income increases, the count of felony arrests decreases.

<matplotlib.axes._subplots.AxesSubplot at 0x7f78c9ecfdd8>

png

Pearsons correlation: -0.695

	Count	Avg Median Household Income
count	54.000000	57.000000
mean	39.851852	64095.289474
std	36.971895	35963.574680
min	1.000000	19490.000000
25%	12.250000	39268.500000
50%	30.000000	50977.500000
75%	53.250000	77206.500000
max	154.000000	181521.000000

In the data above, we can see that the top three beats (34X, 30X, 19X) by felony arrests have a range of median income of [33273.50, 39263.50]. The bottom three beats (25Y, 13Z, 14Y) have a range of median income of [103670.0, 159291.33]. Again, it appears that the counts increase among lower income neighborhoods and decrease among higher income neighborhoods. The correlation coefficient calculated from the linearization of the plot is -0.695, which is a moderate correlation between felony arrests and income. This is the highest correlation calculated out of all of the results of encounter in this section.

4.6 Overall Assessment of the Data

Looking at all of the descriptive statistics above, we have made a few observations. 34X, 30X, and 19X appear in the highest counts for results of encounter than any other beats. 25Y, 13Z, and 13Y appear more commonly as the lowest number of counts across all the results of encounters we analyzed. Additionally, while the correlation coefficients are moderate to strong correlation, this does not mean that lower income beats cause higher arrests, citations, or warnings. The highest correlation was among warnings and median income, at -0.731. The lowest correlation was among citations and median income, at -0.632. Moreover, we were surprised to observe that the pattern stayed consistent with no action taken reports, as we expected them to have higher counts in higher median income beats. However, this could be because the initial stops disproportionately target lower income beats in the first place, and policing does not occur as heavily in higher income beats. The crime rates for higher income beats may also be lower than in lower income beats, explaining the disparity in counts as well.

Looking at the plots correlating income with a particular result of encounter, we found that lower income beats experienced higher counts of all results of encounter when compared to higher income beats. This speaks to the more overarching finding that in Oakland, lower income beats are policed more heavily than higher income beats.

Our correlation coefficients depicted moderately strong negative correlations between income and result of encounter, showing that as income increased, there were less felony arrests, citations, warnings, and reports filed with no action per beat.

In the larger context of policing in Oakland, it appears that lower income beats are policed more heavily. This may be due to higher crime rates, and also to the implicit bias of police to patrol and dole out consequences to individuals within these lower income beats.

4.7 Looking at the Data Overall

	ReasonForEncounter	index
4	Traffic Violation	63415
1	Probable Cause	11208
3	Reasonable Suspicion	5302

The most common types of crime are traffic violations, probable cause, and reasonable suspicion.

	Beat	Count
54	34X	3816
46	30X	3319
35	23X	3315
29	19X	2792
5	04X	2470

png

This graph and table represents the amount of traffic violations per beat in Oakland. The beats with the most amount of traffic violations are 34X, 30X, and 23X. The beats with the least amount of traffic violations are 35Y, 13Y, and 25Y.

	Beat	Count
29	19X	1340
9	07X	614
30	20X	556
8	06X	550
54	34X	535

png

This table and graph represents the beats the amount of probable cause stops per beat in Oakland. The beats with the highest amount of probable cause stops are 19X, 07X, and 20X. The beats with the lowest amount of probable cause stops are 13X, 13Z, and 25Y.

	Beat	Count
29	19X	404
35	23X	314
46	30X	268
9	07X	265
54	34X	230

png

This table and graph represents the amount of reasonable suspicion stops in Oakland. The beats with the highest amount of reasonable suspiscion are 10X, 23X, and 30X. The beats with the lowest amounts of reasonable suspicion stops are 13X, 16X, and 25Y.

	Beat	Count
54	34X	4803
29	19X	4715
46	30X	4364
35	23X	4137
8	06X	3257

png

This table and graph displays the amount of total crime per beat in Oakland. The beats with the most crime are 34X, 19X, and 30X. The beats with the least amount of crime are 35Y, 13Y, and 25Y.

This map represents the median income per police beat in Oakland. From this visualization it is easy to see that the median income is higher more inland, and in the hills of Oakland and towards Berkeley. The median income becomes lower as it is closer to San Leandro and the Oakland airport.

This map represents the amount of crime per police beat in Oakland. From this visualization it is clear that there is a higher concentration of crime in the center of Oakland, downtown, and less in the Oakland hills.

6 Exploratory Data Analysis: Reasons for encounter

Traffic Violation       25355
Probable Cause           4000
Reasonable Suspicion     1732
Consensual Encounter      805
Probation/Parole          677
Name: ReasonForEncounter, dtype: int64

As seen above, there are two rows for traffic violations because of the capitalization. The next cell will make the capitalization uniform for the “ReasonForEncounter” column

/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:1: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  """Entry point for launching an IPython kernel.





traffic violation       25355
probable cause           4000
reasonable suspicion     1732
consensual encounter      805
probation/parole          677
Name: ReasonForEncounter, dtype: int64

Traffic violations are the most commonly occuring reasons for encounter. This could be because there are so many cars in Oakland and traffic violations are a pretty common occurence. It is not considered a very big offense and a lot of people violate the complex traffic laws (sometimes unknowingly).

png

The graph above reiterates the same information being conveyed through the table above. From the graph you can notice the steep decline in reported encounters from 25K to 5K.The more serious the reason for an encounter, the fewer the number of encounters. This is intuitively corrent because the more serious the crime, the more serious the punishment therefore this negative correlation is justified.

		ReasonForEncounter
Beat	ReasonForEncounter
01X	traffic violation	165
	probable cause	48
	consensual encounter	12
	reasonable suspicion	5
	probation/parole	1
02X	traffic violation	814
	probable cause	112
	reasonable suspicion	39
	consensual encounter	24
	probation/parole	19
02Y	traffic violation	409
	probable cause	71
	reasonable suspicion	19
	probation/parole	14
	consensual encounter	11
03X	traffic violation	570
	probable cause	116
	reasonable suspicion	35
	consensual encounter	12
	probation/parole	3
03Y	traffic violation	254
	probable cause	64
	consensual encounter	12
	reasonable suspicion	11
	probation/parole	8
04X	traffic violation	694
	probable cause	118
	reasonable suspicion	41
	consensual encounter	13
	probation/parole	8

The most occuring reason for encounter is traffic violation throughout all the beats. Moreover, most beats follow the same trend when it comes to the frequency and type of encounters. The number of traffic violations is relatively higher than the rest of the type of encounters.

	ReasonForEncounter
Beat
34X	2006
23X	1753
30X	1720
19X	1573
06X	1357

As you can seee from the dataframe above, the highest amount of crime occurs in 34X followed by 19X, 30X and 23X, where as beats 25Y has only 67 enocunters. The next step is to futher analyse if there is a correlation between reason for encounter income. In order to do that we will compare the income levels of each beat to the type of crime that occurs in that beat.

	Avg Median Household Income
Count	-0.579247
Avg Median Household Income	1.000000
consensual encounter	0.385981
probable cause	-0.087421
probation/parole	-0.095108
reasonable suspicion	-0.042510
traffic violation	-0.077835

The table above shows the correlation coefficients for each type of encounter and the household incomes. From the table above you can see that there is no strong correlation between these two features. Additionally, we could not control for any factors in this regression therefore we cannot establish causation either.

7 Modeling

7.1 What models are we using, and why?

We chose to use three regression models: Linear, LASSO, and Ridge models. The reason we have chosen to use regression is because regression is useful in scenarios where the value varies on a continuous scale from several features; in this case, the value of stops per beat from the features we selected prior (housing units per beat, education levels, number of married people, and languages spoken). Regression gives us the ability to predict from all of these features at once and develop visualizations of the performance of each model.

Housing units per beat were selected as a feature since increased density of housing units, particularly those that are detached single unit homes, may signify a lower amount of stops. We inferred this could be because such housing units are more common in higher income and lower crime neighborhoods in Oakland, as opposed to areas with more density in housing and thus more density in individuals and households.

Education levels were another factor that we felt could influence the amount of stops in a beat. Beats with lower levels of education, likely caused by wealth disparities and underperforming and underfunded schools could influence implicit bias in policing, and thus cause a greater number of stops.

In terms of the number of married people as a feature, we felt beats with lower counts of married people could contribute to higher stops. Family stability may factor into a propensity for a neighborhood to have higher rates of crime and/or policing versus neighborhoods with higher amounts of married people.

Languages spoken can also factor into a neighborhood’s amount of policing and thus amount of stops. Implicit bias in the institutions of justice and policing cause racial bias against non-English speakers. Language spoken, especially a language other than English, may cause racially biased behavior to occur on the part of the police officer. For example, profiling on the basis that an individual is speaking another language may act on the underlying bias of a police officer to assume suspicion and stop the individual. This can be qualified as a situational risk factor for racially biased behavior (NASEM 253). Police officers forced to react in a situation where they believe a crime is taking place may lead them to act on their implicit biases, which can be based on statistical prediction (believing membership to a particular group creates greater likelihood of criminality) or pre-existing bias against a particular racial or ethnic group (known as racial animus) (NASEM 253).

7.2 Import Packages

7.3 Creating the Training, Test, and Validation Sets

7.4 LASSO Regression Model

/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_coordinate_descent.py:476: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 12337970.397719575, tolerance: 6269.840071875
  positive)

png

05789564930222735

0129203861065

5422742724676747

The LASSO model has a root mean squared error of 941.698, and an r2 score of 0.54868. The score for the model is very low, at -0.01885, indicating the model does not perform well.

7.5 Ridge Model

png

-0.08360700107105545

970.2110920526445

0.5195745716758682

The Ridge model has a root mean square error of 966.537, and an r2 score of 0.52320. While the r2 score is lower than the LASSO model, the rmse is greater. The score for the model is higher than LASSO’s score, at -0.13619.

7.6 OLS Model

png

-0.2619380775923914

889.3851921737518

0.5962864706007645

The linear model performs the best out of the three, with the lowest RMSE of the three at 878.798. The r2 score is also the highest of the three models, and its score is highest at -0.45261.

7.7 AIC and BIC Information Criterion Approaches

Calculating the AIC and BIC for each model gives us another set of indicators for which model we should prefer. AIC and BIC differ by how they penalize the number of parameters our model has.

11

32

AIC

-12.344722751936025

-12.093593520426552

-12.441526479576709

Based on the AIC, we should prefer the linear model since its score is the lowest.

BIC

7351100721822

2839697144331

71704236803055

Based on the BIC, we should prefer the linear model, since its score is the lowest.

##7.8 Tune the model on the Validation set.

8294852584764
0078757442892
8377397551568

009827154659

-5.185048296942249

Based on the validation set, which is unbiased since it was not fit to the training data, the LASSO model performs the best. The RMSE on the LASSO model and the validation set is the lowest. The RMSE for the LASSO model and the test data is 2182.864002, and the r2 score is higher at -5.6954753.

7.9 Cross Validation

Using cross-validation avoids the issue of overfitting the model to the training data, which is found when doing the train-test-validation split. It is also optimal for small data sets such as ours to use cross-validation.

LASSO Cross-Validation

/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_coordinate_descent.py:476: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 12861921.19848317, tolerance: 6777.242675
  positive)
/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_coordinate_descent.py:476: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 7847536.395470062, tolerance: 4543.069097222223
  positive)
/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_coordinate_descent.py:476: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 11569613.45276578, tolerance: 4979.738255555555
  positive)

0.22054192637707537

png

Ridge Cross-Validation

0.23379352417585209

png

Linear Cross Validation

0.08097854311459352

png

Out of the three cross-validations we ran on the models, Ridge regression appears to perform the best, with the highest r2 score at 0.228. Linear regression has the lowest, at 0.0811. LASSO is in the middle at 0.215.

7.10 Modeling Results

The model that performed the best on the most rigorous, unbiased form of testing (cross-validation) is the Ridge model, and therefore the model that we feel can predict the number of stops for a beat best for our data. Their performance is generally not too far off from one another; we tried multiple approaches and none of them resulted in a low enough error to where we’d be able to make a super accurate prediction about the data.

Just as a way to see how our features were performing for our prediction, we created a correlation matrix of all of the features on our ‘features_final’ table. As a threshold, those features with correlation values higher than 0.5 were stronger predictors of stops in our model. We felt it necessary to include all of the features in our original model, since we wanted to explore how all of them influenced the number of stops.

/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing as tm

	Household Income by Race	1, detached	1, attached	Mobile home	Boat, RV, van, etc	2.0+	High School or less	Associates/Bachelor Degrees	Masters/Professional/Doctorate Degrees	Speak Non-English language Cnt	Total Married	Count
Household Income by Race	1.000000	0.294009	-0.262967	-0.153458	-0.174559	-0.256025	-0.649825	0.348535	0.764901	-0.580317	0.537695	-0.475726
1, detached	0.294009	1.000000	-0.136557	-0.249121	-0.028812	-0.990653	0.002407	-0.146259	0.117859	-0.200127	0.557928	-0.304095
1, attached	-0.262967	-0.136557	1.000000	-0.085952	-0.018752	0.002660	0.101897	0.060249	-0.214969	0.037716	-0.283799	0.135529
Mobile home	-0.153458	-0.249121	-0.085952	1.000000	-0.082537	0.241819	0.295883	-0.255140	-0.268772	0.396339	-0.083641	0.120465
Boat, RV, van, etc	-0.174559	-0.028812	-0.018752	-0.082537	1.000000	0.019823	0.063548	0.043097	-0.137709	0.067393	-0.011393	0.139398
2.0+	-0.256025	-0.990653	0.002660	0.241819	0.019823	1.000000	-0.024647	0.145858	-0.081504	0.187501	-0.525435	0.285106
High School or less	-0.649825	0.002407	0.101897	0.295883	0.063548	-0.024647	1.000000	-0.862184	-0.908103	0.869174	-0.294612	0.538243
Associates/Bachelor Degrees	0.348535	-0.146259	0.060249	-0.255140	0.043097	0.145858	-0.862184	1.000000	0.570826	-0.828943	-0.078829	-0.337832
Masters/Professional/Doctorate Degrees	0.764901	0.117859	-0.214969	-0.268772	-0.137709	-0.081504	-0.908103	0.570826	1.000000	-0.723535	0.543464	-0.593181
Speak Non-English language Cnt	-0.580317	-0.200127	0.037716	0.396339	0.067393	0.187501	0.869174	-0.828943	-0.723535	1.000000	-0.128165	0.511770
Total Married	0.537695	0.557928	-0.283799	-0.083641	-0.011393	-0.525435	-0.294612	-0.078829	0.543464	-0.128165	1.000000	-0.471643
Count	-0.475726	-0.304095	0.135529	0.120465	0.139398	0.285106	0.538243	-0.337832	-0.593181	0.511770	-0.471643	1.000000

png

From the matrix above, the values that perform the best as predictors are the count of Non-English Language Speakers, and High School or Less as a level of education. While language does not correlate directly to the frequency of stops in a beat, this still could suggest that non-English speaking communities are being adversely affected by increased police presence. Additionally, in a place like Oakland where the school to prison pipeline is a prominent feature of educational institutions, many students are not encouraged to strive for higher education (Nance 2016). Disciplinary action on behalf of law enforcement becomes a first resort for school administration rather than dealing with disciplinary infractions internally (Nance 2016). Once students are pushed into this system, pursuing higher education becomes difficult due to the lack of academic support and increased intervention of law enforcement (Nance 2016). This can increase the likelihood of criminal behavior for individuals who have a high school education or less (Nance 2016).

8 Conclusion

Initially, we believed that higher median household income police beats lead to more lenient outcomes, more justified stops, and less felony and misdemeanor arrests. We also hypothesized that lower income police beats experienced higher frequencies of stops, and harsher outcomes during these stops. This was based on our belief that gentrifying, higher income neighborhoods would experience more policing as a result of being in formerly lower income neighborhoods in Oakland. After exploring the data from the original OPD dataset, we found that as household income increased, the overall occurrence of stops, and various causes and outcomes of these stops decreased. This leads us to believe that higher income police beats in Oakland experience less policing overall than lower income beats. Our dataset did not include reported crimes, and only stops, which may be why the analysis appears this way. Looking to the models we developed, our Ridge model was the highest performing in terms of using our features to predict the number of stops per beat. This model can be useful if analyzing how different factors may inform unconscious bias when it comes to policing and conducting stops in particular neighborhoods. With the knowledge that language spoken (non-English) and education level are the best indicators of the number of stops conducted, we speculate that more rigorous bias training is necessary for the OPD when they make decisions to stop particular individuals. More overarchingly, the factor of education as a predictor of the number of stops points to remedying the school to prison pipeline within Oakland schools.

9 Further Areas of Exploration and Analysis

Future research that can be derived from our research can explore how education levels per police beat correlate to stops, searches, and their outcomes. Since our feature correlation matrix showed education levels as a stronger predictor of stops per beat, we feel that this is an area that can be further explored in conjunction with the available OPD dataset. Additionally, it may be pertinent to explore other factors like rates of gentrification to see how that change over time has changed stops and searches in particular beats, and relate this to existing racial and income data. This research may yield important results in terms of seeing how policing is disproportionately affecting particular races, and if this is exacerbated by gentrification. In addition, our study would be improved if our police stop data was recorded using census tracts instead of police beats, since our method of merging the two together resulted in a loss of overall accuracy.

10 Works Cited

Fayyad, A. (2018, January 4). The Criminalization of Gentrifying Neighborhoods. Retrieved from https://www.theatlantic.com/politics/archive/2017/12/the-criminalization-of-gentrifying-neighborhoods/548837/

Montojo, N. (2018, December 12). Understanding Rising Inequality and Displacement in Oakland. Retrieved from https://www.kcet.org/shows/city-rising/understanding-rising-inequality-and-displacement-in-oakland

Nance, J. (2016) Dismantling the School-to-Prison Pipeline: Tools for Change. Retrieved from http://scholarship.law.ufl.edu/facultypub/767

National Academies of Sciences, Engineering, and Medicine (NASEM). 2018. Proactive Policing: Effects on Crime and Communities. Washington, DC: The National Academies Press. https://doi.org/10.17226/24928.