The role of data in criminal justice
Summary
TLDRThe discussion centers around the growing role of data-driven systems in the criminal justice system and their limitations. While data is often seen as objective and a solution to human bias, the speakers argue that it frequently reflects existing racial biases and systemic inequalities. They raise concerns about the use of predictive tools and risk assessments, which may reinforce past injustices rather than address the underlying issues. The conversation emphasizes the need for a paradigm shift, focusing on social justice interventions instead of relying solely on technology to predict or prevent crime.
Takeaways
- 🧑⚖️ Data-driven systems in criminal justice are viewed with skepticism due to concerns that they replicate systemic biases, particularly racism and inequality inherent in historical data.
- 📊 Data used in criminal justice often reflects enforcement patterns rather than actual crime rates, leading to distorted outcomes when predicting crime or behavior.
- 🤔 Many believe that data is objective, but it often fails to capture unreported crimes or the nuanced reasons behind certain behaviors, leading to unreliable predictions.
- 🔍 There's a need for a paradigm shift in criminal justice—away from predictive models that attempt to foresee human behavior and towards addressing root causes of crime through social interventions.
- 🚫 Critics argue that current data-driven tools focus too much on individuals caught in the system rather than on those perpetuating it, such as police officers, judges, and prosecutors.
- 🪞 Algorithmic tools are often seen as mirrors reflecting existing biases rather than correcting them, leading to recommendations that preserve the status quo rather than drive meaningful change.
- ⚖️ The rapid development of technology often outpaces policy reform, making it difficult for legal systems to adapt to new tools and ensure that they are used in ways that are fair and effective.
- 📉 There’s growing advocacy for shifting focus from prediction to intervention, using data to test what measures actually help reduce crime and recidivism rather than just forecasting risk.
- 💡 Social justice approaches, such as investing in community resources and decriminalization, are seen as more effective than criminal justice responses in addressing the root causes of crime.
- 🔄 Concepts like pretrial risk and danger are often ill-defined, subjective, and legally recent, complicating the fair use of data in criminal justice decision-making.
Q & A
What is the main concern about data-driven systems in the criminal justice system?
-The main concern is that data used in criminal justice may be tainted by the country’s history of racism and inequality, potentially leading to biased outcomes and perpetuating systemic injustices.
Why do people trust data-driven systems despite these concerns?
-People tend to trust data-driven systems because they are perceived as more objective and less prone to human error, which can improve decision-making in some aspects of criminal justice.
What is a key misconception about data in the criminal justice system?
-A key misconception is that the data is purely objective. In reality, it often reflects enforcement patterns, not actual crime rates, and can be influenced by biases in policing.
How can domestic violence data be unreliable for criminal justice systems?
-Domestic violence is often underreported for various reasons, including fear of police or social stigma, leading to unreliable data that doesn't accurately represent the true extent of the issue.
What is the 'garbage in, garbage out' problem mentioned in the context of predictive tools?
-This refers to the idea that if biased or flawed data is used in a predictive model, the results will also be flawed and perpetuate existing inequalities.
Why is it difficult to ‘scrub’ data of racial bias?
-Racial bias and inequality are deeply woven into many aspects of American society, such as housing, education, and law enforcement, making it challenging, if not impossible, to completely remove racial influence from the data.
What alternative uses of data tools were suggested by the speakers?
-The speakers suggest using data tools to assess and modify the behavior of actors within the justice system, such as police officers, judges, and prosecutors, rather than predicting the behavior of individuals.
Why do the speakers doubt the ability of predictive tools to improve the criminal justice system?
-They believe that predictive tools often fail to understand the complexity of human behavior and risk, and may over-rely on flawed data that reinforces existing biases in the system.
What is the role of prediction in criminal justice, and why is it problematic?
-Prediction is used to foresee future actions, like the likelihood of reoffending, but it is problematic because it is based on biased historical data that may not accurately reflect individual or community realities.
What is a more forward-looking approach to criminal justice reform mentioned in the discussion?
-A forward-looking approach involves using causal inference methods to evaluate the effectiveness of interventions, rather than relying on predictive tools that reflect past enforcement patterns. This would focus on identifying what policies and changes actually work to improve outcomes.
Outlines
💡 The Role of Data in Criminal Justice: Risks and Realities
This paragraph discusses the evolution of data usage in the criminal justice system, highlighting how data-driven systems are trusted for their supposed objectivity. However, concerns arise over the historical biases embedded in data, such as racism, which could lead to flawed outcomes. The speakers express skepticism about the effectiveness of data-driven decision-making, emphasizing that biased data only perpetuates systemic inequalities. The conversation points out that while awareness is increasing, the fundamental issue lies in the data's flawed origins.
🔍 Misconceptions About Data Objectivity
The misconception that data is inherently objective is addressed. The speakers argue that criminal justice data reflects enforcement patterns rather than actual crime rates. For instance, underreporting of crimes like domestic violence skews the data, making predictive models unreliable. The paragraph explores how different groups face different arrest patterns, leading to biased outcomes. It also questions whether these biases can be overcome, suggesting that certain crimes might show less disparity but that systemic issues remain prevalent.
🤖 The Complexity of Human Behavior and Prediction Tools
Here, the focus shifts to the challenges of predicting human behavior using machines. Speakers share personal experiences with unpredictable court appearances and emphasize the limits of data in capturing the nuances of human behavior. They criticize the over-reliance on tools that attempt to predict outcomes and argue that human unpredictability makes such predictions flawed. The paragraph advocates for a paradigm shift, suggesting that trying to fix a broken system with these tools misses the point.
⚖️ Objectivity in Criminal Justice: A False Promise?
This section delves deeper into the perceived objectivity of criminal justice tools, suggesting that they often reinforce existing biases rather than offering real solutions. The speakers argue that relying on tools to make decisions without understanding the underlying human factors leads to shallow assessments. They criticize pre-trial risk tools for overlooking important variables, like personal hardships, that affect court appearances. They also touch on how slow policy reforms struggle to keep pace with rapid technological advancements.
🔄 Historical Data and the Challenges of Policy Reform
Technological advancements in criminal justice are often based on historical data that doesn't reflect new policies or positive changes. This paragraph examines how counties and cities lack the resources to properly implement data science, leading to outdated or inaccurate predictions. For example, even though text reminders can reduce court no-shows, predictive tools still use old data that doesn't account for these improvements. The speakers stress the need for forward-thinking solutions that break away from reliance on past data.
🔬 Moving Toward Interventions Instead of Predictions
The conversation here shifts to the idea of 'interventions over predictions,' where the focus is on determining which interventions work best rather than using predictive models to forecast criminal behavior. The speakers propose using new statistical techniques to create data on the effectiveness of interventions, such as whether a program helps someone avoid future interaction with the justice system. They also advocate for investing in social services like education and job creation in neighborhoods rather than increasing police presence.
📈 Reassessing Risk: A Call for Systemic Change
The speakers explore the broader issue of how concepts like 'risk' and 'danger' are applied in the justice system. They argue that these terms are often vague and subjective, complicating the use of data-driven tools. The discussion questions the validity of risk assessments and whether they can effectively predict future behavior. The speakers suggest that risk is a fundamental part of life and that trying to eliminate it entirely from the criminal justice system is impractical.
📚 Using Science for Sentencing and Decriminalization
This paragraph highlights how science and data can be useful for reforms, such as decriminalization, sentence reduction, and early release. The speakers advocate for using evidence-based practices that focus on reducing mass incarceration and improving rehabilitation efforts. They point out that while positive data exists, it is often ignored in favor of punitive measures. The focus should be on systemic change that prioritizes alternatives to incarceration, especially for minor offenses and non-violent crimes.
🧑⚖️ Addiction and Criminalization: Rethinking Rehabilitation
The final paragraph discusses how addiction is criminalized instead of being treated as a health issue. The speakers reflect on the idea that repeated attempts at rehabilitation, like trying to quit smoking, should be normalized rather than punished. They suggest that risk assessments could be recalibrated to measure the negative impact of pretrial detention on individuals. The goal is to create a system that incorporates restorative practices, addressing root causes like addiction rather than funneling people into the criminal justice system.
Mindmap
Keywords
💡Data-driven systems
💡Bias in data
💡Risk assessment tools
💡Predictive policing
💡Racial bias
💡Pre-trial detention
💡Objective data
💡Causal inference methods
💡Social justice response
💡Addiction and criminalization
Highlights
The role of data in the criminal justice system has evolved, with a growing skepticism towards big data as it is applied to justice.
Concerns are raised about data-driven decision-making replicating historical biases, particularly racism and inequality embedded in the data.
Data in criminal justice is often misunderstood as objective, but it is shaped by enforcement patterns rather than representing actual crime.
Predictive policing tools, like pre-trial risk assessments, rely on data that can be statistically unreliable, such as underreported crimes like domestic violence.
Algorithms used in the criminal justice system are criticized for simply mirroring existing biases rather than providing objective predictions.
There is a call to shift focus from predicting criminal behavior to addressing the behaviors of police officers, judges, and attorneys to reduce biases.
Risk assessment tools often ignore important contextual factors, such as economic hardship or past experiences, that affect court appearances.
There is a disconnect between the rapid evolution of technology and the slow pace of policy reform, making it challenging to ensure fair application.
Interventions over predictions: Instead of predicting risks, some suggest focusing on interventions that address underlying issues, such as lack of resources or support.
Social justice responses, like providing more jobs or schools, are proposed as alternatives to using predictive tools to address crime in neighborhoods.
Terms like 'risk' and 'danger' are often ill-defined in the criminal justice system, complicating the use of data in risk assessment tools.
There is skepticism about whether any risk assessment tools, as currently designed, can effectively improve safety or justice outcomes.
Data showing positive outcomes from decriminalization or diversion from the criminal justice system is often underutilized in shaping policy.
Reframing risk assessments to consider the impact of detention on individuals’ lives, rather than just predicting future crimes, is suggested.
A paradigm shift is needed in criminal justice, moving away from efficiency-focused reforms to more fundamental changes that prioritize social equity.
Transcripts
thank you both for being here thank you
thank you as we've heard a bunch today
the role of data in the criminal justice
system has evolved a lot in the last
decade people tend to trust data-driven
systems they remove supposedly some of
the human errors that have you know laid
waste to the best parts of the criminal
justice system how are you seeing folks
balance the desire to combat you know
this injustice in the system with the
growing skepticism of big data as it's
applied to criminal justice so I mean I
think you know fundamentally one of the
problems with this kind of drive towards
data-driven and kind of algorithmic
decision-making is you know the concern
that the data that people are relying on
is infected with this country's history
of racism and inequality and you know
essentially what you're gonna do what
you're doing is replicating that
terrible history by using data that is
essentially garbage and kind of putting
it through a machine and getting garbage
out and getting garbage predictions and
so I think that's the kind of
fundamental problem I think people
becoming more and more aware of that
problem as there's been more of an
effort to rely on these types of tools
to try and help and improve criminalists
decision-making I think that that part
of it kind of the acknowledgment
understanding is a really good thing and
a positive development but I I remain
kind of incredibly skeptical about the
usefulness of using these approaches
Logan what are some of the the
misconceptions as you're looking at you
know data being heralded as one of these
really positive things for the criminal
justice system what are some
misconceptions about the way that it's
actually applied so I think one helpful
thing is I think frequently we think
that data is objective and it represents
the sort of like natural existence of
something occurring in the world but of
course when we're talking about the
criminal justice system the criminal
justice system you know criminologists
have long studied this and said that
really this underlying data is
representative of enforcement patterns
it's not necessarily representative of
quote-unquote underlying crime data in
the actual commission of crime and we
have to remember that this is
representative of how police officers
are responding to things in their
community and I think that's a
fundamental disconnect
like frequently I'm talking with
different individuals who are saying hey
if I want to adopt something like a
pre-trial risk assessment tool or
predictive policing system this is just
sort of objective data that's a rest
level data but of course we know that
different things are arrested at
different patterns for different groups
for example we know that domestic
violence is often not reported and so if
you're going to build a model that was
going to forecast where domestic
violence crimes might be committed or
forecast you'd have statistically
unreliable data simply because that's
not reported and you can go into various
different reasons why that might not be
reported for you know fear of the police
fear of reporting many different things
but I think there's just just this myth
that this this crime day is inherently
objective in some way can can that be
overcome i I think it can be a big
question I don't know if it can
necessarily be overcome because in some
sense all police departments will
inherently be constrained right there
going to be some number of units of
officers who can only respond to some
number of things right I think one thing
to think about is potentially there are
maybe some crimes that don't show sort
of underlying disparities and arrests
level data that communities may be truly
are shown to truly care about where
they're struggling with maybe saying
domestic violence or aggravated assault
things like that but I don't think I see
many strategies that are improving that
today yeah you know I'm not sure if we
can be able to come I don't think that
it can actually I think we can confront
it I think we may be able to account for
I'm not a kind of a data scientist or or
a technologist in that way but I think
there may be ways to try and try and
account for some measure of where have
the racism or the bias it's kind of
influencing the data but I think
fundamentally there's so much inequality
and so much racial bias woven into the
DNA of this country that kind of
whatever system you tie it back to
they're gonna have some racial impact
right and so and of what is whether
you're looking at housing education
employment age at first arrest where
police are deployed all those things are
tied in to race and so I think in order
to kind of scrub the data so to speak of
the racism I think is amazing
that'll be difficult if not impossible
task at least my view so how do we
account for it well you know my thing is
we should be just stop using these tools
you know we kind of go in a different
direction and and fundamentally right
now all the tools are being pointed at
the people who are being consumed by
these systems I would like to see tools
that are aimed at police officers judges
prosecutors defense attorneys
determinedly you know you know if we if
we have a concern about trying to change
the system itself and the behavior of
actors in the system why don't we trying
to change the behaviors of those who are
kind of replicating some of the biases
we see already why why are we trying to
trying to predict what people might do
the reality is as a public defender I
you know I there are some people who I
knew would come back to court some
people I thought would definitely come
back to court without without any
problem some people I thought would
never show up again and you know I was
surprised regularly about what would
happen you know the human condition is
really difficult to predict I think the
idea that we voted it better with
machines fundamentally misunderstands
how people work I see you nodding your
head yeah on that point I think I think
something that's been interesting is
sort of when we think about sort of big
data or algorithms a lot of this is
focused on prediction generally and of
course there's sort of prediction is you
know ubiquitous throughout the criminal
justice system and you know individuals
will be predicting things on their own
but I think hopefully many people did
not need this reminder but as new tools
have been developed you know you have
sort of wrote more actuarial base
regression statistics tools and then you
have more sort of machine learning based
tools I think as you sort of try and
refine this along the path what you're
really doing there's a great new paper
by sandy Mason
that'll be published in the Yale Law
Journal that basically says what we're
doing is really just sort of putting up
a mirror that is just more and more
accurate and the solution isn't so been
the mirror to your liking so it's not to
account for like okay how can we adjust
the model like what matters is what
you're seeing in the mirror right and
that is the underlying thing that you
should really be trying to adjust for
yeah I mean I think that can I guess is
a fundamental point that's what we need
to do is have kind of a paradigm shift
about the way we do criminal justice I'm
not kind of like taking around edge
and making a bad system work a little
bit more efficiently I think that's what
these tools often purport to do yeah it
sounds like you know both of you have
brought up that basically these
technologies are just kind of preserving
the same system under the guise of
scientific objectivity talk about how
that perceived objectivity is kind of
stifles progress well I mean I think in
many ways there's this sense that if
you're if you're kind of relying on a
tool and tool tells you that this
individual mate may or may not do this
this particular thing
you're never gonna kind of do any kind
of deeper inquiry into what the kind of
human condition is about and so you're
never gonna find out like so for example
I think we think about a risk assessment
tool that's designed to figure out
someone's gonna kind of appear in court
or not the type of data they were
usually looking at are kind of a two
first arrest prior warrant history the
things that made that they may think
correlate with fears to appear but there
are other data points that that are
never kind of examined you may not
return to court because you just don't
have the resources to get back to court
you may not return to court cuz they
have a terrible experience with court
before and you're worried about what's
gonna happen you may not return to court
because you're you have family
circumstances or other things that may
have may have interfere with your
ability get back to court a data point
that's not kind of considered at all is
never gonna be measured and never gonna
be part of your range of considerations
so I think that's that's problematic in
and of itself Logan we were talking a
little bit backstage about kind of the
slow moving policy reform how quickly
and rapidly technology changes and how
that you know the dissonance between
those timelines makes it really
difficult for policy to catch up talk a
little bit about that yeah so I think
one sort of interesting paradox is sort
of if you look at sort of the current
movement of bail reform as its
conceptualized with certain policymakers
I think there's a sort of unanimous
understanding that money bail system
needs to go away and a lot of
policymakers are looking towards risk
assessment tools to sort of reform away
an unjust system and along the way
they're adopting good policy so money
bail that means you know if you are a
person who can't pay $500 to get out of
jail there are plenty of studies of the
show that your risk of RIA rest will be
increased because
and the 72 hours you've been detained
your life might have fallen apart you
could lose your job we discussed you of
your children things like that there are
other important policies like text
reminders can reduce failure to appear a
lot but the problem is when we implement
these new technologies they're
inherently looking to the Past so if
you're looking to something that doesn't
incorporate the actual new beneficial
policies you're just going to be
forecasting the likelihood of someone's
risk based on to the previous system so
it's really hard to generate these
positive feedback loops because
frequently these counties and cities
don't have the budget to do
quote-unquote data science properly and
it's it's a it's a really hard task to
get right and I think that's an
underappreciated element in the current
debate so how do we you know cast the
our lenses forward what are the
challenges in trying to do something
that's forward-looking instead of
something that uses kind of historical
data that doesn't work so one thing
that's interesting is there's a great
paper that sort of sort of trying to
reframe this entire debate it's called
interventions over predictions and
really what its postulating is instead
of looking at the sort of risk
assessment stuff that's trying to look
at historical data to project things for
it it's saying let's look at something
called causal inference methods so
that's basically trying to do new
statistical techniques where you're
trying to say what invent interventions
really actually work so by doing that
you're creating new data that can
actually say okay this actual new change
help someone get back to court this new
change help this class of defendants
never get touched by the criminal
justice system again and I think that's
one thing that's sort of more
forward-looking and to me I mean to me
that sounds a little bit kind of like a
like more of a needs assessment
intervention than anything else and so
like you know we think about pretty good
policing and the notion that is gonna
predict where where a crime might take
place or who might be involved in crime
rather than send if you think a
neighborhood is gonna be the site of
some criminal activity why are we
sending police officers into that
neighborhood why don't we why don't we
figure out like okay that neighborhood
needs more schools and there needs more
jobs than neighborhood to do social
no to me I'm oh I'm always struck by the
fact that our response to some of these
inequities as a criminal justice
response rather than kind of a social
justice response rather than a response
that actually lifts up people and helps
them avoid capacity now and so I you
know
Hardwell what I can see is that type of
methodology instilled as well absolutely
before I take it to the audience for a
question I wanted to ask one bigger
broader question we talked about
pretrial risk tools and predictive
police say and we're often using these
big terms like risk and danger and
they're often ill-defined and
subjectively applied how does that
complicate the use of data obviously it
sounds like you both think that that
neither of those tools are a step in the
right direction but how does one even
begin to calculate a tolerable amount of
risk in a community or a tolerable
amount of danger I would just note that
when we think about bail it wasn't legal
necessarily for judges to predict future
dangerousness until about 1987 when the
Supreme Court codified it so this is a
new sort of legal regime that we've
established so there's nothing
necessarily I mean we can have a
spirited debate about the underlying
sort of Court's decisions leading to the
predictions of future dangerousness but
I just I I just sort of question how I I
mean from one perspective we have sort
of REO rests data standing in for the
proposition of sort of public safety but
so many things that people are really
say nothing about violence to a
community or Public Safety at all and I
I just colored me skeptical that any
risk assessment tools as sort of
currently proposed and implemented will
sort of succeed in in their hands and
and you know I'll just say very quickly
I think fundamentally risk is part of
life you know I think you can't we could
walk outside get hit by bus you know but
you don't stop crossing the street
because of that like that's that's part
like risk is what the promotion system
has to accept as part of the cost of
doing business and so I think notion
will be able to predict who's riskier
who's not to me just doesn't it just
doesn't kind of really make any sense
do we have a question in the audience I
think we have one down here in the front
hi it's me again
Vivian mixer so I want to ask what roles
do you think using science that actually
would help like science about sentencing
reform and decriminalization to do to to
eliminate some of the problems we're
facing so I mean I think there there is
like - except you're looking at data and
think about science I think so you're
holding up kind of good results from
from things like the criminalization
from things like the incarcerating
places from things like releasing people
early competin Singh from sentences from
diverting people from the criminal
justice and I think that data for some
reason is never kind of the stuff that
people cite or think about we're
thinking about kind of evidence-based
practices right and so I think those are
all data points that would be very
helpful in informing kind of
decision-making about the way in which
we should kind of shape our criminal
justice policy more generally I'm all
for using data in that way I think to me
it's just problematic when you're
deciding how long a sentence somebody
for or if we gonna release them on
parole even in the bill context I mean I
think there's a place to say here's an
entire class of people and by the way
the vast majority people who are
released from from from pretrial
incarceration return to court so it's
not as though we have people kind of
just skipping court willy-nilly all the
time but here's an entire population of
folks or a cohort folks who can be
released without ever even going through
the court system at all and so I think
there are probably ways you can use it
in a positive way but unfortunately
those don't seem to be the wins that are
kind of advanced that often
hi annoying umm so really quickly Thank
You number one I just read this book by
James Forman jr. called locking up our
own which was amazing if you guys
haven't read it but he really just talks
about how these innate community
barriers that should be more of a human
services issue have been criminalized so
when you deal with things like addiction
we've turned addiction into a
criminalization so when you have someone
who commits crimes as a result of their
addiction we're we want to lock them up
and instead of sending them back to
rehab and his whole statement was why
don't we view why don't we view rehab
the same way that we view put the prison
system like the first time you try to
quit smoking you're probably not going
to get it so why don't we send them back
there as opposed to putting them into
prison because they didn't kick the
habit the first time so my question is
in what way if possible do you think
that we can start to it could possibly
incorporate these community conditions
and these restorative types of practices
so that everyone who suffers from
everyone who's not just a hardened
criminal isn't caught up in that
whirlpool if you live I think one
interesting idea to kind of combine both
questions is I think you could rican
septa lies risk assessments to include
another factor so for example the risk
of any pretrial detention greater than
24 hours to that person what impact
would that have on their life wasn't it
you know we know of all these sort of
negative outcomes actually sort of
formalizing that for the process could
really I think recalibrate and get us
closer at motive of course when we're
thinking of addiction you know your
first time trying to struggle through
that it's not necessarily going to sort
of latch your first time you have to try
and try and try I think sort of maybe
potentially having this sort of if we
were to put this person inside the
system what how would that impact them
in negative way I think that that might
be one potential solution Vincent Logan
thank you so much thank you
[Applause]
Ver Más Videos Relacionados
5.0 / 5 (0 votes)