Anne Milgram: Why smart statistics are the key to fighting crime

Summary

The speaker, a former attorney general, recounts their journey to reform the criminal justice system through data-driven approaches. They highlight the inefficiencies of relying on instinct and experience, instead advocating for a 'moneyball' strategy that uses data analytics to assess risk and guide decisions. The development of a universal risk assessment tool, backed by a vast dataset, aims to help judges make more objective decisions about pretrial detention, ultimately aiming to enhance public safety, reduce costs, and improve fairness in the justice system.

Takeaways

• 📚 The speaker began as the attorney general of New Jersey, with a background in criminal prosecution.
• 🔍 Upon taking office, the speaker found a lack of data on who was being arrested and charged, and how decisions were made for public safety.
• 😤 Frustration led to a realization that most criminal justice agencies did not track critical information effectively.
• 👮‍♂️ A day spent in Camden's police department highlighted the absence of data-driven policing, with crime-fighting relying on rudimentary methods.
• 🚨 The speaker recognized a systemic failure in the criminal justice system, with a lack of data and analytics in decision-making.
• 🏈 The concept of 'moneyballing' was introduced, drawing a parallel between data-driven sports management and criminal justice reform.
• 📉 In New Jersey, data and analytics were successfully applied, leading to a significant reduction in crime rates.
• 🏛️ The speaker's work at the Arnold Foundation focused on improving the critical decision of whether to detain or release an arrested individual.
• 📊 A universal risk assessment tool was developed, utilizing a vast dataset to predict recidivism and the likelihood of violence.
• 🌐 The tool aims to provide judges with objective risk assessments, complementing their instinct and experience for better decision-making.
• 📈 The speaker's vision is for all U.S. judges to use data-driven risk tools, transforming the criminal justice system into one based on data and analytics.

Q & A

What significant change did the speaker experience when they became the attorney general of New Jersey?

The speaker experienced a significant change in perspective when they became the attorney general of New Jersey. They realized that the criminal justice system was failing due to a lack of data tracking and analytics, which led them to introduce data-driven decision-making into the system.

Why were the initial efforts to gather data on criminal cases in New Jersey frustrating for the speaker?

The initial efforts to gather data on criminal cases were frustrating because the speaker found that most big criminal justice agencies, including their own, did not track the necessary information. Detectives had to manually go through case files case by case, which was time-consuming and inefficient.

What was the impact of the speaker's changes in the Camden Police Department?

The impact of the speaker's changes in the Camden Police Department was significant. They reduced murders by 41 percent, which equates to saving 37 lives, and reduced all crime in the city by 26 percent.

What is 'moneyballing' in the context of the criminal justice system as described by the speaker?

In the context of the criminal justice system, 'moneyballing' refers to the use of data and analytics to make more informed decisions, similar to how the Oakland A's used smart data and statistics to pick players. The speaker aimed to apply this approach to criminal justice to improve decision-making and outcomes.

How did the speaker's approach to criminal justice differ from traditional methods?

The speaker's approach differed from traditional methods by focusing on data-driven decision-making, using analytics and statistical analysis to determine the risk posed by individuals in the criminal justice system. This was in contrast to the traditional reliance on instinct and experience.

What was the problem the speaker identified with the current bail system in the United States?

The speaker identified that the current bail system in the United States often leads to the incarceration of low-risk offenders who cannot afford bail, while high-risk offenders are sometimes released. This is due to subjective decision-making by judges without an objective measure of risk.

What is the purpose of the risk assessment tool developed by the speaker's team?

The purpose of the risk assessment tool developed by the speaker's team is to provide judges with an objective, scientific measure of risk for individuals in the criminal justice system. It helps predict whether someone will commit a new crime, an act of violence, or fail to appear in court if released.

How does the risk assessment tool work, and what information does it use?

The risk assessment tool works by analyzing data such as the defendant's prior convictions, history of violence, and failure to appear in court. It uses a universal dataset of 1.5 million cases to predict the likelihood of new crimes, acts of violence, and court appearance failures.

What was the outcome of implementing the risk assessment tool in Kentucky?

The outcome of implementing the risk assessment tool in Kentucky was not explicitly stated in the script, but the speaker's goal is for every judge in the United States to use a data-driven risk tool within the next five years, indicating a positive view on the potential impact of the tool.

What is the speaker's ultimate goal for the criminal justice system in the United States?

The speaker's ultimate goal is to transform the American criminal justice system by introducing data-driven decision-making tools, making the streets safer, reducing prison costs, and ensuring the system is fairer and more just.

Outlines

00:00

📚 Realizing the Flaws in Criminal Justice System

The speaker, a former attorney general, shares their journey of realization about the shortcomings in the criminal justice system. Initially a prosecutor, they became the attorney general of New Jersey in 2007. Upon taking office, they sought to understand the demographics of those arrested and charged, and whether the system was making decisions that increased public safety. They found that most criminal justice agencies lacked the necessary data tracking. Frustrated, they observed detectives manually reviewing case files, a process that yielded disheartening results, such as focusing on low-level drug cases near their office. The speaker also recounts a day at the Camden, New Jersey police department, which was the most dangerous city in America at the time, and noticed the lack of data-driven policing, with officers using yellow sticky notes instead of analytics. These experiences led the speaker to the conclusion that the system was failing, lacking data on critical aspects and not utilizing analytics to improve decision-making and reduce crime.

05:01

📊 Moneyballing Criminal Justice

The speaker compares the traditional approach to criminal justice with the 'moneyball' strategy used by the Oakland A's in baseball, which leveraged data and statistics to select winning players. They decided to apply a similar data-driven approach to criminal justice in New Jersey, aiming to transform the system. The speaker's team focused on using data and analytics to improve the critical decision of whether to detain or release an arrested individual based on their risk to public safety. They highlighted the high costs and inefficiencies of the current system, with 2.3 million people in jails and prisons, and a recidivism rate that is among the highest globally. The speaker's work at the Arnold Foundation involved creating a universal risk assessment tool using a dataset of 1.5 million cases from across the United States. This tool identified nine key risk factors that could predict an individual's likelihood to commit a new crime, engage in violence, or fail to appear in court. The tool was designed to be easy to use and to provide judges with an objective measure of risk, which could then be combined with their instinct and experience to make better decisions.

10:03

🛡️ Transforming the Criminal Justice System with Data

The speaker discusses the implementation of their data-driven risk assessment tool, which provides judges with a dashboard displaying the risk of new criminal activity, the risk of violence, and the likelihood of a defendant returning to court. They emphasize that this tool is not meant to replace a judge's instinct and experience but to provide an objective measure of risk to complement their decision-making. The tool has been implemented statewide in Kentucky and is set to be introduced in other jurisdictions. The speaker's vision is for every judge in the United States to use a data-driven risk tool within five years. They are also working on similar tools for prosecutors and police officers, with the ultimate goal of modernizing the American criminal justice system to be more data-driven, efficient, and just. The speaker concludes by expressing optimism that data science, or 'moneyballing' criminal justice, can lead to a system that makes streets safer, reduces prison costs, and ensures fairness and justice.

Keywords

💡Attorney General

The Attorney General is the chief legal officer of a state or country, responsible for advising on legal matters and representing the government in legal proceedings. In the context of the video, the speaker became the Attorney General of New Jersey, which marked a significant shift in their perspective on criminal justice. This role is crucial as it allowed the speaker to oversee and reform the criminal justice system from a systemic level.

💡Criminal Prosecutor

A criminal prosecutor is a legal representative of the government who brings forth cases against individuals accused of crimes. The speaker's prior experience as a criminal prosecutor in the Manhattan district attorney's office and the United States Department of Justice provided foundational knowledge and insights into the criminal justice system, which later informed their approach as Attorney General.

💡Data-Driven Policing

Data-driven policing refers to the use of data and statistical analysis to inform and guide law enforcement strategies and decisions. The speaker highlights the inadequacy of traditional methods, like using yellow sticky notes, and advocates for a more analytical approach to crime reduction. This concept is central to the speaker's vision for modernizing the criminal justice system.

💡Moneyball

Moneyball is a term derived from the book and movie of the same name, which describes the use of statistical analysis to make more informed decisions in baseball, particularly in player selection. The speaker uses 'moneyball' as a metaphor for applying data and analytics to criminal justice, aiming to make decisions based on objective data rather than instinct or experience.

💡Risk Assessment Tool

A risk assessment tool is a system used to evaluate the likelihood of a person committing a crime or failing to appear in court if released. In the video, the speaker discusses the development of a universal risk assessment tool that uses data analytics to help judges make more informed decisions about bail and release, aiming to improve both public safety and the fairness of the justice system.

💡Recidivism

Recidivism refers to the tendency of convicted criminals to reoffend. The speaker points out that the U.S. has a high recidivism rate, with nearly seven in ten released prisoners being rearrested, indicating a cycle of crime and incarceration. This statistic underscores the need for a more effective approach to criminal justice, which the speaker believes can be achieved through data-driven decision-making.

💡Public Safety

Public safety encompasses the protection of the general public from harm, whether from crime, accidents, or other threats. The speaker emphasizes that public safety is the most important function of government and that the current criminal justice system is failing in this regard. The goal of using data and analytics is to enhance public safety by making the system more effective and just.

💡Arnold Foundation

The Arnold Foundation is not directly mentioned in the script, but it's implied as the organization where the speaker continued their work on criminal justice reform after leaving the position of Attorney General. The foundation is likely involved in the development and implementation of data-driven tools and policies aimed at improving the criminal justice system.

💡Low-Level Crimes

Low-level crimes are minor offenses that typically result in misdemeanor charges. The speaker notes that the majority of arrests are for such crimes, yet the system spends a disproportionate amount of resources on them. This highlights an inefficiency in the criminal justice system that the speaker aims to address through data-driven reforms.

💡State and Local Corrections Costs

State and local corrections costs refer to the financial expenditures associated with the operation of prisons and jails at the state and local levels. The speaker mentions that the U.S. spends $75 billion annually on these costs, indicating a significant financial burden that could potentially be reduced through more efficient and data-driven criminal justice practices.

💡Pretrial

Pretrial refers to the period before a trial takes place, during which a defendant may be held in custody or released on bail. The speaker discusses the importance of using data to assess the risk posed by defendants during this period, as it can significantly impact decisions about bail, release, and public safety.

Highlights

In 2007, the speaker became the attorney general of New Jersey, shifting from a prosecutor's role.

Upon becoming attorney general, the speaker faced the reality of a lack of data in criminal justice decision-making.

The speaker discovered that most criminal justice agencies did not track critical data, leading to frustration.

Manual, case-by-case review was the only method available to gather data, highlighting inefficiency.

The speaker found that the focus was on low-level drug cases rather than more serious crimes.

A visit to Camden, New Jersey, the most dangerous city in America at the time, revealed the absence of data-driven policing.

The realization that the criminal justice system was failing due to the lack of data and analytics.

The speaker's past decisions as a prosecutor were based on instinct and experience, not data.

The introduction of 'moneyball' in criminal justice,借鉴了奥克兰运动家队使用数据分析挑选球员的方法.

Significant reduction in crime rates in Camden by applying data-driven strategies.

The shift in prosecution focus from low-level drug crimes to statewide important cases.

The importance of public safety as a fundamental government function.

The high cost and over-representation of low-level crimes in the criminal justice system.

The challenge of high recidivism rates and the need for a more effective criminal justice system.

The development of a universal risk assessment tool to aid judges in making objective decisions.

The tool's ability to predict recidivism, violence, and the likelihood of a defendant appearing in court.

The goal for every judge in the U.S. to use a data-driven risk assessment tool within five years.

The potential for data and analytics to transform the American criminal justice system into a fairer and more efficient one.

Transcripts

In 2007, I became the attorney general

of the state of New Jersey.

Before that, I'd been a criminal prosecutor,

first in the Manhattan district attorney's office,

and then at the United States Department of Justice.

But when I became the attorney general,

two things happened that changed the way I see criminal justice.

The first is that I asked what I thought

were really basic questions.

I wanted to understand who we were arresting,

who we were charging,

and who we were putting in our nation's jails

and prisons.

I also wanted to understand

if we were making decisions

in a way that made us safer.

And I couldn't get this information out.

It turned out that most big criminal justice agencies

like my own

didn't track the things that matter.

So after about a month of being incredibly frustrated,

I walked down into a conference room

that was filled with detectives

and stacks and stacks of case files,

and the detectives were sitting there

with yellow legal pads taking notes.

They were trying to get the information

I was looking for

by going through case by case

for the past five years.

And as you can imagine,

when we finally got the results, they weren't good.

It turned out that we were doing

a lot of low-level drug cases

on the streets just around the corner

from our office in Trenton.

The second thing that happened

is that I spent the day in the Camden, New Jersey police department.

Now, at that time, Camden, New Jersey,

was the most dangerous city in America.

I ran the Camden Police Department because of that.

I spent the day in the police department,

and I was taken into a room with senior police officials,

all of whom were working hard

and trying very hard to reduce crime in Camden.

And what I saw in that room,

as we talked about how to reduce crime,

were a series of officers with a lot of little yellow sticky notes.

And they would take a yellow sticky and they would write something on it

and they would put it up on a board.

And one of them said, "We had a robbery two weeks ago.

We have no suspects."

And another said, "We had a shooting in this neighborhood last week. We have no suspects."

We weren't using data-driven policing.

We were essentially trying to fight crime

with yellow Post-it notes.

Now, both of these things made me realize

fundamentally that we were failing.

We didn't even know who was in our criminal justice system,

we didn't have any data about the things that mattered,

and we didn't share data or use analytics

or tools to help us make better decisions

and to reduce crime.

And for the first time, I started to think

about how we made decisions.

When I was an assistant D.A.,

and when I was a federal prosecutor,

I looked at the cases in front of me,

and I generally made decisions based on my instinct

and my experience.

When I became attorney general,

I could look at the system as a whole,

and what surprised me is that I found

that that was exactly how we were doing it

across the entire system --

in police departments, in prosecutors's offices,

in courts and in jails.

And what I learned very quickly

is that we weren't doing a good job.

So I wanted to do things differently.

I wanted to introduce data and analytics

and rigorous statistical analysis

into our work.

In short, I wanted to moneyball criminal justice.

Now, moneyball, as many of you know,

is what the Oakland A's did,

where they used smart data and statistics

to figure out how to pick players

that would help them win games,

and they went from a system that was based on baseball scouts

who used to go out and watch players

and use their instinct and experience,

the scouts' instincts and experience,

to pick players, from one to use

smart data and rigorous statistical analysis

to figure out how to pick players that would help them win games.

It worked for the Oakland A's,

and it worked in the state of New Jersey.

We took Camden off the top of the list

as the most dangerous city in America.

We reduced murders there by 41 percent,

which actually means 37 lives were saved.

And we reduced all crime in the city by 26 percent.

We also changed the way we did criminal prosecutions.

So we went from doing low-level drug crimes

that were outside our building

to doing cases of statewide importance,

on things like reducing violence with the most violent offenders,

prosecuting street gangs,

gun and drug trafficking, and political corruption.

And all of this matters greatly,

because public safety to me

is the most important function of government.

If we're not safe, we can't be educated,

we can't be healthy,

we can't do any of the other things we want to do in our lives.

And we live in a country today

where we face serious criminal justice problems.

We have 12 million arrests every single year.

The vast majority of those arrests

are for low-level crimes, like misdemeanors,

70 to 80 percent.

Less than five percent of all arrests

are for violent crime.

Yet we spend 75 billion,

that's b for billion,

dollars a year on state and local corrections costs.

Right now, today, we have 2.3 million people

in our jails and prisons.

And we face unbelievable public safety challenges

because we have a situation

in which two thirds of the people in our jails

are there waiting for trial.

They haven't yet been convicted of a crime.

They're just waiting for their day in court.

And 67 percent of people come back.

Our recidivism rate is amongst the highest in the world.

Almost seven in 10 people who are released

from prison will be rearrested

in a constant cycle of crime and incarceration.

So when I started my job at the Arnold Foundation,

I came back to looking at a lot of these questions,

and I came back to thinking about how

we had used data and analytics to transform

the way we did criminal justice in New Jersey.

And when I look at the criminal justice system

in the United States today,

I feel the exact same way that I did

about the state of New Jersey when I started there,

which is that we absolutely have to do better,

and I know that we can do better.

So I decided to focus

on using data and analytics

to help make the most critical decision

in public safety,

and that decision is the determination

of whether, when someone has been arrested,

whether they pose a risk to public safety

and should be detained,

or whether they don't pose a risk to public safety

and should be released.

Everything that happens in criminal cases

comes out of this one decision.

It impacts everything.

It impacts sentencing.

It impacts whether someone gets drug treatment.

It impacts crime and violence.

And when I talk to judges around the United States,

which I do all the time now,

they all say the same thing,

which is that we put dangerous people in jail,

and we let non-dangerous, nonviolent people out.

They mean it and they believe it.

But when you start to look at the data,

which, by the way, the judges don't have,

when we start to look at the data,

what we find time and time again,

is that this isn't the case.

We find low-risk offenders,

which makes up 50 percent of our entire criminal justice population,

we find that they're in jail.

Take Leslie Chew, who was a Texas man

who stole four blankets on a cold winter night.

He was arrested, and he was kept in jail

on 3,500 dollars bail,

an amount that he could not afford to pay.

And he stayed in jail for eight months

until his case came up for trial,

at a cost to taxpayers of more than 9,000 dollars.

And at the other end of the spectrum,

we're doing an equally terrible job.

The people who we find

are the highest-risk offenders,

the people who we think have the highest likelihood

of committing a new crime if they're released,

we see nationally that 50 percent of those people

are being released.

The reason for this is the way we make decisions.

Judges have the best intentions

when they make these decisions about risk,

but they're making them subjectively.

They're like the baseball scouts 20 years ago

who were using their instinct and their experience

to try to decide what risk someone poses.

They're being subjective,

and we know what happens with subjective decision making,

which is that we are often wrong.

What we need in this space

are strong data and analytics.

What I decided to look for

was a strong data and analytic risk assessment tool,

something that would let judges actually understand

with a scientific and objective way

what the risk was that was posed

by someone in front of them.

I looked all over the country,

and I found that between five and 10 percent

of all U.S. jurisdictions

actually use any type of risk assessment tool,

and when I looked at these tools,

I quickly realized why.

They were unbelievably expensive to administer,

they were time-consuming,

they were limited to the local jurisdiction

in which they'd been created.

So basically, they couldn't be scaled

or transferred to other places.

So I went out and built a phenomenal team

of data scientists and researchers

and statisticians

to build a universal risk assessment tool,

so that every single judge in the United States of America

can have an objective, scientific measure of risk.

In the tool that we've built,

what we did was we collected 1.5 million cases

from all around the United States,

from cities, from counties,

from every single state in the country,

the federal districts.

And with those 1.5 million cases,

which is the largest data set on pretrial

in the United States today,

we were able to basically find that there were

900-plus risk factors that we could look at

to try to figure out what mattered most.

And we found that there were nine specific things

that mattered all across the country

and that were the most highly predictive of risk.

And so we built a universal risk assessment tool.

And it looks like this.

As you'll see, we put some information in,

but most of it is incredibly simple,

it's easy to use,

it focuses on things like the defendant's prior convictions,

whether they've been sentenced to incarceration,

whether they've engaged in violence before,

whether they've even failed to come back to court.

And with this tool, we can predict three things.

First, whether or not someone will commit

a new crime if they're released.

Second, for the first time,

and I think this is incredibly important,

we can predict whether someone will commit

an act of violence if they're released.

And that's the single most important thing

that judges say when you talk to them.

And third, we can predict whether someone

will come back to court.

And every single judge in the United States of America can use it,

because it's been created on a universal data set.

What judges see if they run the risk assessment tool

is this -- it's a dashboard.

At the top, you see the New Criminal Activity Score,

six of course being the highest,

and then in the middle you see, "Elevated risk of violence."

What that says is that this person

is someone who has an elevated risk of violence

that the judge should look twice at.

And then, towards the bottom,

you see the Failure to Appear Score,

which again is the likelihood

that someone will come back to court.

Now I want to say something really important.

It's not that I think we should be eliminating

the judge's instinct and experience

from this process.

I don't.

I actually believe the problem that we see

and the reason that we have these incredible system errors,

where we're incarcerating low-level, nonviolent people

and we're releasing high-risk, dangerous people,

is that we don't have an objective measure of risk.

But what I believe should happen

is that we should take that data-driven risk assessment

and combine that with the judge's instinct and experience

to lead us to better decision making.

The tool went statewide in Kentucky on July 1,

and we're about to go up in a number of other U.S. jurisdictions.

Our goal, quite simply, is that every single judge

in the United States will use a data-driven risk tool

within the next five years.

We're now working on risk tools

for prosecutors and for police officers as well,

to try to take a system that runs today

in America the same way it did 50 years ago,

based on instinct and experience,

and make it into one that runs

on data and analytics.

Now, the great news about all this,

and we have a ton of work left to do,

and we have a lot of culture to change,

but the great news about all of it

is that we know it works.

It's why Google is Google,

and it's why all these baseball teams use moneyball

to win games.

The great news for us as well

is that it's the way that we can transform

the American criminal justice system.

It's how we can make our streets safer,

we can reduce our prison costs,

and we can make our system much fairer

and more just.

Some people call it data science.

I call it moneyballing criminal justice.

Thank you.

(Applause)