Numbers Don’t Lie (But People Do)

At Hercules and the Umpire, Judge Richard Kopf takes Attorney General Holder to task for his speech before the National Association of Criminal Defense Lawyers.

“By basing sentencing decisions on static factors and immutable characteristics—like the defendant’s education level, socioeconomic background, or neighborhood—they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society,” Mr. Holder told the defense lawyers. Criminal sentences, he said, “should not be based on unchangeable factors that a person cannot control, or on the possibility of a future crime that has not taken place.”

Judge Kopf sees the AG as sticking his head in the sand, that he is “naïve and manifestly wrong.”

Indeed, a former policy analyst for the federal courts wrote three years ago that:

Evidence-based sentencing is based upon social science. Criminological meta-analysis has identified fifteen key variables that are significantly related to recidivism: 1) criminal companions, (2) antisocial personality, (3) adult criminal history, (4) race, (5) pre-adult antisocial behavior, (6) family rearing practices, (7) social achievement, (8) interpersonal conflict, (9) current age, (10) substance abuse, (11) intellectual functioning, (12) family criminality, (13) gender, (14) socio-economic status of origin, and (15) personal distress. If those variables can be used in sentencing, it may be possible to safeguard public safety while reducing the financial and social costs associated with mass incarceration.

J.C. Oleson, Risk Assessment at Sentencing, ASU Law Journal (June 20, 2011).

This is the age of empiricism, which many, myself included, will argue is a great step forward from the age of wives tales, ignorant assumptions and the dreaded “common sense.”

In my opinion, the use of empirical methods to assess risk at sentencing and upon supervised release is the most important aspect of the present movement toward criminal justice reform. We can and should aspire to depopulate our prisons. We can and should aspire to treat certain offenders less harshly. We can and should do a better job of helping offenders on supervised release. But if we believe that public safety is or should be a central goal of our criminal justice system we ought not to ignore the truth–certain characteristics that we have shied away from in the past because we worried too much about vague notions of “equality” or “fairness” tell us a lot about future danger.

This is a highly provocative assertion, that in order for empiricism to do its job, we cannot remove “static factors” such as race and gender from the calculus because of our overarching belief in equality and fairness.  The numbers tell a story, regardless of where we want that story to go.

I agree with Judge Kopf.  Judge Kopf is wrong.

The strong trend toward an evidence-based view, whether in recidivism or any other human endeavor, is far more meaningful and useful than our reliance on commonly held beliefs that lack, or defy, evidence.  Beliefs are great for religion, but lousy for law.

But when this argument is made, it must be fully made, fully explored.  Empiricism isn’t born of guys with guns and shields doing what they usually do, rounding up the usual suspects in neighborhoods where people with darker skin live.  Garbage in, garbage out.

Empiricism must be real, suck up all of reality, not just the easy stuff a bad system stuffs into the numbers box.  There are reasons why wealthier neighborhoods provide better education to their youth.  There are reasons why kids from middle class families who are convicted of crimes aren’t prone to recidivism.  There are reasons why black and Hispanic kids are tossed,  arrested, prosecuted, convicted, imprisoned in grossly disproportionate numbers.

If we’re going to adore empiricism, then we need to understand it fully and appreciate what it requires before worrying about what it gives us back in return.  The effort to create a sentencing regime based on empirical analysis of who is most likely to commit crimes after their punishment is completed is a worthy one, but one that can’t possibly be accomplished. At least not now.

At this point in our history, the system spews out on the back-end of convictions nothing better than what’s taken in on the front end.  It would clearly suggest that people of color are more prone to be criminals in the first place, and more prone to recidivism afterward.  That’s the story the numbers tell. But the numbers are only as good as the input.

More young black men are recidivists because more black men are arrested, even though there is evidence that they don’t commit more crimes than white men. They just get arrested more, because they are who the police spend their time arresting. This gives rise to an internal community spiral, missing fathers (because they’re in prison), poor education, poor job prospects, etc.  Toss in a criminal conviction and ask empiricism why they aren’t being hired for well-paying jobs at IBM.

So I completely agree with Judge Kopf that there can be no viable empiricism when we introduce politically correct factors into the analysis.  The numbers speak for themselves, and if empiricism is to work, we can’t screw with the numbers to make them conform to political notions of equality and fairness.

And I completely disagree with Judge Kopf that there is any possibility of this happening under the existing regime, because its input is so rife with bias that counting heads on the way out the door of prison does nothing more than reinforce the prejudice of poor black and Hispanic young men being taken in day after day, with only lousy options before and far worse options afterward.

Let’s definitely do empiricism. And let’s definitely include all factors, no matter how we want the number to turn out.  Just not until we can do better than garbage in.  And that’s all we have now, garbage in, and all empiricism will do for us until this changes is give us garbage out.  That’s not empiricism. That’s confirmation of the lie that the criminal justice system isn’t still rife with prejudice.

26 thoughts on “Numbers Don’t Lie (But People Do)

  1. Tom

    Is it so much a problem with the criminal justice system or is it really just the police? Most prosecutors will prosecute whatever is put in front of them. Judges can only rule on what the prosecutors bring before them. It seems like the real problem in the system is the police and if that is the case, what have we done so wrong that police prejudice appears to be so systemic across so many geographically different departments? Is it merely that prejudice seeks power and color of the law is the best way to get it in this society? Perhaps instead of allowing individuals to apply for jobs as officers we should have a national draft like the military used to do to select our local police officers.

    1. SHG Post author

      Each cog in the wheel serves a purpose and has a role to play. Nothing is ever so simplistically reducible to “the real problem.” Whenever a cog in the wheel fails to do its job, it’s a “real problem.”

  2. Brad Walters

    There is a reason social “sciences” are not in the same building with hard sciences. Maybe social science is somewhat of an oxymoron. Social sciences start with a hypothesis and attempt to prove the truth of it. Real scientists start with a hypothesis and try to disprove it rather than assuming its validity. That is how the garbage gets in instead of sifted out.

  3. adam kadmon

    at the risk of not getting a tummy rub doesn’t the Constitution prohibit bloid libel am i wrong or just off topic? or does the science interpetation of the moment trump the constitution too?

    1. SHG Post author

      If what you’re trying to say is the Equal Protection Clause prohibits the use of race, etc., as a factor in sentencing, it’s an interesting question. I don’t know how it would apply to empirical analysis.

  4. Pingback: Please read Scott Greenfield’s post “Numbers Don’t Lie (But People Do)” « Hercules and the umpire.

  5. EH

    Sure, emptying prisons is one goal. But of course it is far from the only goal–we are running up against the age-old conflict between “justice” and “fairness” here. Just to run with specific examples, I’ll use the common assumption that, say, being black correlates with certain risks of criminality and that everything else is equal. I DO NOT THINK THIS IS ACTUALLY TRUE, but it is necessary to demonstrate the argument:

    1) If you consider race in sentencing, you end up having a larger %age of black people than white people who are in jail for similar conduct. You would, obviously, have some ethical questions w/r/t civil rights. However, in theory you would also have a more accurate sentencing policy.

    Functionally, choice #1 protects society at the expense of those actually-docile black prisoners who are incorrectly classified as “more dangerous” due to their race, and gives an ancillary benefit to actually-horrible white prisoners who are incorrectly classified as “less dangerous” due to their race.

    2) If you don’t consider race in sentencing and if you take the “maximally protective” view, then you would sentence everyone as if they were black. That would increase the prison population–but would ensure that everyone above a certain cutoff of risk was imprisoned.

    Functionally, choice #2 protects society at the expense of those actually-docile prisoners of ALL races, who lose the opportunity to be classified as “less dangerous” due to their race. But the actual cost is borne by actually-docile white prisoners: the inherent racism in the initial assumption (‘blacks are more dangerous’) means that blacks would never be eligible for special treatment based on race, so the loss of that special treatment isn’t a cost. (It denies any benefits to actually-horrific white prisoners who would have seen a benefit due to race.)

    3) If you don’t consider race in sentencing and if you take the “maximal civil rights” view, then you would sentence everyone as if they were white. That would reduce the prison population–but it would mean that some people were released who should be in prison. If you continue to maintain the initial assumption, then the actual people who would be released are “everyone who is black and who would not have been released in #1 above, but who is released in #3.” Unlike the other examples, the cost of #3 is spread more across society–in particular, across everyone who has contact with a person that harms them, where the harm would have been prevented if that person had been in prison.

    Frankly none of those situations are ideal. But #3 is the best one by far.

    It isn’t that #3 is more accurate. It isn’t: as of right now, race matters in criminal risk. But obviously it is more accurate to say that “the effects of society on people as a result of race produce differing outcomes w/r/t criminal risk” rather than “the genetic factors of race are related to criminality.”

    So in that sense, #3 is a better match for OTHER social incentives. Most obviously #3 is the only good match to the goal of NOT producing differing outcomes in members of society as a result of their race.

      1. EH

        Because we should reduce, not increase, the effects of luck on outcomes.

        I am a relatively upstanding member of society–well, except for being a lawyer. But the fact that I am how I am instead of “some other way” is heavily related to the fact that I happen to be white, healthy, and born to educated parents who were willing to work hard and help their kids. I coach youth soccer; I give pro bono time; I’m a “good member of the community.”

        Alternate-universe me was switched shortly after birth into a family with uneducated parents, different moral codes, and a bad neighborhood. AU me doesn’t do pro bono work (AU didn’t make it to law school) or volunteer much (AU is trying to make enough to get by) and so on. AU probably did a bit of petty theft at some point; AU may smoke (I don’t) or drink more; AU is exposed to a lot more people doing things which I don’t experience.

        AU is more likely to create criminal risk. Not as many people care about, or support, AU.

        If you sentence us as “human beings” as we are now then AU gets a longer sentence. But if anything, you should be giving ME a longer sentence, because I had something good and screwed up anyway–fewer excuses on my side.

        1. SHG Post author

          While I’m still disinclined to succumb to sentencing stereotypes, whether regular you or AU, interesting point.

  6. Richard G. Kopf


    Throw out race because, although it may be predictive (not causal), the federal criminal justice could never swallow that whale.

    What about using large data bases compiled on age, gender, socio-economic background, etc., to make predictions about future violence? Should those be ignored? (Humor me. Assume for a moment that the data sets are good, meaning that 50 bishops would attest their objectivity and scientific methodology.)

    All the best.


    1. SHG Post author

      Do you want to be the one to tell poor, young black kids that they’re going to spend most of their adult lives in prison because the computer says so? That strikes me as a self-fulfilling prophesy. and if one believes, as I do, in Blackstone’s ratio, then how do we imprison people for what they have yet to do, because big data says so?

      Predictions based on big data will probably prove very reliable, based upon our existing societal norms. But it will still harm people who shouldn’t be (collateral digital damage) and fix all the negative aspects of prejudice, poverty, etc., making rising above them more difficult, if not impossible.

      Not my happy world.

      1. Jake DiMare

        Why must sentence-o-matic 1000 use any demographic information? Why can’t this just be about the crime, the individual criminal’s record, and the statistics associated with similar or identical situations?

        From a programmatic perspective it feels like the addition of demographics (with or without race) is just grasping for a class (literally an object oriented programming term, not referring to social class) that will provide a shortcut to calculating the odds associated with a specific individual case.

        1. SHG Post author

          This is why programmers scare us so very much.

          Mother Teresa to her lawyer: But why didn’t the Sentence-O-Matic 1000 consider that my life has been spent helping the poor and infirm?

          Lawyer to Mother Teresa: Because the programmer didn’t make a box for that. You’ll be executed at midnight.

          1. Jake DiMare

            But this is a point on which reasonable people may disagree, whether or not one of us is a programmer.

            As a citizen, not as a programmer, I prefer a system where defendants are sentenced purely based on the nature of the specific crime they have been found guilty of committing AND one class of mitigating circumstances: their individual criminal record.

            Now, we would have to have a completely separate conversation about what I believe sentences should be…And my POV might make more sense if we did but I’m not going to jam up this conversation with all the details.

            The one think I will say, and I know this is going to make your head explode, I don’t believe in free will. Therefore sentencing in my imaginary, idealistic world is a very, very different game which is much more about rehabilitation than incarceration in all but a small minority of the most violent, repeat offenders who have proven to be beyond rehabilitation.

            I appreciate your allowing me to take part in this conversation. I know I am out of my depth in many ways, substituting the specific for the wildly philosophical, and probably violating 3 or 4 other term of SJ commenting use.

            1. SHG Post author

              This gives me a far better idea of why you think something as complex and filled with ambiguity as sentencing can fit into a neat little package. As you suspect, I disagree completely, but at least I now understand where you’re coming from.

  7. Joe

    “More young black men are recidivists because more black men are arrested, even though there is evidence that they don’t commit more crimes than white men. They just get arrested more, because they are who the police spend their time arresting”

    It would be nice if you would provide a citation for this claim.

  8. Alex Bunin

    “Evidence-based” is so 2008. It’s all “research-based” now. I should know, I just sat through a two-hour strategic planning session. Anyway, I listened to the AG’s speech in Philly and I generally agree that many of these metrics end up being proxies for class and race. However, left to their own devices, judges limit their decisions to the charges and criminal history anyway. When research/evidence-based models expand the information and choices for the decision-maker they are good. Otherwise, they just make everyone feel good.

  9. Pingback: News Roundup « North Carolina Criminal Law

  10. Pingback: May the Odds Be (Never) in Minorities’ Favor? Breaking Down the Risk-Based Sentencing Divide | Freelance HoneyBadger

  11. Pingback: May the Odds Be (Never) in Minorities' Favor? Breaking Down the Risk-Based Sentencing Divide | Jessica Eaglin

Comments are closed.