Justice, Quantified

Amy Bach, whose book, “Ordinary Injustice: How America Holds Court,” is a must-read for anyone interested in the criminal justice system, has an op-ed in the New York Times  urging the creation of a “Justice Index” as means of quantifying how good a job our courts are doing.

But while millions of Americans deal with their local criminal courts as defendants and victims each year, there is no comparable way to assess a judicial system and determine how well it provides basic legal services.

This lack of data has a corrosive effect: without public awareness of a court system’s strengths and weaknesses, inefficiencies and civil liberties violations are never remedied.

That’s why America needs a “justice index” to show how the essential aspects of our local courts are working. The index, compiled according to national standards, would function roughly like college rankings, evaluating county courts on factors like cost, recidivism, crime reduction and collateral consequences, including whether people lose their jobs or homes after contact with the criminal justice system.

It’s not clear that the lack of data has a corrosive effect.  Rather, it plays into the prejudice of the public, there being nothing but anecdotal evidence to support one’s view of the system.  When someone you think should be convicted is acquitted, the system sucks.  When some judge rules against something you’re for, the judge is an activist.  The effectiveness of the system is entirely dependent upon achievement of the preferred outcome, which can be conviction for one person and acquittal for another.

The final chapter of Amy’s book suggests that the “ordinary injustice” of the American legal system exists because no one is watching.  While she is quite right in one sense, that no one is watching, the problem with any solution is who one gets to watch and what they are watching for.  If filtered through my eyes, rather than, say, Bill Otis, chances are that the results would be polar opposite with regard to our views on how good a job the courts are doing. 

Amy proposes a system to perform this function:

[A] panel of lawyers, community representatives, statisticians and law professors would establish standards for the measurements — for example, the percentage of people who plead guilty without an attorney or average bail amounts, because a high bail figure often compels defendants to plead guilty.

Another critical measurement would be the percentage of certain types of cases that get thrown out after a defined period of time, a possible indicator of inefficiency as well as disregard for traditionally under-prosecuted crimes. The index would also assess whether a county court has certain legal protections in place, like requiring that interrogations and confessions be taped.

The rationale for putting this diverse group on the panel is obvious; bringing together both the different interests as well as the skills needed to collect and understand the data.  Only the inclusion of law professors is a bit confusing, as they don’t seem to bring much to the mix, though they are likely to be the only ones available for all the meetings.  In the same vein, the group suffers from the same problems that pervade all issues around criminal justice, that they come with an agenda, a bias, and seek data to support changes that will produce their desired outcomes, whether more convictions or more acquittals.

The data collected would then be analyzed:

The information would be analyzed by a nonprofit organization, then posted to a Web site in a ranked order and in terms clear enough for the public to understand. Users would be able to shuffle the rankings by focusing on data related to specific areas like civil liberties or crime reduction, in the same way college applicants can look at which schools are best for student life or athletics.

This compels me to ask why?  Students get to apply to the college of their choice.  Criminal defendants don’t get to argue for a change of venue to a court that ranks higher in civil liberties.  As for communities reviewing the ranking of their local court system, I fear that the locals will march up the courthouse steps with torches and pitchforks if the data shows that the courts are doing too good a job protecting the constitutional rights of defendants. 

Amy’s desire to find an empirical method of assessing the viability of the legal system is completely understandable and, though I’m not sure that the comparisons with hospitals or universities holds true, clearly directed to the goal of improving the functioning and transparency of the system.  These are obviously important goals.

But just as use of sentencing statistics in the federal sentencing guidelines serves to work huge injustices, by way of disproportionate sentences and the inability to take into account individualized factors, any attempt to quantify something as amorphous and contentious as the legal system is going to be fraught with problems and likely end up serving the agenda of those with an ax to grind.  And it’s usually not the side of the defendants or constitutional rights that comes out of these things well.  We just don’t have a lot of strong advocates among the powerful.

As fine a goal as this may be, it’s likely that this will serve as a mechanism to justify more mindless, inflexible, sentencing-guidelines-type, zero-tolerance type, three-strikes-type programs, the sort that are invariably introduced as the magic bullet to fix a broken system.  Easily digested by the public and invariably wrong.  This is another good idea that is more likely to cause grave harm to the very people it’s intended to help.

4 thoughts on “Justice, Quantified

  1. Bad Lawyer

    Yeah, I read this op-ed this AM and it made my brain melt. I’ll have to read her book, but seriously, I had visions of Finnish Statisticians sitting in life guard charis with clipboards mysteriously checking boxes during court. You nail the problem, while we want the rule of law, since the time of Torah and Talmud, justice requires both judgment and mercy.

  2. Anne

    The Facebook section of our little blog found this proposition hilariously distant from reality. Simple to do! Why, just collect some numbers and crunch ’em. Really? We sentenced her to a week’s worth of hard labor on NCSC’s Court Statistics Project.

    Yes, there are empirical approaches but they are not simple or simply implemented.

  3. SHG

    Everything is simple until you actually have to do it.  I’ve never heard of any attempt at statistical analysis of court performance that was capable of assessing anything as amorphous and subjective as “justice”.  It’s not that I wouldn’t like to see it happen (provided that it was my definition of justice), but that they all have significant flaws.

  4. Lee

    That’s such a preposterous idea in that even if you can find things to take objective measurement of, if you’re going to “rank” the systems based on some scoring and weighting of those various measurements, the objective measurements have to be given subjective value. She acts as if this will be something we can measure once and for all and come to agreement. Even with a ringing endorsement from you for her book, with an idea that naive, I’m skeptical about how good her book could possibly be.

Comments are closed.