Following a debate between Judges Kopf and Bennett over the whimsy of discretionary sentencing, the omni-geeky Jake DiMare came up with the idea of the Sentence-O-Matic 1000.

Judge Richard Kopf raised the question of whether the language was so devoid of meaning and guidance as to render § 3553(a) worthless.  Judge Mark Bennett responded “Sentencing requires us to weigh that which cannot be measured,” to which Judge Kopf replied: “Let’s be honest then and declare that sentencing is entirely a matter of discretion…”  If so, this raises the specter of sentencing being so arbitrary and capricious, so captive to any judge’s whim, as to be a total crapshoot.

Jake offered this reaction:

Was there ever a task in the courtroom more ripe for automation?

Jake had a point.Remove the feelz component from the mix and reduce sentencing to cold, hard empiricism.

As a representative of the ignorant masses, I find comfort in the notion that everyone would be given sentences using the same criteria, and never again subjected to the whimsy of some of the judges.

There is, without question, an allure to the notion that people’s lives would never again be at risk of the vicissitudes of judges. Then again, the outcome would be at the mercy of the algorithm. No one could create an algorithm that would cover every variable, or balance them adequately, given the nature of the human condition.

As Judge Bennett noted, “Sentencing requires us to weigh that which cannot be measured.”  If it can’t be measured (and it can’t), then it can’t be input into the Sentence-O-Matic 1000.  And even if it was input to the best a bunch of programmers could manage, it couldn’t be subject to the arguments that real life human beings require to address the individualized circumstances that exist in every case and every sentence, because that would obviate the goal of consistency and return us to the whimsy of the judge.

Pshaw, you geeks and empiricism-lovers reply? Or perhaps, it’s imperfect but it’s less imperfect than one crazy old coot in a black robe who thinks he has magic powers because somebody handed him a gavel? Fair enough, except the hard, cold brutality of the algorithm has already found itself in the crosshairs of social justice.

The ACLU has begun to worry that artificial intelligence is discriminatory based on race, gender and age. So it teamed up with computer science researchers to launch a program to promote applications of AI that protect rights and lead to equitable outcomes.

You have science. You have “equitable outcomes,” whatever that means. So mix and shake?

Opaque and potentially biased mathematical models are remaking our lives—and neither the companies responsible for developing them nor the government is interested in addressing the problem.

This week a group of researchers, together with the American Civil Liberties Union, launched an effort to identify and highlight algorithmic bias.

Can math be biased?

Algorithmic bias is shaping up to be a major societal issue at a critical moment in the evolution of machine learning and AI. If the bias lurking inside the algorithms that make ever-more-important decisions goes unrecognized and unchecked, it could have serious negative consequences, especially for poorer communities and minorities. The eventual outcry might also stymie the progress of an incredibly useful technology (see “Inspecting Algorithms for Bias”).

Of course it’s not the fault of math, but measuring outcomes demonstrates a disparate impact based on race. If there is a disparate impact coming out the back end, then there must be bias in the algorithms on the front end. How do we know this? Because we believe that race cannot be a factor. It’s not that algorithms expressly take race into account, but use proxies for race, such as poverty, educational attainment and prior arrests and convictions. And reliance on algorithms is showing bias elsewhere as well.

Examples of algorithmic bias that have come to light lately, they say, include flawed and misrepresentative systems used to rank teachers, and gender-biased models for natural language processing.

Cathy O’Neil, a mathematician and the author of Weapons of Math Destruction, a book that highlights the risk of algorithmic bias in many contexts, says people are often too willing to trust in mathematical models because they believe it will remove human bias. “[Algorithms] replace human processes, but they’re not held to the same standards,” she says. “People trust them too much.”

This is true, and simultaneously false. An algorithm is only as good as its creation, the factors and weights given. It’s not as if AI has ill intent, but may be programmed in a way that incorporates biases. Prior convictions, for example, are certainly a critical factor in determining the efficacy of punishment, but if cops are far more inclined to toss black guys than white guys, then black guys are going to have more significant criminal records. Garbage in, garbage out.

The problem, of course, is that infusing the algorithms with bias to thwart bias means that we traded one bias for another, not that we’ve eliminated bias from the mix and the Sentence-O-Matic 1000 suddenly works. To blame it on the algorithm deflects responsibility for the problem. Algorithms only do what they’re told to do. They don’t feel happy or sad, love or hate, fair or unfair. They’re cold. Not like some judges are cold, but cold in the exercise of their harsh objectivity, which is only as objective as the inputs.

More to the point, we trust the math. Even worse, by trusting the math, the power of empiricism, it relieves humans of the emotional and intellectual burden of performing unpleasant tasks. Algorithms have no feelings, so if the output says one year or one lifetime, we can shrug, blame it on the Sentence-O-Matic 1000 and sleep well that night, knowing that it wasn’t our fault that some guy got life, but the machine’s.

So are the geeks and the ACLU fixing the problem?

MIT Technology Review reports that the initiative is the latest to illustrate general concern that the increasing reliance on algorithms to make decisions in the areas of hiring, criminal justice, and financial services will reinforce racial and gender biases.

If the inputs reflect bias, then so too will the outputs. Yet, if geeks game the inputs to eliminate perceived bias, it will still reflect bias, but of the opposite sort. The underlying premise is the belief that there can be no racial or gender distinction, such that any disparate impact must, by definition, mean that the algorithm is biased.

But if we’re to hand over discretion to ruin lives to machines, as geeks would like, and relieve judges from the unpleasant task of making hard decisions, will the purely objective outcome of AI be better if tweaked to reflect the purely subjective input of feelz? More to the point, no matter how we tweak the algorithm, will we ever be capable of a less biased outcome than we do under a regime where we can argue to judges whatever factors specifically apply to the individual before the court, rather than the 32 inputs some geek decided fits all defendants?

In many instances, an algorithm may well produce a less emotional, less biased outcome than a human judge who is, or isn’t, in touch with his feelings. But given the way in which the legal system falls back on technology, mindlessly accepting whatever outcomes it spews out, are we prepared to hand over lives to a machine whose purely objective, and absolute, determinations are as reliable as the humans who screwed with the objective factors to tweak the impact? Once we tweak empiricism with feelz, the only guarantee is that the output isn’t objectively sound.

Bear in mind, even if you’re good with the ACLU playing with the geeks to game AI at the moment, somebody will invite DoJ or the NDAA to the table at some point to seek their input on your client’s outcomes. If it’s not going to be purely empirical, purely objective, then whose finger is on the button matters. And even with the ACLU in the mix, will eradication of racial and gender bias mean black guys get a break, or that white guys and women get longer sentences?