Would You Be A Tool To Validate A Tool?

The head of the public defender’s office got an email with a modest request:

I am a Professor at the University of Pennsylvania Law School and I am writing to see whether you would be interested in collaborating on a research project aimed at improving plea bargain outcomes and reducing disparities.

In previous work, I have found evidence of substantial disparities in public defender outcomes, primarily due to differences in plea bargaining.  With this project, my colleague Megan Stevenson and I want to test a new tool that may improve plea bargain outcomes through the provision of more information to public defenders.

We have built a predictive tool that can estimate the range of potential case outcomes for a client based on certain characteristics (offense, criminal history, etc.).  We believe this tool may augment defender experience in deciding when to take a deal and when to push for a better one.  This could be particularly useful for defenders with less experience.  It could also help a defender demonstrate to a skeptical client that they are getting a good deal.

The key to this project is finding a public defender’s office to partner with to establish the efficacy of the new software.  This will involve randomizing which PD’s receive the tool initially, and which present the information to clients.

We are applying for funding for the project this month, so if you might be interested, I would very much appreciate it if you could let me know this week, so that we can begin the discussion.

Thank you for your time and I look forward to hearing from you.

Best regards,
David S. Abrams

It’s understandable that a law professor wants to use actual living poor defendants to validate his predictive tool. Such interests, tools, are very much in vogue, given the appreciation of empiricism, the expectation of technology as the future of the law and legal practice, and the scholarly pursuit of newer, better ways.  It will certainly make for a great law review article, and perhaps even a new business that will be hugely successful for its owners, and even beneficial for its intended audience.

It’s perfectly understandable for David Abrams to ask.

And it’s perfectly understandable for the public defender he’s asked to mutter, “is he fucking nuts?”

These aren’t guinea pigs, lives of such utter inconsequence, that they can be laid to waste in the process of validating this predictive tool, whatever the hell that is supposed to mean.  Nothing in the email begins to explain how well-conceived, or utterly idiotic, this tool might be.  Abrams’ background offers little comfort.

  • Ph.D. (Economics) – MIT – ’06
  • M.S. (Physics) – Stanford – ’01
  • A.B. (Physics) – Harvard – ’98

No, he’s not a lawyer. No, he didn’t go to law school. No, he’s never held a defendant’s hand before the cell door clangs shut, not to reopen for a few decades. But then, that’s not where Abrams’ interested lie:

David Abrams is one of the leading young economists working in empirical law and economics. His work covers a range of topics, tied together by goal of understanding and measuring how individuals respond to incentives in various legal contexts. Criminal justice is one of his major areas of expertise, where Abrams has investigated a variety of questions, including whether longer sentences deter crime, how defendant race impact judicial decisions, to what extent attorney skill affects case outcomes, and how much individuals value freedom.

How “criminal justice” is one of Abrams’ “major areas of expertise” is something of a mystery.  He possesses no apparent skill, by training or experience, that relates in any way to criminal law.  What his background suggests is that his mad skillz relate to the combination of empiricism and technology.  Criminal law may be his “passionate interest,” and he may be very well intentioned in pursuing it in conjunction with his background, but he is about as far away from being an expert as anyone could possibly be.

What makes me say such a harsh thing about an academic who clearly wants to apply his actual training to a cause he can only see from a great distance?  The fact that he sent this email without the recognition that he is asking public defenders to put their clients at risk, actual human lives at risk, to play with this Ivory Tower toy.

But what if it works?  Wrong question. What if it doesn’t?  Public defenders can’t volunteer to turn their clients into mice in some dilettante’s maze.  They can’t sell out their duty to their clients to help some self-proclaimed crim law expert figure out if his baby is unbearably ugly.  And if David Abrams was even remotely as much of an expert as he claims to be, he would know this.  He would never have asked public defenders to risk their clients to validate his toy. He would know that a lawyer does not offer up his client’s life because some guy sends him an email about his mystery tool.

Does that make David Abrams a dangerous person?  Why yes, it does. But he’s not necessarily the most dangerous person in this situation.  Even worse than some academic dilettante who has nothing to recommend his qualifications in criminal law is the public defender who would receive his email and respond, “sure, why not?”

Because your duty is to your clients, not Abrams’ validation of his new baby, that’s why not.  The only way academics like Abrams can combine their ignorance of the responsibility of a public defender with their adoration of technology and get away with it is when someone charged with the representation of other human beings is even less cognizant of where his duty lies.  The poor are not fodder for empirical experiments. Their lives are not expendable to fill an academic’s needs.

And yet, you ask, what if Abrams’ predictive tool is a good thing, a great thing, that can help thousands of defendants?  Take a risk with your own life if you want, but no public defender, no lawyer, can play with the life of his clients to find out. It’s nuts.

79 comments on “Would You Be A Tool To Validate A Tool?

  1. AP

    And there will be future-of-law tools out there who will simultaneously praise this professor’s magic tool and decry you for condemning this project.

  2. Jay Wolman

    I would say his approach needs work. But, I have to disagree with part of your assessment. This is a question of computer modelling to predict outcomes. Physicists and economists tend to be the ones who have the most experience developing these models. Physicists have been employed in many non-physics settings (pharmaceutical companies, movie studios, etc.) to tap into their skills in this field. He is not a lawyer, nor should he be. This is about data and the ability to provide potentially useful information.
    How that information is gathered and used directly requires an experienced criminal defense attorney, but the actual modelling is his expertise. If there is to be such a tool developed that could be of assistance, someone with his background is precisely who should play one of the most significant roles.

    1. SHG Post author

      You are no longer allowed to represent anyone. You have just forsaken any pretense to the ethical responsibility required to be a lawyer.

      Give my best to your tech overlord.

      1. Jay Wolman

        Reread my comment. Representation is left to the lawyer. But I am referring to a person of his background as developing the model. I have forsaken nothing. Will the model have any utility? That awaits to be seen. But what is the harm in developing a model that may provide your client with better predictive power? It is a question of risk management. Insurance claims departments have tools to predict risk; why shouldn’t the criminal defense bar?
        When your client asks “what are my chances?”, wouldn’t it be nice to say: according to available data, the chance of X is Y. That is not the be-all and end-all of your job, as it would be to help him/her understand the prediction and assess his/her chances against other factors that may not be part of the model.
        Knowing there’s a 35% chance of rain isn’t going to provide all of the information needed to determine whether to carry an umbrella today, but it certainly can assist in the determination. And, if there is a tool that can provide useful information to our clients, it is our duty to consider it. This isn’t about a physicist intruding into your fiefdom. Personal injury plaintiff and defense lawyers hire medical doctors to give an expert opinion as to medical prognosis of the injured party to assess damages. This is no different, except that it might actually have better predictive value than the doctor relying on his/her individual experience.

        1. Fyodor

          Hi Jay

          Here’s what I would consider to be different from your analogy.

          The weather models are transparent and have been subjected to outside review for their underlying methodology.

          The weather models are put together by physicists who have a first hand understanding of how weather works and can understand whether the underlying data they are using is the right data and isn’t being misapplied.

          The weather models have been tested and validated so both their reliability and the conditions where they are not as reliable have been established.

          The meteorologists using the weather models are in a position both through statistical training and disclosure of the underling models to weigh what it gives them in an informed manner.

          1. SHG Post author

            There’s one additional, kinda huge, problem with the analogy. When a meteorologist gets it wrong, someone gets wet, not goes to prison for life.

          2. Jay Wolman

            Hi Fyodor,

            From what I can see, the thrust of the disagreement seems to be some assumption that clients are being experimented upon as though attorney counsel is to be displaced. Nothing in Prof. Abrams’s letter, as presented by Scott, makes any such suggestion. Instead, it says some PDs will get to use a tool to present information to some clients. That’s it. We know nothing else. If the tool presents information, nothing about the letter says that the PD must do anything with it in contravention of his/her duties. The rules of ethics trump, so a PD cannot be required to present X or counsel the client to do Y, if it is against his/her professional judgment. If Prof. Abrams chimes in with additional requirements that interfere with professional judgment, then I might take issue with the project, but for now I am simply refusing to rush to judgment.

            Getting back to the weather analogy, all we know is Prof. Abrams is the principal investigator. There is no indication that he has not worked with experienced criminal defense counsel to build the model. Again, this is a rush to judgment. I find the Dunning-Kruger and Heinlein references amusing, because one could just as easily flip the references around, with criminal defense experts narrowly thinking that an outside field couldn’t possibly provide assistance.

            Lawyers, like meteorologists, could be taught how to use the information. On the civil side, insurers have used these types of models for years to predict likely exposure, but ultimately it is a human who authorizes settlement or trial. This is an extension of that use. Given that the lawyer must always exercise professional judgment, there is minimal risk that if the model spits out something absurd that the lawyer will actually counsel the client absurdly. But, a lawyer individually has a narrow sample set–his/her own experience plus what he/she has heard. Aggregation and modeling gives the lawyer a chance to access information he/she doesn’t have.

            What’s ironic in Scott’s reply is that he misses that when weather models get it wrong, people die in floods, mudslides, wildfires, and other calamities. Yet we still use and adapt models and we still transmit information, though the public takes it with a grain of salt. Here, too, the information should be taken by the PD with a grain of salt and he/she should always rely on his/her professional judgment.

            1. Marc

              If I was to explain to you, Jay, everything that’s totally fucking idiotic about your comment, it would just cause you to write another, lengthy, similarly idiotic comment because there is something seriously fucking wrong with you.

              But Jay, you are an idiot. That is all.

            2. Sgt. Schultz

              Yet another exasperating Dunning-Kruger moment. It seems like there’s something wrong with this guy, kinda wacky. Maybe because he just won’t stop and runs on so long and stupid that you just want to strangle him. EH, on the other hand, is almost always a flaming asshole. Reminds me of Max in a way. Clueless, yet belligerent.

  3. Corporate Tool

    “With this project, . . . I want to test a new tool that may improve plea bargain outcomes” regardless of quality of evidence or innocence.

    1. fledermaus

      “We have built a predictive tool that can estimate the range of potential case outcomes for a client based on certain characteristics (offense, criminal history, etc.). ”

      So he found a sentencing grid?

      1. SHG Post author

        I would assume there’s more to it, or it would just be a shitty version of the sentencing guidelines grid, but what that might be is mystery.

  4. Ted Folkman

    It seems to me that with the informed consent of the client–something not mentioned in the email–this kind of thing can be helpful to lawyers and clients in the same way that a randomized clinical trial can be helpful to doctors and patients. If you sit down with a client and say, “some mad professor is going to provide us with information we don’t already have, and if and when the time comes to make a decision about a plea agreement, we will evaluate it together and give it whatever weight we think it is worth, what do you say?”–what’s wrong with that?

    1. SHG Post author

      Putting aside the irony that the only lawyers who see any merit in this are lawyers who don’t practice criminal law, it still fails. There can be no informed consent, as the lawyer has no ability to advise his client to accept, reject, act upon or not, this tool because it’s validity is entirely unknown. One cannot advise a client to be influenced by something about which nothing is known. One cannot discount the influence that some unknown science-ish type tool will have in making a decision that could destroy a person’s life. A lawyer cannot absolve himself of his ethical duty by shifting it onto a client incapable of making a competent decision.

      But in medical research, they test first on a small animal. Then a larger animal. See who dies. That’s what’s wrong with it, and why your medical analogy fails. We don’t kill our clients in the furtherance of technology. Well, at least criminal lawyers don’t. Your mileage may vary.

    2. Keith

      Clinical trials must also go through an ethical review board. I don’t see how treating some defendants as eggs to be cracked in order to perfect an omelet would meet a standard of ethical review. But I’ll leave that to those more knowledgeable.

      1. SHG Post author

        Good point. I forgot all about the ERB. Even experimenting on the poor has to go before an ERB before you can destroy their lives.

      2. Mark Sinton

        This is clearly research that involves human subjects. It would have to clear his institution’s human subjects research board (which could be the same thing as an ethics review board) before he could even attempt to start doing this research.

        I doubt that any HSRB would grant clearance for such a proposal, but if they did, he’d need to gather signed informed consent forms from all involved: the clients, the public defenders, the prosecutors, and perhaps even all other court officials.

        What a boondoggle!

        1. fledermaus

          Adams shows he knows nothing about plea offers, the bifurcated criminal process, standard court procedure, the ethics of using human beings for social experiments, or real world sentencing issues. But by god his model is super-terrific and full of data and math. How can anyone disagree with data and math?

          Wait, let me guess, he has a PhD in Economics

  5. Keith

    And it’s perfectly understandable for the public defender he’s asked to mutter, “is he fucking nuts?”
    Hanlon’s razor, being applied, I hope he actually responded.

      1. John Barleycorn

        ….and did you drop Abrams a note asking him for a bit of clarification as to his “key”.

        If you didn’t I am a bit bummed out you didn’t give him the opportunity to fondle the rope.

        Drop the guy an email esteemed one he might share some math-S with you. You need more math-S in your life anyway no mater which way it goes.

  6. Steve Hartson

    “It’s understandable that a law professor wants to use actual living poor defendants to validate his predictive tool….”

    No, it’s not understandable. Research involving human subjects requires tremendous forethought, great empathic consideration, pre-approval and ongoing oversight at multiple levels, and tons of paperwork. Moreover, his proposed subjects (prisoners) are a well recognized “vulnerable population.” Credible researchers are highly cognizant of the ethical issues associated with anything involving human subjects, and Dr. Abrams doesn’t get a pass on this one just because he’s “not a lawyer.”

  7. pavlaugh

    “This will involve randomizing which PD’s receive the tool initially, and which present the information to clients.”

    This seems like the incredibly troubling aspect of the study. The statement suggests that some lawyers will receive the tool (information) although not be allowed to present the information to the clients. What kind of fuckery is this?

    Imagine the lawyer recommends a deal in accordance with the tool, and the client asks why he should take it. If the lawyer responds with anything other than “there’s an app for that,” we’ve got a bit of a problem.

      1. Keith

        Where’s AVVO on this? Seems like a job for tech entrepreneurs, not law profs.

        Click here to find out how much time you’re going to be facing and we’ll recommend an attorney based on your prospective sentence.

      2. pavlaugh

        I’ve got less of a problem with that ethically if (1) the methodology of the tool is understood and can be explained by the lawyer; and (2) the tool is not the sole basis for the recommendation, i.e., there is still some independent counsel/thought/judgment exercised by the lawyer based on real life factors and experience, given that no tool like this can take into account and process the myriad of facts and circumstances known to the lawyer.

        Based on comments above, you seem to suggest both these conditions can’t or won’t be met. You might be right, especially if the tool “works” and baby lawyers therefore rely on it unflinchingly.

        1. SHG Post author

          I have strong doubts as to the efficacy to begin with, largely because the experience with tech-types and empiricists is that their lack of grasp of how things function in the trenches, as opposed to theory, cause them to make significantly mistakes. But that’s a general beef; here, the tool is unvalidated, which is why they want human lab rats. Without revealing everything there is about the tool to the lawyers (which they won’t, as that would ruin the study), and until it’s validated (but how can we validate it if no one will use it? Tough shit. You still can’t use poor people as guinea pigs), it can’t ethically happen.

          So how are they supposed to test out their tool? Beats me, but they still don’t get to destroy poor people’s lives in the hope of the next great technological advance.

            1. Matt B

              Jesus, you have one too many turkey dinners and glasses of wine before a spacewalk, and all of a sudden everyone’s a joker.

    1. Mark W. Bennett

      To do a proper study they’ll have to give one group of clients correct numbers, and another group wrong numbers, without knowing which is which.

  8. Mark Draughn

    You really seem to love (hate) Professor Abrams. He’s also the author of the study you blogged about in “Is The Trial Penalty A Myth?” which purports to show that there’s no trial tax. On the other hand, in “The Definitive Answer: Age Over Beauty” you rather liked his conclusion about the benefit to clients of lawyer experience over a fancy law degree. Maybe you and he should collaborate on a study?

    1. SHG Post author

      Bite me, Windy. I have no issue with his using his mad empirical skillz within the parameters of his abilities. In the Trial Tax study, he was utterly clueless as to how it worked, and thus produced a deeply flawed study. My post about the Age study (that’s a pretty old one) was mostly tongue in cheek. I really didn’t give a shit that his study confirmed my experience, which I thought was pretty apparent from my tone.

  9. Shannon Severance

    From Abrams’ email “It could also help a defender demonstrate to a skeptical client that they are getting a good deal.” [emphasis in original] makes it sound like this is a tool for selling the plea. Isn’t selling the plea the job of the prosecutor who wants an easy conviction and to move on? “It could also help a prosecutor demonstrate to a skeptical criminal that they are getting a good deal.”

    And with lives in the balance, one would hope that the tool is written with the same care as the software in medical devices and avionic. Or at least as much care as the software controlling acceleration in a Toyota Prius. That would be much more than the care put into the latest “big data” project to figure out how to sell more stuff.

    1. SHG Post author

      Deft: But I’m innocent.
      PD: But the tool says you have to take this sweet, sweet deal, only 197 years!!!
      Deft: But I’m innocent. Oh fuck it, I’m gonna lose with you anyway.

  10. Robb Fickman

    (read this aloud Doing your best Dr. Strangelove impersonation)

    Sir -Ju misunderstand ze greeeattt advances ziss tool vill make in the Kriminall Justus System. With our Enigma 5000 ve Vill be able to ascertain within 15 minutes the precisely Korrect Sentence for the Masses. Ve vill enter ze data in one end of Ze Enigma 5000 machine and the Korrect sentence will come out the other end. Like a perfectly formed Shtruddle the Sentence will be a delight to behold. With ze mass data entry ve can determine sentences for millions of crimininals in ze most efficient manner. All for the greater goot of ze Fatherland. A goot scientist can Always help in the systematic processing of criminals, immigrants or other undesirables.
    If Herr Trump vins and becomes our beloved Furor, ze Enigma 5000 vill be a velcomr new tool.

    Best regards to Eva,

      1. Robb Fickman

        Me. Unlike our friends, I was actually raised in West Texas and I sound Texan. Plus I got a new hat.
        You know they wanted Peter Sellers to play the Slim Pickens part. Great movie.
        The notion that any lawyer would advise his client to take a deal because the machine or the algorithm says so, is just plain stupid. It’s always dangerous when people who no nothing about criminal defense, pretend otherwise.

          1. Robb Fickman

            An old hat is like an old friend. You never throw out an old hat.
            I thought Gerry Spence gave you a good hat. Send me a picture of the hat he gave you.

            1. Robb fickman

              A fur felt hat has an X factor rating. The higher the X factor the better quality. 4X is a good hat. You can wear that hat with pride.

  11. EH

    At heart, you seem to be assuming that the program is potentially very harmful to clients. But it merely provides information, and–perhaps–attempts at advice. A test like this is akin to a big firm saying “let’s give the Chicago division access to a test algorithm, which they can use if they want, and see if that division starts producing better results than the Boston division.”

    Those are not all that dangerous; PDs deal with information and advice all the time, and they are trained to balance bad versus good. I’m sure you don’t take all the advice you get, right? Nor do you trust everything you read online, correct? So where’s all the potential harm?

    Obviously, the PDs in question should know that the program is not approved, so they should use their judgment. Obviously, the test should not seek to interfere with what the PDs disclose (or don’t disclose) to their clients. And obviously, it would need to be vetted for accuracy of factual data presented to make sure the data is accurate. But if it’s transparent and based on accurate data, what’s the issue?

    Imagine it says: “This charge shows in 1,937 arraignments in the past five years. In the past year, 76% of Outcomes are below. Click to obtain statistics for year / county / court / judge / date / plea outcome / trial outcome etc. ” It could be helpful to some–and not to others.

    Just information, right?

    1. SHG Post author

      Amazing how you have created an entire fantasy world around a tool you know nothing about, in an area of law you know nothing about, devoid of the slightest grasp of what you’re talking about. Masturbation is best done in private.

      1. EH

        Heh. I probably know a shitload more than you do about running studies and presenting data; that was my field before law. You don’t have solid ground here.

        What’s next? Someone writes a book and asks a few PDs to read it and share it with their clients, and you start screaming about ethics? Information is information. Why is it magically unethical to provide THIS information, but not OTHER information, to attorneys? There’s no devil hiding in a computer screen.

        1. SHG Post author

          Well, I’m sure your brilliance would bear out, if only I wasn’t so distracted by your stream of stupid comments. Of course, how they could validate this tool (about which you magically know so much) by ignoring it and failing to use it remains a mystery, but that’s probably because it’s all about running studies and has nothing to do with the underlying use of the tool, about which you know nothing.

          And yes, it’s exactly the same as asking PDs to read a book. That certainly proves how much more you know about this than I do.

          1. EH

            You may claim to be distracted by my comments–but i know you are really distracted by my amazing physical beauty. Alas, my heart is pledged to another.

            On topic, though: Trying something new is routinely done in all sorts of areas of law. Big firms do it. Little firms do it. The book analogy was yet another an attempt to point out the issue you keep ducking: do you think that EVERYONE who tests this sort of thing is unethical? Or do you think PDs are super special exceptions?

            1. SHG Post author

              And there you go and make me laugh. Dammit.

              Read the comments, and note that every criminal defense lawyer thinks this idea is totally outrageous and horrible. Ask yourself what they see that you aren’t seeing. Say to yourself, “what could I, not being a CDL, possibly be missing?” You’re a smart guy. I bet you can figure it out.

    2. JAV

      “But if it’s transparent and based on accurate data, what’s the issue?”
      One problem is there’s nothing to prove that the data is accurate, that’s the point of the “The key to this project is finding a public defender’s office to partner with to establish the efficacy of the new software…”.

      How many bad deals are okay based on bad information just to tweak the tool for the next defendant to get a slightly less bad deal, until it truly becomes “accurate data”?
      Ultimately, if I was ever in a defendant’s shoes, and my attorney told me to take the deal because the table said so, it would be a close race to see if I lost hope or got extremely pissed off first. I think my life and freedom could use a little irrationality from the person trying to defend it.

    3. Fyodor

      I don’t think that if it was just providing average sentences for courts and judges with transparency about how they’re gathered and calculated it would be as problematic. The lawyer could make an informed decision about how to weigh the information.

      From what’s described above it sounds though like there’s a proprietary prediction algorithm that considers a lot of factors (criminal history, age, etc) and spits out a predicted sentence/outcome. Without both (A) transparency about how it performs its prediction and (B) some sort significant history of accurate prediction I can’t see how it would be responsible to use this. And as Scott notes, the arcane nature of criminal procedures/crime classification/etc could make it very difficult for an outsider to even gather and use the data correctly.

      You wouldn’t (or shouldn’t) be able to introduce expert testimony that works this way into court because there’s no reason to trust that the signal would be greater than the noise.

      I’m also not aware of any other tool that lawyers use that works this way, where you feed information into a complete black box and get a result.

      1. SHG Post author

        Like EH, you’re still indulging in far too much speculation about this tool. We know nothing about it. Why speculate? But far more importantly, why put a client’s life at risk to test the efficacy of this black box? That’s the part that EH fails to grasp, that to participate in a test of this tool, with all that requires, it means allowing it to influence, if not dictate, choices that will impact a person’s life. That is a complete abdication of our ethical responsibility, no matter what the tool does or doesn’t do.

        Again, the lives of defendants (especially the poor) can never be offered up as guinea pigs to test any shiny tech, even if it later turns out to be the greatest thing ever. We cannot agree to take that risk with our clients’ lives.

  12. Fyodor

    Aside from the ethical problems of having criminal defendants be beta testers,, I’m not understanding how they are supposed to be validating the tool (or establishing its efficacy) in the first place. Is the idea that you track the overall outcomes of people who consider the tool vs. those who don’t?

    If the goal of the tool is to predict sentencing/plea outcomes based on a particular fact pattern it seems like you could more easily validate it by just collecting case information, predicting the results, and seeing how well it matches up after the fact.

    1. SHG Post author

      Comparing the results has failed pretty miserably for Abrams in the past as well. Without a grasp of law and practice, it’s impossible to distinguish the critical variables. They focus on what they think ought to matter, instead of what matters. It ends up a huge, worthless mess.

  13. David Woycechowsky

    Reminds me of some software tools developed for patent law, primarily by non-patent lawyers. Maybe good in theory, but inevitably awful in practice. I have turned down similar requests for help on a couple of occasions.

    1. SHG Post author

      As discussed in the context of legal tech, people who are really good with coding put their fairly bad idea into action and can’t understand why nobody wants to buy their ugly baby. Practicing lawyers try to explain why it sucks, and they scream louder about how they know everything there is to know, and it’s all those lawyers who just hate technology.

  14. Marc R

    “In previous work, I have found evidence of substantial disparities in public defender outcomes, primarily due to differences in plea bargaining.”

    Based on further statements I assume he means among PDs in the same office, as opposed to PDs versus private counsel. I agree with you in result (malpractice to use such a chart in place of anything else in deciding what deals to recommend) but let’s walk it through briefly with his sound premises.
    1. There are oftentimes large differences in PD outcomes (assuming the same charges and similar scoresheet/history).
    2. The differences usually are because of “differences” in plea deals.

    I think 1 and 2 are sound (true and logically safe) since most cases end in a plea. But the difference in pleas is because of (1) experience in negotiating, (2) the strength of the gov’t case and the weakness of your reasonable defenses, (3) publicity, (4) the status of the victims, (5) the judge, (6) the client;s finances, (7) witness quality and literally dozens of more reasons.

    In fact, the one reason not relevant is a matrix of probabilities. Versus private counsel, PDs have access to other PDs’ results and they could likely make their own chart far more accurate for the variables they deem important. But they haven’t made charts like this…nobody thought like this econ prof, er, law professor; OR the idea is a waste of time and counterproductive.

    New PDs need to learn how to make deals with the government. I don’t understand how a plea deal chart will help.

    Suppose ASA Bob offers your client 90 days on his third simple cocaine possession charge. His first arrest was a deferred prosecution, say drug court and fines then a nolle prosse and the second arrest was an adjudication and time served (the 20 days between bond hearing and arraignment). The chart, now exists (and won numerous super lawyer, dragon lawyer, and SHG blawg commendations) so you lay it on your desk between you and your client.

    “Well, 42% of direct pleas with three prior arrests and one adjudication (obviously no chart has info on scoresheets and any other individual specifics) result in an excess of 90 days in jail. Also, the government wins 89% of jury/bench trials with this charge, LEO eyewitnesses, and no C.I.”

    Suppose the chart has that many details and permutations. What am I supposed to do with it? Advise my client to take the deal? Avoid trial? But my client has a solid alibi? But surely within that 89% others had alibis too.

    The chart would be merely a waste of time, if nobody thought such a chart was a legal, moral or pragmatic way to decide anything about taking a deal, directly pleading to the court, and taking it to jury.

    Sorry for the length but usually “new wave tech” spam is ignored. The idea this can help, or even not harm, is sorely wrong. As a physicist and economist surely he appreciates zero means void of any value.

    1. Patrick Maupin

      Oh, it has value all right.

      “My boss is getting on my case because this program shows I’m not tightening the screws enough. What the hell — I’m a nice guy, so I’ll still go with my gut and offer you 60 months even though this program shows that any of my co-workers would have offered you 65. But you need to take this before my boss gets wind of it and makes me retract it.”

      1. Marc R

        I guess “work-product” exception to deny the State access…just kidding, I would love the government to rely on such a chart. Then at least they could be predictable. So, maybe this chart will help CDLs (assuming it’s all a ploy to give more mitigating statistics and make sure the government terms it the “plea offer bible”.) This econ prof maybe has some rube goldberg mechanisms than initially meet the eye.

        1. Patrick Maupin

          Predictable? Careful what you wish for; the end game is depressingly predictable.

          When the good professor finishes his Plea-O-Matic 2000 Mark V neural net, it will rival Skynet the FICO program in power inscrutability. Also in the magnitude of the single number (of months) that it spits out onto a perforated 1/3 page sheet using an OkiData dot matrix printer that needed a new ribbon 15 years ago.

          But the FICO number has a large voting bloc interested in pulling back the curtain to figure out which parameters cause terrible results in each case. Your client’s number has… you. And the DA won’t be terribly interested in revealing either algorithms or the sooper-sekret bad stuff he knows, real or imagined, about your client.

          Usually, when you protest that 20 years for a simple possession of half a joint can’t be right, you’ll be told “Oops, we forgot to enter something. It’s really 25 years.” But very occasionally, you may be told “Well, it would have been 25 years if it weren’t for (this Brady material that you wouldn’t have otherwise seen.)”

      2. dm

        PM, you hit upon exactly one of my main concerns. Particularly, how long until prosecutors adjust their offers to offset any “advantage” offered by the software. That the Defender’s office is using this software is not going to remain a secret for very long (people love to talk) and just as offers can, in most jurisdictions, be made worse depending on how far towards trial a defendant marches, so to could offers become crappier at the outset just to offset the supposed benefits of the sentencing/plea bargaining program.

        1. SHG Post author

          Meh. It will never come to that. What academics invariably fail to grasp is that we can’t force a prosecutor to make a fair offer. We can argue all we want, but the decision is theirs, and we can take it or go to trial. And no app will ever be capable of addressing the million nuances of a trial outcome in any particular case.

          The app says take the plea? So what? What if he’s innocent? What if the ADA is a screw up? What if the judge decides to ram a disposition down everyone’s throat at jury selection? What if the CW is a closet junkie? The app says don’t? What’s the app gonna say when the verdict comes in? It’s all nonsense.

    2. EH

      Here’s the science perspective. Sorry, Scott, it’s not short.

      Marc R
      …The difference in pleas is because of (1) experience in negotiating, (2) the strength of the gov’t case and the weakness of your reasonable defenses, (3) publicity, (4) the status of the victims, (5) the judge, (6) the client;s finances, (7) witness quality and literally dozens of more reasons

      As the sampling size gets large enough, there’s a lot of flattening out. Sure, Pete’s case may be different from Annie’s case. But once Pete and Annie have each done enough similar cases then #5, 6, and 7 will tend to average out. Absent some assignment specialization, #3 and 4 are also likely to average out.

      So . . .

      [Ed. Note: It can be short. Poof. It’s now short. That was easy.]

      1. SHG Post author

        There’s probably a name for this phenomenon, though I don’t know it. Because you’re knowledgeable (or, at least you claim you are) about the methodology prong, you have totally failed to see, or grasp, why this cannot either be tested on real people, and why it has essentially no chance of succeeding. Note that there has been a longitudinal experiment on the use of Big Data in criminal law. It was called the federal Sentencing Guidelines, and it was a horrific failure.

        But to the point, even if this should, miraculously, be beneficial to the majority of 1000 defendants, it would nonetheless require the sacrifice of every defendant below the mean. As every defendant, every single one, is entitled to individualized effective assistance of counsel, no lawyer can use a tool that would sacrifice the interests of his client for the greater good. That’s what makes it unethical. That’s what precludes its use.

        And as for testing its validity on real defendants by using the tool, it’s so outlandish as to require no explanation. We cannot use clients as guinea pigs. Somehow, all this has completely eluded you, while you obsess over the big data methodology and your baseless bias in favor of simplistic solutions. Now, you’re done.

        1. Keith

          There’s probably a name for this phenomenon, though I don’t know it.

          “Expertise in one field does not carry over into other fields. But experts
          often think so. The narrower their field of knowledge the more likely
          they are to think so.”
          – Robert A. Heinlein

Comments are closed.