So Sayeth The Code

At Fault Lines, Jessica Gabel Cino emphasized the magical influence DNA has had on the criminal justice system.

With the exception of DNA, no single forensic technique has the potential ability to establish a definitive link of an evidence sample to its source.   Potential, however, can be very different from reality, and even DNA evidence has its limitations and weaknesses. The latest reality check: the statistics utilized in the interpretation of DNA mixture evidence could be so wrong that they require retesting of thousands of cases.

That reality check, brought to you courtesy of the Texas Forensic Science Commission, which was aware of the problem for a while but finally got around to making it public, is that DNA mixture analysis not only failed to meet the one in gazillion probability to which prosecution analysts typically testified, but was in fact one in fifty.  Oops. Sorry. We bad.

But that comes from the maturation process of science, a reality that’s always denied until it can’t be denied any longer. Today’s gold standard is tomorrow’s total malarkey.

Studies have disputed the scientific validity of pattern matching in bite marks,arson, hair and fiber, shaken baby syndrome diagnoses, ballistics, dog-scent lineups, blood spatter evidence, and fingerprint matching. Massachusetts is struggling to handle the fallout from a crime laboratory technician’s forgery of results that tainted evidence in tens of thousands of criminal cases. And the Innocence Project reports that bad forensic science contributed to the wrongful convictions of 47 percent of exonerees.

They rise. They are accepted as real. They convict. They fall. This is the forensic scientific method, as distinguished from real scientific method. It’s not like lives actually depend on the reliability and accuracy of this science. Oh wait.

But there is yet another, darker, deeper level of problems that is emerging, and worse yet, being accepted.  The proprietary code problem.

Take California. Defendant Martell Chubbs currently faces murder charges for a 1977 cold case in which the only evidence against him is a DNA match by a proprietary computer program. Chubbs, who ran a small home-repair business at the time of his arrest, asked to inspect the software’s source code in order to challenge the accuracy of its results.

Chubbs sought to determine whether the code properly implements established scientific procedures for DNA matching and if it operates the way its manufacturer claims. But the manufacturer argued that the defense attorney might steal or duplicate the code and cause the company to lose money. The court denied Chubbs’ request, leaving him free to examine the state’s expert witness but not the tool that the witness relied on. Courts in Pennsylvania, North Carolina, Florida, and elsewhere have made similar rulings.

So you want to have a business that makes you rich by creating some code that puts people in prison?  How nice for you.  That this is your position, that your secret code is magic and wonderful, but you can’t reveal it or someone else is going to steal your loot, is a perfectly viable argument to make in a capitalist society where making money is an admirable goal.

But then, there is still that gatekeeper role judges are expected to play.  You don’t want to reveal your code?  Fair enough. But then, your code can’t be used to form the basis of an expert opinion at trial. You’re allowed to make money. You are not entitled to have your magic code used in a court of law.

Some judges, however, haven’t seen their role as clearly as other judges.

We need to trust new technologies to help us find and convict criminals but also to exonerate the innocent. Proprietary software interferes with that trust in a growing number of investigative and forensic devices, from DNA testing to facial recognition software to algorithms that tell police where to look for future crimes.

While “trust” is certainly a virtue, it is not the test of admissibility of scientific testing in court.  When it comes to determining whether expert testimony should be admitted into evidence, jurisdictions rely on one of two tests, Frye or Daubert.

The Frye test is essentially one of simplistic deference: if a scientific technique is generally accepted by the relevant scientific community, then it’s good enough for court.  Daubert, on the other hand, would require the scientific technique to be reasonably proved as valid by scientific method.  Trust is a feeling. Science is not.

So how can either test be met when the underlying code is proprietary?  After all, it’s not like any other person in the scientific community has a basis aside from the proprietary source to reach consensus that it’s valid, and certainly it can’t be proved by scientific method since no one else has the code to test.

Jeez, those are tough questions for someone whose job it is to serve as gatekeeper, to determine whether junk science gets to stroll into the courtroom as if it was real.  Except that it’s really not hard at all, unless the judge feels some overwhelming need to accommodate the prosecution’s desire to introduce expert testimony based on proprietary coding rather than do his job.

Isn’t subjecting the testifying expert to the crucible of cross good enough? As much as cross-examination, as a concept, is invaluable, in this instance, it’s tantamount to crossing a black box. Black boxes don’t talk. It’s a pointless endeavor when the only response is, “the black box says so.”

The problem here, which is not only very real, but may well come back to be yet another disgrace in the annals of forensic junk science despite there being so little room left on the Wall of Shame.  Unlike so many past failures, the problem here isn’t based upon judicial scientific cluelessness combined with the arrogance of power that allows judges to believe that they know enough to make a science-y decision.  While that’s a huge problem, it isn’t the problem here.

The rules would strike anyone who isn’t desperately trying to facilitate the prosecution’s introduction of evidence to convict as remarkably clear and straightforward: You can keep your proprietary code a big secret, or you can have your cash cow used as evidence in a court of law, but you can’t have both.  There is no other legally acceptable option. Problem solved.

43 thoughts on “So Sayeth The Code

  1. mb

    Creating problems, like the use of potentially worthless evidence in court, to solve problems that don’t exist, like defense attorneys and their teams stealing proprietary software, is not supposed to be a necessary element of a free, capitalist society. And yet, here we are again.

    1. SHG Post author

      It’s possible that the code might get out. So what? That’s the price of having it admissible in court, that it be subject to the scientific method. No one promised you get to have secrets plus the benefit of it being admissible in evidence.

      1. mb

        It could happen. It won’t, but it could. And they can put other protections in place. And they could buy insurance in case it did happen. And there would definitely be someone who they could sue if it happened.

        But yes, it could happen.

      2. Gregg

        It seems like there could be a middle ground between keeping the code completely secret and releasing it entirely to the public. Are there third parties (gov’t organizations, defense attorney groups, etc.) who could examine the code for potential flaws, while still protecting the intellectual property in the code from outside interests?

        1. Anne Ominous

          No, there is no middle ground. Because in order to test it scientifically, one must know how it works. In order to know how it works, one must see the code. There is no third option.

          One way to bypass this would be to have it verified by an independent expert auditing agency that is bound by confidentiality and trusted by all parties. Needless to say, such animals are rare. And even then, they make mistakes.

          1. Akatsukami

            I strongly disagree. “Black box” testing is preferred in software development, as it eliminates unconscious bias toward “happy path” testing on the part of the testers.

    2. Lurking Again

      The problem with any software is that some of the algorithms are not that easy to code correctly. Also, the software company did offer up of the validation testing done on the code. As a programmer this is a serious red flag because validation requires using inputs with known outputs with particular attention paid to “edge cases”.

      1. mb

        The nonexistent problem I was talking about is software companies going out of business because the court system leaks their intellectual property. Even if it were an issue that a computer could do perfectly, we’re still discarding a real concern to the rights of the accused for a phony, not going to happen, concern about the health of the tech development sector.

  2. Patrick Maupin

    Unlike you, I have seen Mark Perlin’s source code, and I am prepared to testify that it is very sophisticated, and infallible in its binary decision of whether he should use his ouija board or his Magic 8 Ball to decide the outcome of any particular case.

    It would be impossible for the layman to understand this, or even for a specialist to replicate it, for the simple reason that it is highly tuned for exactly that ouija board and that Magic 8 Ball, and any competent expert knows that (like people) no two Magic 8 Balls are identical.

    Mark Perlin’s reluctance to release his source code is heroic; we cannot trust lesser programmers to be able to properly adapt it to whatever random occult objects are lying around, and rather than unleash the inevitable havoc that would arise were such adaptations attempted; Mark has wisely saved all of us from looking at source code that would no doubt permanently disfigure any who gazed at it long enough to scratch the surface of understanding.

    1. SHG Post author

      So it won’t prove out, but not because it’s wrong, but because no one else knows how to us it because it’s too brilliant. It’s a brillianty self-serving argument.

      1. zoe

        I’m gonna use the same argument as a jury member. I am a genius and the other jury members are not. No need to argue. No need to question how I came to my opinion. Only my opinion matters. As a finder of facts, I can not find any facts for a code I can not see.

        1. SHG Post author

          And what’s your point? A juror can always believe or disbelieve testimony. The fact is that most, if not all, jurors believe scientific evidence as if it was the word of God. And as a juror, you would have no clue about the undisclosed code. That wouldn’t be a part of the trial.

          1. bud

            And the attorney -defense or prosecution, whichever one is depending on this “evidence” – will work like hell to keep anyone with any sort of scientific background who might be skeptical because of their own experiences with less than perfect experiments.

            How do I know this? I’ve been booted out of the jury pool on two different occasions with peremptory challenges. Both of those cases (civil) involved “expert” testimony from ‘scientists”. I have a BSEE.

            1. SHG Post author

              You raise an issue that’s worthy of a post in itself. While I personally prefer jurors who are smart and well-educated, particularly with science backgrounds, as they are less inclined to the fuzzy thinking and rhetoric of humanities types, it’s almost invariably the prosecution who strikes them. Remember, the prosecution is the side admitting the scientific evidence, not the defense.

              But it raises a secondary problem: a juror with too much expertise in an area that is the subject of expert testimony at trial becomes an “unsworn witness” in the jury room, using his background to explain the testimony to the other jurors. Sometimes this juror knows his stuff. Sometimes, even with a BSEE, he’s a moron who talks the talk but is otherwise clueless. The other jurors will listen, however, because he’s got the credentials. And since all of this is happening in the privacy of deliberations, no one knows if the juror is correct or just a nutjob with a degree convincing the other jurors who lack a science degree which way to vote.

            2. Rick J

              If the other jurors are also educated then it reduces the chances they are impressed by the “expert’ juror’s credentials alone.

              The best solution is letting everyone see the evidence, (in this case the code that supposedly predicts whose DNA is whose), and letting all sides enter evidence which validates or refutes the way in which the code does its work.

            3. SHG Post author

              Just because jurors are well educated doesn’t mean they have the same educated. A French Lit PhD doesn’t have much to offer on science questions, and will still defer to the engineer/scientist.

              But you’re confusing two issues. The issue of admissibility of the science (the subject of the post) is a question for the judge, and will never be heard by the jury. If the code isn’t valid, it shouldn’t come in at all.

            4. RickJ

              Well, SHG, this educated man has the confidence to not blindly accept your expert option in this matter. I trust others would be likely assured of their own powers to discern.

              And, yes, unless it is entered as evidence the jury would never get to vet it, that was my point, it needs to be a part of the testimony to increase the chances for a correct decision. The Judge in this case did not allow that common sense opportunity.

      2. Patrick Maupin

        Could be brilliant, could be a dart board, could be a legal pad and pen. How can anybody know? The Magic 8 Ball is arguably more scientific than whatever he’s got.

        Somebody just went to significant trouble to prove that VW diesels pollute. This should have been as obvious to anyone who’s ever been behind one — just as obvious as the twin concepts that both Firestone tires and vehicles with high centers of gravity are terrible ideas. Nevertheless, the proof is now out there, and it’s easily replicable by anyone willing to buy a VW diesel (those should be cheap now, right?) and willing to buy or build a NOx detector.

        But that cannot possibly happen with this DNA stuff — if the methodology is not disclosed, how can you challenge it? It is the antithesis of the scientific method, and if the defense attorney cannot explain that to the judge, then at least one of them needs to be taken out back and (what is it this week — shooting or wood chipper? I always forget).

        1. Micha Elyi

          “…as obvious as the twin concepts that both Firestone tires and vehicles with high centers of gravity are terrible ideas.”–Patrick Maupin

          That’s not obvious. (Hint: Ford insisted on underinflating the Firestone tires. Had they been underinflated Goodyear or Michelin tires then you’d be repeating the same bromide but about a different brand.)

    2. EndOfPatience


      Well, that’s great! Maybe Mark Perlin could turn his piercing intellect and incredible talents to solve other pressing problems, such as Catastrophic Anthropogenic Global Warming (TM).

      Oh, wait …

  3. Fubar

    My new black box invention’s dramatic,
    and it makes prosecutors ecstatic.
    How it works, I conceal
    and I’ll never reveal.
    It’s called the Conviction-O-Matic™!

  4. Gavin

    I’m very confused by the need to keep the code secret. Right now, the code is copyrighted and it’s a trade secret. Sharing the code will mean it’s not a trade secret, but it’s still copyrighted.

    Since any other program made for examining evidence will also have its code shared, it seems that it shouldn’t be too hard to keep an eye out for copyright violations.

    I suppose that the code’s authors might argue that their clever ideas will be stolen if they lose trade secret protection; but if the ideas are truly clever, they can patent them.

    So, really, I don’t see much loss to the owners if the code is published. The arguments by the owners just seem specious. Though, in an amazing coincidence they serve the interests of their customers, the prosecutors.

    1. SHG Post author

      Damn, this is getting beyond stupid. It doesn’t matter whether the code’s authors are right or wrong, whether there are alternatives or not, whether it’s great code or awful code. NONE OF THIS IS MATTERS!!!

      The only thing that matters is they refused to reveal it and it was allowed anyway. That’s the point, not whether the decision not to reveal it was stupid, or whether there were alternatives. They didn’t. That’s the point.

  5. John S.

    This has come up multiple times that I know of in the context of DUI cases with breathalyzers, in both Minnesota and New Jersey. Notably, both times the defendant won, because this couldn’t be a more duh situation.

    So even ignoring the fact that this couldn’t be more duh, this judge managed to fuck up a call so easy, even New Jersey got it. I mean, true they’re no Florida, but when the standards of the “great” “state” of “New” Jersey are out of your grasp, you really have to ask where you went wrong in life.

    1. Andrew

      I don’t think I’d say the defendant “won” in Minnesota. True, a defense expert eventually got to see the source code, and said expert found problems. But, who cares what’s in the source code when the machine’s operator magically knows there is nothing wrong with the machine? In re Source Code Evidentiary Hearings, 816 N.W.2d 525 (Minn. 2012).

      1. SHG Post author

        On the contrary, Andrew, the first battle is to get the source code, and the defendant won. It’s not that code is good or bad, but it was tested. That’s the issue here.

  6. David

    If the program was used by the investigators to suggest a match but the actual match was done by an independent expert, is the program actually relevant? The cited articles don’t really give enough technical details to say what the program actually did and if the results of the program are being introduced as evidence.

    If the program suggested a possible match but the states expert is comparing the two DNA samples and testifying as to the probability of this being a unique match, then it seems reasonable that only the expert testimony can be attacked.

    If the expert relied on the program to provide the answer and is just reciting the output, then the situation would seem quite different. The issue seems to hinge on the specific use of the program and whether is actually generated evidence.

    For example a program that searches fingerprints on file and provides possible matches for a human to review and determine if a match exists, is this relevant to a match?

    1. Mark Draughn

      From the description in the Background section of the opinion, it sounds like the code in question was used to calculate the relative likelihoods of several people’s contributions to a mixed partial DNA sample found on the victim, based on known frequencies of the various genetic markers in the human population. So basically the software said the defendant was 1.62 quintillion times more likely to be the murderer and that’s what the expert testified to. It sounds like the software’s accuracy was very relevant.

      1. Micha Elyi

        “1.62 quintillion times” zero is still zero.

        Even if “1.62 quintillion times” very long odds means that the defendant is 10 times more likely to be guilty than a random person grabbed off the street, that’s still not guilt beyond a reasonable doubt. 100 times, even.

        Law school is popular ’cause there’d be no math.

  7. Pingback: Instapundit » Blog Archive » DNA EVIDENCE: not as simple as it looks on CSI….

  8. hmi

    I’d love to know if there would be any obstacle to hauling the developer of said proprietary code to the witness stand, to be asked to swear under oath that the code takes proper account of a complete list of scientific guidelines, as well as to provide details (without revealing trade secrets, of course) of the decision tree implemented in the code.

    1. SHG Post author

      Well, there would be no reason for SJ to exist other than to answer things you would love to know, even if they’re entirely irrelevant to any point of the post whatsoever. Yes, there would be an obstacle.

      Q: Do you solemnly swear that your code is absolutely, positively, fabulously wonderful?

      A: I do.

      Prosecurtor: Well, there ya go, Judge. Problem solved.

      Judge: Okay then. Proceed.

    1. SHG Post author

      No, for two basic reasons. First, because the judge is gatekeeper, not proxies. Second, because our system of law is an adversary system, where due process requires that the defense gets to serve its function.

      1. Akatsukami

        I don’t see the significance of this (which doesn’t mean that it’s not significant, just that I don’t see it). I would imagine the conversation going about like this:

        Witness: The FOO software gave a probability of 99.44% that the defendant’s DNA matches the sample.
        Defense Counsel: Objection, Your Honor! It has not been established that FOO is reliable.
        Prosecution: Your Honor, we submit these n test suite results as showing that FOO is reliable.
        Judge: {Answer probably depends on the value of n.

  9. Master Coder

    Give me a week and I will have a DNA testing machine that will find the defendant NOT GUILTY. No your honor, you can NOT see my code or know how my machine works. All you need to know is that it says you must release the defendant.


  10. Dragoness Eclectic

    I’m a software engineer, not a lawyer–from a software engineering viewpoint, the judge got this wrong. Without inspecting the source code–or better yet, decompiling the code that’s in the machine, because how do you know that they are providing you the real source code?–that machine could be doing anything, including generating random numbers to amuse the prosecutor. There are more than a few cases in the annals of computer programming of “black box”, proprietary programs that cost a lot of money not actually doing what they claimed to do.

  11. pigpen51

    The solution seems quite simple to me, a under educated but fairly high i.q. common man. From this day forward, DNA will be allowed to exonerate, but not to convict.
    Am I missing something? Or just a little naïve? ( Or a lot ) I sometimes forget and go back to that paper called, what is it, oh yeah, the Constitution.

Comments are closed.