Tonight, How Science Turned Out To Be 95% Wrong

This story is one that would have generated an argument so boring and tedious that no one would be able to stand following it through from beginning to end.  At the beginning, there was science.  Maybe not science to real, honest-to-god type scientists, who would use charts and graphs, statistics, maybe even a Venn diagram for the hard-o-thinking, to make their point.

This was not science, but we, lawyers, and the lawyers who quit to become judges, aren’t scientists either.  So when some guy from the FBI Lab, which everybody knows is the best there is because everybody keeps saying it’s the best there is, says this is scientifically valid, who are we to question it?

Gamso lays it out.

They call themselves criminalists or forensic scientists or something with a fancy name. They come into court explaining that they’ve done these tests hundreds of times.  There are, they say, a whole bunch of points of comparison.  And when they find a match, they can point to those points.

You probably can’t see those things certainly not without the forensic scientist pointing them out.  That’s because you haven’t been properly trained.  Had you been, just like those forensic scientists, you could see them.  And like those forensic scientists, you’d know.  Because, as they like to say on the witness stand, they’re never wrong.

Except, of course, they are.

It’s about hair analysis this time.  There is an analysis for pretty much everything that’s left behind at a crime scene, where some guy paid by the government puts it under a microscope (they’re like magic, you know, and make things look BIGGER) and, boom, he can tell stuff about it that you and I can’t see.

Since we don’t have microscopes, and we wouldn’t know what to look for as our eyes are blurry from watching Real Housewives of Somewhere now that they took Honey Boo Boo off the air, we can’t really say for ourselves, but this is the FBI.  If you can’t trust the FBI, the best law enforcement outfit in the world, which everybody knows is the best there is because everybody keeps saying it’s the best there is, who can you trust?

Nearly 20 years ago, Fred Whitehurst blew the whistle on FBI’s flawed, and sometimes dishonest, forensic work.  The Inspector General issued a scathing report.  The Department of Justice promised to review everything, opening case after case.  And then to right the wrongs.

We’re still waiting.  Protip: When you get caught being all full of shit, say you’re sorry, assure everyone you will right the wrongs, and they will all mumble, “well, okay then,” and walk away and forget all about you. You don’t actually have to do anything because people have the memories of gnats and, well, just don’t care all that much unless it affects them.

Tonight, Al Jazeera America* (which is one of the few media outlets to take the time to look deeply into the workings of the criminal justice system) will be presenting Under the Microscope: The FBI Hair Cases.”  They were kind enough to let me watch an advance showing, and it’s excellent.

It relates the stories of innocent people convicted, at least in large part, on the testimony of forensic hair analysis.

[I]t tells of Joseph Sledge who spent 37 years in prison for a pair of rape murders he didn’t commit.  It was an horrific crime scene.  Blood everywhere.  Everywhere except on Joe Sledge.  There were palmprints in the blood.  They weren’t Joe’s. But there were also a handful of hairs.  Those, the FBI’s crack hair analysts said, those were just like Joe’s.  There are 16 points of comparison.  They matched up at all 16.

Sound familiar? Like fingerprints. Here 16 points matched. What are the chances they could be wrong? Well, 100%, as it turns out.

It tells the story of Kirk Odom, arrested at 18 for a rape he didn’t commit.  But the hair, the hair.  And Santae Tribble.  He was 17 when the cops got him for murder. There were 13 hairs.  It was a murder case.  The FBI had two of their forensic scientists look at them.  Match his hair, they both said.

The stories of the wrongfully convicted jerk some tears, as well they should, but for the lawyerly viewer, they serve merely to show that the science, which everyone believed with certainty to be real and accurate, was neither.  But the examples were the ones who were fortunate to learn of the error, to walk free.

You know what happened next.  DNA testing on the hairs.  On the hairs that convicted Joe and Kirk and Stantae.  On the hairs that weren’t theirs.  In Santae’s case, one of the hairs that sent him down for murder, one of the 13 hairs hairs that the crack guys from the FBI told the jury was just like his, so, you know, . . . . That hair came from a dog.

A fucking dog.  Who he says probably committed the crime.

Forensic hair analysis was used thousands of times by the FBI experts, whose somber faces and serious voices persuaded jurors that these were scientists just doing their job, taking killers and rapists off the streets, using proof beyond dispute.  It worked spectacularly well, and, but for the fact that it was all nonsense, let the rest of us sleep well knowing we were safe because of the FBI’s hard science-ish work.

But it wasn’t just a failure. It was an epic failure.

The bureau’s released some preliminary reports on its latest round of reviews.  26 out of 28 of the forensic scientists made false claims at trials.  David Colapinto, General Counsel of the National Whistleblowers Association explains,

We can now say, based on a statistically sizable sample of cases they have reviewed, [the FBI] were wrong 95% of the time.

And yet, the rest of the defendants convicted on hair analysis remain in prison, unaware of the fact that they were convicted on expert testimony that was wrong 95% of the time.

As we learned from Fred Whitehurst, the use of junk science wasn’t limited to hair analysis, but was pervasive at the FBI laboratory.  When you watch the show, remember that this is but one bit of bad science. If you view it as an expose on hair analysis only, you will miss the larger point. Substitute bite-mark analysis, duct tape analysis, finger-print analysis, hand-writing analysis, voice analysis, drugs, then do the math.

This isn’t science. This is manufactured industry created for the sole purpose of using science-sounding words to remove oversight by a jury and replace it with the conclusion of an FBI analyst, so as to guarantee conviction.  And it works spectacularly well, even though it’s largely bullshit.

*  They have a series they call “Fault Lines.” It’s not the same as this Fault Lines. Yes, one is video and the other is writing. Please don’t ask.


11 thoughts on “Tonight, How Science Turned Out To Be 95% Wrong

  1. Bob

    Popular Science has been at the forefront of examining Forensic Science. Or what passes for it.

    Fingerprint analysis which all of us who grew up on “The FBI” with E Zimbelis Junior was infallible. But further looksee says, that fingerprint analysis is a statistical match. Based on the sectioning of parts of the fingerprint to a known print. The match comes with a statistical confidence level. Remember the Confidence Interval from college stats days? But how often do we hear that CI?

    Smoke patterns and heat patterns have put many guys away when the patterns decry an arson event. Smoke and heat patterns have been absolutely trashed by a major university, little to no bearing. A friend who is an insurance investigator says the Arson Scientists are considered the jokes of the industry and that it really takes “I saw him pour gas and throw a match” to get someone convicted nowadays. Might as well kill a goat and look at the guts or read tea leaves.

    My personal favor is blood spatter analysis, when I hear what some guy with little to no formal training get up there and say with great confidence, he was this tall and he held the knife at this angle, swinging down with this rate, ….I shudder. As a physicist, seriously the variables are huge the variation in those variables are huge and the medium itself blood is not a consistent input to the whole process.

    The various CSI TV shows are part of the problem, yes the dedicated and quite good looking agents pour over the trunk and behold a fiber has been found. All rests on that fiber. Can we trace the fiber to the suspect???? Of course we can and by the end of the hour no less. And don’t get me started the cybergeniuses who with a couple of keystrokes make inferences and locate suspects within a 50 mile radius, who are blonde, 5 ft 5 in and have a dog named Fluffy.

    Just waiting for DNA analysis to fall apart. Do people really understand what DNA analysis entails?

    1. SHG Post author

      You can’t see me, but I’m shaking my head trying to understand what part of a post promoting a documentary gave you the idea that this was a great opportunity for some guy named Bob on the internet, who’s read Popular Science, to take over my soapbox and write a thousand words all about what Bob thinks of forensic science?

      Yes, Bob, I do know a bit about what DNA analysis entails. But what the fuck does that have to do with this post promoting a documentary, and who the fuck are you, a pseudonymous commenter on the internet, to hijack my blog to give a lecture here?

      1. Bob

        It’s not just hair. The whole system is scary.

        You asked…

        When you watch the show, remember that this is but one bit of bad science. If you view it as an expose on hair analysis only, you will miss the larger point. Substitute bite-mark analysis, duct tape analysis, finger-print analysis, hand-writing analysis, voice analysis, drugs, then do the math.

  2. Jonah

    One thing that’s never been clear to me from these stories (which the Washington Post covered extensively): was there any awareness by anyone of the weakness of hair-identification at the time? (And if so, we’re there attempts to preclude the evidence, or to bring in contrary evidence)? If not, then it’s not clear to me how a different legal rule would have accomplished anything — although it his certainly provides a cautionary tale about the need to examine such evidence closely as an advocate.

    1. SHG Post author

      There are always challenges when newly discovered forensic evidence is introduced, but it’s done by the criminal defense lawyer, all by himself, without benefit of Radley Balko and the blawgosphere and National Academies of Science, because nobody knows about it at first except that one criminal defense lawyer left to his own devices to fight off the FBI crime lab, the faux Association of Forensic Duct Tape Examiners who proclaim itself a real science and the Journal of Forensic Duct Tape.

      And once it gets its nose under the tent, by the time the rest of us find out it exists, it’s already admissible evidence and there’s some article written about how a duct tape examiner caught a heinous killer. By the time powerful voices speak out, it’s a done deal.

      1. Wrongway

        Ya Know, I’ve never looked at it that way..
        the State, with all of those resources & money (which are pretty much unlimited..) titles, degrees, expertise, labs, not to mention the ability to take as much time as they need..
        the defendant (with very limited resources & money..) with their CDL & a Law degree.. with a legal pad, & a person in jail..

        No wonder Plea deals are getting more popular.. Geez!

    2. zoe

      Those forensic examiners that knew of the weakness were either fired for questioning the weakness of the data, or they quit to become career ditch-diggers (a much more ethically-friendly occupation). Go figure, ditch-diggers don’t get call in to testify as “experts” on hair analysis.

      1. lawrence kaplan

        All those expert forensic duct tapists
        Are really great at capturing rapists.
        So what if their science is full of crap
        After all, it’s not they who’ll take the rap.

  3. DaveL

    So, let’s say we had a professional organization of witch dunkers, all with suitable degrees in witch dunking. Suppose abunch of them worked for the FBI. What are the chances their testimony would be admissible, say, in a New York criminal court?

Comments are closed.