National Academies of Science: Trust Nothing

The ancient Greek philosophy of Skepticism posits that there is no such thing as truth or certainty.  The National Academies of Science has released its long awaited report, “Strengthening Forensic Science in the United States: A Path Forward.”  Skeptics are throwing parties.

The report is long, and only available to read on the NAS website on a painfully limited basis unless you prefer to purchase it.  I have yet to read more than portions of the summary and the press release, but it’s enough to get a flavor.


Forensic evidence is often offered in criminal prosecutions and civil litigation to support conclusions about individualization — in other words, to “match” a piece of evidence to a particular person, weapon, or other source. But with the exception of nuclear DNA analysis, the report says, no forensic method has been rigorously shown able to consistently, and with a high degree of certainty, demonstrate a connection between evidence and a specific individual or source.

In other words, everything that was taken as true, as real, as a necessary and inherent part of almost every criminal case, is no longer worthy of scientific reliance.  From drug tests to strange little black boxes that magically provide evidence that convicts people, the entire forensics paradigm is now suspect.  They aren’t saying that science doesn’t work, or that science isn’t real.  They are saying that the science used in courtrooms across our nation is not trustworthy.


Rigorous and mandatory certification programs for forensic scientists are currently lacking, the report says, as are strong standards and protocols for analyzing and reporting on evidence. And there is a dearth of peer-reviewed, published studies establishing the scientific bases and reliability of many forensic methods. Moreover, many forensic science labs are underfunded, understaffed, and have no effective oversight.

I can hear every “expert” across the country saying, “maybe someone else, but not me.”  How many will raise their hand with a sheepish grin across their face, admitting that their testimony is what the NAS is talking about?  I don’t see any hands raised.  I didn’t expect to. 

So the cat is out of the bag.  Decades of reliance on forensics are now in doubt.  But this isn’t the first time, and won’t be the last.  As scientific knowledge and tools developed, we kept getting the “new improved” version of whatever tools existed to prove facts.  Of course, if a scientific tool was so reliable that it could put a man in prison for life, what did it say when a new tool came along that was that much more reliable?  Courts embraced the new tool, confident that it was the latest and greatest in scientific proof, and lost no sleep over the demise of yesterday’s absolutely certain scientific method.  No one got hot and bothered by the conundrum.

So the NAS rips existing forensic practice to shreds and offers a roadmap to correcting junk science going forward.  In a rational world, somebody would ask, “What do we do with all those people who have been convicted on science that we now know to be so flawed as to be unreliable, or at best cannot be sufficiently certain is reliable to admit as evidence in court?”  By somebody, I mean a judge. 

The answer is that the prison doors will not be thrown open, with tens of thousand, hundreds of thousand, told that they are free to leave.  Society couldn’t possibly tolerate such a thing, and it would no doubt wreak havoc with public safety since it would allow the release of violent, dangerous and sociopathic prisoners.  And some of the innocent people wrongfully convicted in the process too.  It can’t happen, and it won’t.  Someone will craft a very thoughtful explanation about the finality of convictions, even in the face of pervasive scientific doubt, and the past will fade away.

The future is an open question.  As I earlier contended, judges are particularly poor gatekeepers when it comes to science.  They defer because they lack the ability to make a meaningful assessment.  Of course, the NAS report suggests that our nation’s “scientific experts” may be no better at making that assessment either, supporting their own personal disciplines for their own personal reasons while ignoring or dismissing the rampant flaws that permeate their expert opinions.  Perhaps there is no one left to trust.  Perhaps there is no truth to be found, whether by judge or scientist.

To the extent one can appreciate the depth and scope of the report without actually reading it in its entirety, it would seem to demand that there be an immediate stop to all use of forensic testimony in the courtroom until such time as the flaws are addressed and its reliability and consistency is satisfactorily established.  I do not suspect this will happen.

For those who wonder how many guilty men should go free lest an innocent be convicted, the answer has just philosophically risen to infinite.  We now know that the evidence routinely used to convict people is unreliable, both legally and scientifically.  The NAS has done its job, and the issue is now kicked to the courts. 

In a rational world, judges would refuse to admit scientific evidence from today forward, until such time as the evidence is proven legal reliable.  But today, judges will continue to allow the same evidence admitted yesterday and the day before, even though today we know it is not reliable.  They do this because to do otherwise would shut down the system, and that can never happen. 

So an ugly truth is shown, clearly and irrefutably:  Rhetoric about evidence and proof and reliability and integrity notwithstanding, it is more important that the wheels of the criminal justice system continue to grind, even if it means that bad evidence is used to convicted innocent people.  The ancient Greek Skeptics were wrong; we know with certainty that there is no such thing as justice.

Blawgversation:  Check out Grits post on the report as well

4 thoughts on “National Academies of Science: Trust Nothing

  1. Windypundit

    I haven’t read the report yet, but just from Grits’s excerpt, this part is awful: “…for many other forensic disciplines–such as fingerprint and toolmark analysis–no studies have been conducted of large populations to determine how many sources might share the same or similar features.” I don’t know how you lawyers see it, but from a scientific standpoint, that’s a devastating criticism.

    I suppose it’s possible that a strong enough result could be established without needing to study a large population, but the implication is that no numerical error estimates have been obtained. I’m not surprised that voodoo like bitemark analysis has never been rigorously tested, but I thought fingerprints were well established scientifically.

  2. martin

    re. the last paragraph of the post.

    Since apparently 100% scientific accuracy of proof cannot be obtained, the question now is how bad does the evidence have to be before it is rejected? What magic percentage number accuracy will be needed? How will we know we have reached this arbitrary number? I have no answer. I do know that currently too many cases of dubious evidence are allowed, see that bitemark nonsense or some of those agenda-driven activist influence. Sometimes it works, as in the largely discredited repressed memory cases, but not until after immense harm has been caused.
    However, not to allow the system to grind to a halt is not only a self-preservation measure by the system. If it ceased to function an awful lot of very unsavory people would roam the streets. I doubt anyone wants to pay that price.

    @windy:

    Since we don’t know for sure, we must assume that more than one person with the same print pattern exists. What are the chances that due to coincidence the wrong person is prosecuted? Miniscule.
    Question is, do we need to live with that?
    What are the chances that due to shoddy lab work or due to incompetence on any level the wrong person is prosecuted?
    Considerable.
    Do we want to do something about that? I doubt it. There already is much kicking, screaming and trench digging.

  3. Ken Mackenzie

    “Since we don’t know for sure, we must assume that more than one person with the same print pattern exists. What are the chances that due to coincidence the wrong person is prosecuted? Miniscule.”
    With computer data matching the chances are actually quite high. A person in the system with identical (or near identical) biometrics would be “matched” and prosecuted. I have a case where a fingerprint found in a stolen car was “matched” to my client 7 years later, long after he can be expected to account for where he was and what he was doing at the time.

Comments are closed.