Ah, science.  The mere mention of the word by a lawyer conjures up the image of hugging a leper.  No matter how hard we try to gain an expertise, or how well we put on a show, we are invariably trumped by the expert witness who is imbued with the aura of infallibility by his post-name letters and official title.  We can nip at their heels a bit, but it’s almost impossible to draw blood.

Of the many aspects where the pronouncement of infallibility causes jurors’ brains to seize, DNA has become the most damning and difficult.  DNA is a god in the courtroom.  The mere mention of three letters brings inquiry to a dead stop.  Oh, DNA?  Well, that’s that.  But of course, it’s only the beginning of the story rather than the end.

An article in New Scientist explains how DNA evidence, and the testimony that accompanies it, is simply a matter of chance.  It shows how the statistical probability of a match, you know, that 1 in 79 billion number that witnesses toss about to show that the sample comes from the defendant, can vary wildly based upon the methodology employed.  The same evidence can appear conclusive, or worthless,

In the Smith case, the sample containing another person’s DNA showed alleles at seven out of a possible 15 loci, but at four of these loci, the alleles matched those of both the victim and the defendant. “The 1 in 95,000 figure in effect treated these alleles as full-weight evidence that the DNA came from the victim, ignoring the alternative possibility that the allele we saw could have been from the defendant,” says Balding. If the opposite position is taken, and these alleles are ignored, you come up with a figure closer to 1 in 13. “It’s a question of which loci you consider,” he says.

Of course, who are we to question the methodology used in determining which “alleles” (whatever they may be) are used to confirm the match.  The science-talking guy says his numbers are absolutely accurate, and we’re just lawyers.

These cases are not isolated incidents. A recent study by John Butler at the US National Institute of Standards and Technology (NIST) in Gaithersburg, Maryland, found widespread variation in the statistics reported by individual laboratories. When he gave the same DNA evidence to 69 different US labs and asked them to provide conclusions about whether or not the suspect was a match, some labs reported RMP, others reported RMNE, while others gave no statistic at all. “There was a difference of about 10 orders of magnitude in terms of the statistical results that were obtained on the same samples,” says Butler.

Even among the labs using the same statistic- the RMNE- there were differences in the figure they came up with, depending on which alleles they chose to include or discard. What it could mean is that a jury presented with evidence from one lab is told that the chance of a match is around 1 in 100,000, say, while a different lab might say it is 1 in a quadrillion (1015).

Perhaps you’re getting the sense that all is not as “scientific” as its’ made out to be?  Well, this being a science article, the only logical step was to test the theory.

Our own survey of DNA laboratories backs up these conclusions. We asked 19 labs around the world which statistics they report when dealing with complex DNA mixtures. Six said that they reported RMP, six reported RMNE, while two said they reported LR. Five labs said they either report no statistics or a CPI/E. The type of statistic reported even varied within the same US state.

Efforts are being made within the scientific community to standardize practices and arrive at a better means of expressing the statistics.  The realization is that it’s too difficult to challenge the current regime before a jury, even though its scientific validity may be crap.

Earlier this year, the Scientific Working Group on DNA Analysis Methods (SWGDAM), which provides guidance to US forensic labs, issued its own recommendations regarding the use of statistics for DNA mixtures. It also proposes LR as a suitable statistic, as well as providing stronger guidance on how RMNE and RMP should be calculated. SWGDAM also insists that analysts must provide a statistic whenever they claim that someone’s DNA might be included in a mixture. “There are some labs that are just reporting that they think it’s a match- in their opinion,” says Butler. “That’s a problem, because the jury says: ‘Oh, it’s DNA? It matches? Guilty.'”

It’s that “guilty” thing that will get you, especially when you come to realize how meaningless the value of the match can be.

Lawyers have a tendency, when confronted with scientific evidence that’s presented as being beyond question, to shrug and walk away.  The thought process works this way: With scarce resources, why waste them challenging evidence that appears irrefutable when they can be put to more productive use.  When a lab returns DNA results that state with a degree of certainty to some ridiculous magnitude, despite the defendant’s seemingly insane claims that it wasn’t him, efforts are put to talking the defendant off the ledge rather than addressing the DNA. 

Maybe, with a far better understanding of the variability of the statistics, and particularly in combination with the other problems with DNA evidence and the omnipresent potential that some lab tech just manufactured the evidence to make sure the guilty guy goes down. the defendant’s pleas won’t seem nearly as insane? 

H/T Keith at Associates Mind.