It’s hard to argue identification when an expert states with certainty that the odds of a misidentification are 1 in 100 billion.  There’s always deterioration, or cross-contamination, but those arguments are hard to make comprehensible and not always available.  After all, it’s 1 in 100 billion, right?

Well, maybe not.  Maybe not even close.  In a lengthy article in the Los Angeles Times, it seems that law enforcement’s statistical showing may fall far short of their courtroom claims.  And the FBI is going to war to try to keep this information under wraps.

State crime lab analyst Kathryn Troyer was running tests on Arizona’s DNA database when she stumbled across two felons with remarkably similar genetic profiles.

The men matched at nine of the 13 locations on chromosomes, or loci, commonly used to distinguish people.

The FBI estimated the odds of unrelated people sharing those genetic markers to be as remote as 1 in 113 billion. But the mug shots of the two felons suggested that they were not related: One was black, the other white.

In the years after her 2001 discovery, Troyer found dozens of similar matches — each seeming to defy impossible odds.

Kathryn Troyer opened a big can of worms.  The Arizona DNA database contained only 65,000 samples, yet found matches within the database that defied the statistics.  Thomas Callaghan, head of the FBI’s CODIS unit was not amused.

This is of no consequence to the use of DNA to exclude identification, by showing that a person’s DNA did not match DNA found at a crime scene.  This is valid science.  But the claims that DNA provides positive identification, to the tune of 1 in a billion, “the FBI’s best estimates,” just got a kick in the teeth.

And what does the FBI have to say about the validity of their courtroom claims?

Several scientists and legal experts as well want to test the accuracy of official statistics using the nearly 6 million profiles in CODIS, the national system that includes most state and local databases.

FBI officials argue that, under their interpretation of federal law, use of CODIS is limited to criminal justice agencies. In their view, defense attorneys are allowed access to information about their specific cases, not the databases in general.

[Thomas Callaghan] urged authorities in several states to object to Arizona-style searches, advising them to tell courts that the probes could violate the privacy of convicted offenders, tie up crucial databases and even lead the FBI to expel offending states from CODIS — a penalty that could cripple states’ ability to solve crimes.

In other words, they refuse to allow their numbers to be scientifically tested.  While defense lawyers can test the database for their individual client, they can’t test the database for validity in general.  The problem with this approach is that the database remains significantly limited.  They don’t have one billion samples to compare your client’s DNA against, and if you can’t find a 9 point match in the existing database, then you’re out of luck.

Complaints about the way Troyer unearthed her matches seek to blunt the impact of her discovery.  They claim that it skews the result, since she compared each of the 65,000 DNA samples in the Arizona database against each other to see how many loci would match up.  In normal testing, one known sample would be compared with the 65,000, giving rise to a completely different result.  As soon as one marker did not match, the test sample would be excluded.

The problem is that the testimonial claims of uniqueness, and the number of matching loci, are all statistically based.  Like most scientific wonders, the basis for these claims go untested and assumed until they become part of the myth. 

In the 1990s, FBI scientists estimated the rarity of each genetic marker by extrapolating from sample populations of a few hundred people from various ethnic or racial groups. The estimates for each marker are multiplied across all 13 loci to come up with a rarity estimate for the entire profile.

These estimates make assumptions about how populations mate and whether genetic markers are independent of each other. They also don’t account for relatives.

Bruce Weir, a statistician at the University of Washington who has studied the issue, said these assumptions should be tested empirically in the national database system.

“Instead of saying we predict there will be a match, let’s open it up and look,” Weir said.

Some experts predict that given the rapid growth of CODIS, such a search would produce one or more examples of unrelated people who are identical at all 13 loci.

Such a discovery was once unimaginable.

It may well turn out that there is a number of loci that are sufficient to make that identification that the FBI would have us believe is beyond dispute.  But we’re not there yet.  In the meantime, judges (not always the best arbiters of good versus junk science because they don’t know squat about science which is why they became lawyers) stick the “tried and true” testimony despite evidence that it may not be accurate.

The article goes on to discuss various attacks on the database around the country, the FBI’s efforts to thwart the attempts to ascertain the validity of their claims and how various judges, some bold and some not so, have dealt with it. 

This article is a great starting point for learning about DNA fallacies, so even if you haven’t had a DNA case yet, eventually you will and this is stuff you need to know.