When SCOTUS Gets The Data Wrong

Whether the Supreme Court’s decisions are grounded in falsehoods or falsehoods are picked up to justify a decision isn’t necessarily clear. It’s likely to happen both ways, as empiricism has become both tool and weapon in lawfare. But the fact that it happens really isn’t in dispute.

The Supreme Court has indeed said the risk that sex offenders will commit new crimes is “frightening and high.” That phrase, in a 2003 decision upholding Alaska’s sex offender registration law, has been exceptionally influential. It has appeared in more than 100 lower-court opinions, and it has helped justify laws that effectively banish registered sex offenders from many aspects of everyday life.

Except it’s simply false, and repeating a falsehood doesn’t make it any less false. It only makes it a legal truth by dint of being firmly established in caselaw. There are many such myths in the law, from the utility of dog hits to establish probable cause to the efficacy of junk science as admissible forensic tools. The problem is that if the Supreme Court says it’s so, then it’s so, even if it’s not so. Lawprof John Pfaff argues that rulings based on falsehoods need to be addressed.

Unfortunately, a report released by ProPublica on Tuesday suggests that the justices struggle with that task. Looking at a random sample of cases from 2011 to 2015, ProPublica found that the court cited faulty research or introduced its own errors in nearly a third of the 24 cases that relied on such facts.

Of course, whether the research was faulty is also subject to question, but that some bad research worms its way into the Court’s decision making is beyond doubt.

As ProPublica itself acknowledges, its sample is too small to draw any solid statistical inferences, but the results are still troubling. They are also not particularly surprising. None of the justices has any serious training in statistics, and the clerks who assist them are almost all recent law school graduates, who rarely have any formal statistical background. Empirical facts are central to what the court does, but its members lack expertise.

Two key points here. Justices are lawyers or lawprofs who become judges. The law is their core skillset and a generally accepted qualification for the job. What they are not is doctors, scientists or statisticians. They’re also not plumbers, engineers or computer hackers. The list of what they’re not can go on and one. No one expects them to possess expertise in every facet of human life of the cases that come before them.

And then there are the law clerks. It’s a huge, and exceptionally remunerative, feather in a baby lawyer’s cap to clerk for a Supreme Court justice, which is great for the baby lawyer but doesn’t really do much for the justice. They have no experience. They may be brilliant students, but they are barely qualified to carry a briefcase as a lawyer. They will studiously shepardize and bluebook, but their two cents isn’t worth, well, two cents. And when it comes to saving their justice from making a glaring factual error, the decisions speak for themselves.

Nor, sadly, do the justices feel terribly concerned about the occasional “oopsie.”

Sometimes justices seem almost amused by that lack. When presented with potentially critical empirical evidence in a major gerrymandering case this month, Chief Justice Roberts joked that “it may be simply my educational background” before describing the material as “sociological gobbledygook.”

Considering that a Supreme Court decision can affect millions of lives and billions of dollars, getting the facts wrong isn’t a small thing. While C.J. Roberts may well be right that much of what’s presented as empirical research is nothing more than “sociological gobbledygook,” if it serves as the hinge for a ruling, then it’s really important, potentially dangerous, gobbledygook.

Nor is the problem helped by amicus briefs pushing their own lies and pseudo-peer-reviewed studies on the Court.

The court has historically relied on amicus briefs, written by outside experts, to provide it with that broader empirical background and help compensate for its own institutional shortcomings. Unfortunately, these briefs are easily abused. In a 2014 article, Allison Orr Larsen, a law professor at William & Mary, pointed out that many amicus briefs include false or unsubstantiated empirical assertions, at least some of which make it into justices’ opinions.

Pfaff contends that this not only is a serious problem, but a problem that can and should be fixed.

In the 1980s, the legal expert Kenneth Culp Davis proposed that the court create an outside research organization, akin to the Congressional Research Service, to do research on its behalf. However worthwhile, the idea went nowhere.

Perhaps a more viable idea is one that Mr. Davis rejected: establish a group of technical advisers to the court. A small team of social scientists and statisticians could help justices sift through empirical evidence. There is no shortage of scholars with Ph.D.s who would be eager to do that work for the court.

There is certainly merit, and some surface appeal, in this proposal, but there’s grave danger as well. Not only does it cut strongly against our adversary system, but given the nature of academia in general, those eager to serve are also likely eager to see their agendas inform the justices. Scholarship hasn’t exactly proven itself unbiased lately, and its often the mother lode of the very faults complained of, making it as much the disease as the cure.

But there’s one additional wrinkle in Pfaff’s point.

Policy is a major part of the court’s docket now, whether it likes it or not. The justices cannot avoid adapting to this, and they cannot simply dismiss evidence they don’t understand as “gobbledygook.”

While it’s naive to believe that the Supreme Court can eliminate policy considerations altogether, judicial modesty suggests that it do everything possible to avoid crafting policy. That’s not its job. They’re just judges, not philosophers or priests, or more to the point, not legislators.

As Oliver Wendell Holmes famously said, “we do law, not justice.” That the Supreme Court has been used by lawfare to fill the gap created by a paralyzed Congress in achieving novel extensions or limitations of rights doesn’t make Nine (or 13 should there be no shift in time) the moral arbiters of good and evil. They’re just justices. Nothing more.


Discover more from Simple Justice

Subscribe to get the latest posts sent to your email.

10 thoughts on “When SCOTUS Gets The Data Wrong

  1. John Regan

    Well, SCOTUS doesn’t do “error correction”, but maybe that should change when it comes to their own errors, since only the SCOTUS can correct SCOTUS errors.

    Some committee of “experts” to tell them what to think seems like a particularly bad idea to me.

  2. orthodoc

    You are correct with the larger point here, of course, but you cannot say the court uttered a falsehood in Smith. That’s because it seems to have expressed only an opinion in its Opinion. “Frightening and high” is not an objective standard (though some people will insist it is, when they reject Derek Chauvin’s claim that he perceived George Floyd be exactly that).
    That said, Smith was based on McKune v. Lile, and that latter case did state a specific and wrong number: 80%. So this is, in fact, grounded in error.
    But let’s say everybody agrees that the true rate is (say) 1%.
    There still can be a good faith disagreement whether 1% is high enough to warrant banishing registered sex offenders from all aspects of everyday life. I say no, and you say no but the life-plus-cancer crowd disagrees (and both sides can scorn the feelz of the other)
    Indeed, that very problem –how much is too much?– underscores the debate on Covid lockdowns. Even if we can all agree that there is a ~1% case fatality rate in at-risk groups, some will find that frightening and high and others will march on (bravely or in denial, take your pick).

    1. Skink

      Please leave lawyerin’ to lawyers and never again post while drunk unless you’re in the Hotel Bar. We accept that because it’s income, but it’s otherwise completely dopey.

  3. Austin Collins

    Two questions/comments. For background, my graduate and doctoral work were in theoretical physics, heavily leaning towards statistical methods. I now work as a “Data Scientist”, developing AI/ML systems.

    1) To what extent, if any, do you consider the problem of courts accepting erroneous scientific arguments/data differently than you would courts mistakenly attributing credibility to a liar? If both sides presents experts that disagree, the court seems in a familiar spot in terms of evaluating the relative credibility of the two witnesses. I say seems as I’m not an attorney, and it’s a genuine question.

    2) It seems, from being a long time reader here and on Popehat, that many of the most basic tenets of at least the hard sciences (math, biology, chemistry, physics, geology, etc) are almost diametrically opposed from basic doctrines that give law/court its structure.

    For example, I believe I well understand how important stare decisis is to jurisprudence; same with the value in finality of a decision.

    The very stability stare decisis and finality seek to achieve is anathema to the scientific method. Any scientific theory, process, or understanding immediately gets kicked to the curb when convincing data emerges that our previous understanding was incorrect. Heck, we scientists *love* being wrong… tat’s when you get to learn and explain something new.

    Clearly the practical differences are immense. When science changes its mind about what gravity is, for example, we didn’t have to go back and readdress every problem(“case”) that was solved successfully the older way is just one example.

    My question is: The courts have underlying principles that have been evolved over a very long time. Many other disciplines have similar institutional principles. Courts presumably are able to function effectively without needed deep expertise in these fields…. why is science different. Sure, it’s basic principles are almost exactly opposite those of jurisprudence. So?

    Genuine questions. I look forward to the responses from this reader base.

    1. SHG Post author

      There’s a reason they call these “comments” instead of “free lawyer questions and answers for non-lawyers,” no matter how “genuine” your curiosity. I, by the way, take paypal.

      1. Austin Collins

        You act as if I were asking for free legal advice, rather than attempting to participate in a discussion that involves the very area of expertise you’re opining on.

        Just as easy to say, ‘your remarks aren’t relevant to lawyers who know better’, ‘Because. Deal with it,’

        Comments section is also called a ‘discussion thread’. You’ve appreciated specific input from several commenters in the past on areas outside your expertise. It was reasonable to post; particular given your poorly informed but quite vehement views on ‘academia’.

        1. SHG Post author

          So now you, whoever the fuck you are who comes here with nothing whatsoever to contribute but asking for my readers to sate your curiosity because you may be an ignoramus but are still sufficiently narcissistic to think this blog exists for you, think you’re in charge of what happens in my comments? Hit the pink button on the sidebar and fill out the form.

          And rent a sense of humor. You might like it.

          1. Miles

            You’re being terribly unfair to this entitled whiner. This was one of the funniest academic self-owns in a long time, and that’s a contribution to my enjoyment.

Comments are closed.