Some very smart people, and Harvard prawf Cass Sunstein, offer a counterpoint to our current obsession with bias, and tacitly with adoration of empiricism. They call it “noise.”
Society has devoted a lot of attention to the problem of bias — and rightly so. But when it comes to mistaken judgments and unfortunate decisions, there is another type of error that attracts far less attention: noise.
To see the difference between bias and noise, consider your bathroom scale. If on average the readings it gives are too high (or too low), the scale is biased. If it shows different readings when you step on it several times in quick succession, the scale is noisy. (Cheap scales are likely to be both biased and noisy.) While bias is the average of errors, noise is their variability.
Why call it “noise”? Other than Sunstein’s bias toward words beginning with “n,” I dunno. It’s not the word I would choose. It’s not specific, it fails to convey the specialized meaning like “bias” does and it’s far too common and easily conflated with other issues. But then, they’re intellectuals, so what do I know? In any event, their point is valuable.
Although it is often ignored, noise is a large source of malfunction in society. In a 1981 study, for example, 208 federal judges were asked to determine the appropriate sentences for the same 16 cases. The cases were described by the characteristics of the offense (robbery or fraud, violent or not) and of the defendant (young or old, repeat or first-time offender, accomplice or principal). You might have expected judges to agree closely about such vignettes, which were stripped of distracting details and contained only relevant information.
To be fair, I would have never expected judges to closely agree, but then I know sentencing is voodoo and can vary wildly. One factor omitted from their study is the impact of advocacy on the variability of sentences. A curious omission, if common in academia.
So why isn’t this “noise” revealed by our plethora of empirical studies that we are reliably informed will save us from our biases?
Noise causes error, as does bias, but the two kinds of error are separate and independent. A company’s hiring decisions could be unbiased overall if some of its recruiters favor men and others favor women. However, its hiring decisions would be noisy, and the company would make many bad choices. Likewise, if one insurance policy is overpriced and another is underpriced by the same amount, the company is making two mistakes, even though there is no overall bias.
This raises one of the problems that’s intuitively recognized by some of us in the trenches, and ignored by proponents of statistical fixes. You may have a 50-50 split of ex-prosecutor judges and ex-public defender judges (note, not criminal defense lawyers, but PDs, because they’re marginalized and trendy). But if you get wheeled out to one, what difference does it make that the other exists? You get what you get, and the stats may equal out but they won’t do you any good. Individual bias will impact any individual outcome, but the big number will look unbiased. That’s noise.
In the sentencing realm, the United Stats Sentencing Guidelines were an effort to eliminate variables, to quiet the noise. They were horrible, not so much because the concept of bringing consistency to sentencing was a bad idea in itself, but because they were formulated out of whole cloth for political reasons and demonstrated, yet again, the one size does not fit all, Trying to eliminate noise might be a good thing, but the execution of such a task is hardly simple.
A third source of noise is less intuitive, although it is usually the largest: People can have not only different general tendencies (say, whether they are harsh or lenient) but also different patterns of assessment (say, which types of cases they believe merit being harsh or lenient about). Underwriters differ in their views of what is risky, and doctors in their views of which ailments require treatment. We celebrate the uniqueness of individuals, but we tend to forget that, when we expect consistency, uniqueness becomes a liability.
Are we not individuals? Are we not intersectional? Are we not unique? We celebrate rugged individualism on one side and empathy of the other, both of which tend to be highly idiosyncratic. Perhaps we think of ourselves as fairly normal, and that our values are correct and shared by most similarly normal people, but why do so many people disagree with us so strongly when we’re so obviously right? They must all be crazy, instead of smart, decent and normal like us.
Recognizing the existence of “noise” is the precursor to dealing with the variables that produce noise errors. But knowing that a problem exists doesn’t mean there’s a solution to it.
Once you become aware of noise, you can look for ways to reduce it. For instance, independent judgments from a number of people can be averaged (a frequent practice in forecasting). Guidelines, such as those often used in medicine, can help professionals reach better and more uniform decisions. As studies of hiring practices have consistently shown, imposing structure and discipline in interviews and other forms of assessment tends to improve judgments of job candidates.
There’s an old adage, everybody is entitled to their opinion, but they’re not entitled to their own facts. There’s another old adage that opinions are like assholes, everybody’s got one. The irony of noise reduction techniques is that they serve to create consistency at the expense of individualized assessment of variables that differ from person to person, situation to situation, by reducing complex decision making to its lowest common denominator.
There’s another old adage (actually a quote from Ralph Waldo Emerson’s 1841 essay, “Self-Reliance” ), “a foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.” Sometimes, you need to make a little noise. We’re hardly as “unique” as we wish we were, but reducing our world to the mean and medium and ignoring differences that matter is just as foolish.
Discover more from Simple Justice
Subscribe to get the latest posts sent to your email.

SHG,
Sure, you just had to take another shot at “The Sentence-O-Matic 1000.”
More seriously, we need more CDLs from the trenches. Just like you.
All the best.
RGK
I’m told that black is slimming.
Black also goes well with pearls.
Sunstein is trying to apply a term (noise) from engineering and real science to social science. The faux sciences do this a lot. In engineering it refers to variations in measurements due to random phenomena, such as the Brownian motion of molecules. Application to the behavior of sentient beings is a bit of a stretch.
The origin is someone far more significant than Sunstein.
[1] A. Einstein, “Über die von der molekularkinetischen theorie der wärme
geforderte bewegung von in ruhenden flüssigkeiten suspendierten teilchen,” (On
the movement of small particles suspended in a stationary liquid demanded by the
molecular-kinetic theory of heat), Annalen der Physik, vol. 17, pp. 549–560, 1905.
Einstein may well have taken the term from others, but his papers publicized the use of “noise” for statistics and established it in the sciences. Extending proper statistical analysis to the social sciences will be hard, and probably awaits a mind of comparable stature. Meanwhile, folks will flail about hampered by the absence of statistical understanding in their education.
(Thanks for the throw away lead-in. I’ve worked signal processing for decades but had not tracked down the origin of the terminology.)
To be fair, remember that Sunstein works at the university that has the third best engineering school on Mass Ave.
Every time I share my perspective here at SJ I steel myself for the greatness of being misunderstood.
There are at least two alternatives that would fix that problem, you know.
I take it you are not married. Misunderstanding appears to me to be the mother’s milk of marriage in particular and life in general. It’s not a problem unless you think you are the only one who is misunderstood and refuse to deal with it. See the Admiral for further suggestions.
Some people understand that communication is a two-part process. Others. not so much.
And some others know the rest of Ralph Waldo Emerson’s quote and are just trying to be funny.
Are we not men?
https://www.youtube.com/watch?v=I3bF0N081TE
(Not agitating for a return to video freedom, but I haven’t noticed Guitardave around these parts lately, so what the hell…)
Fair enough.
Don’t forget. In state court, most sentences are imposed based on an agreement between a prosecutor and a defense attorney. Two contributors — although I’m not sure that the result is an average that quiets the noise.
On the one hand, the role of defense lawyers in these purported miscarriages of justice is almost always ignored. It’s as if this is a system where only prosecutors and judges have any say, and defense lawyers either don’t exist or are as useless as rocks. Yes, we have a say in these outcomes, and if the offer sucks, we can advise the defendant to just say no.
Then again, given the parameters of sentencing schemes, there are constraints all around.