About . . . . Classes . . . . Books . . . . Vita . . . . Blog. . . . Podcast

by Peter Moskos

August 24, 2009

Sentence Length [or lies from the Heritage Foundation]

In a Heritage Foundation foundation report by Charles Stimson and Andrew Grossman, I learned a very surprising fact:
Convicted persons in the United States actually served less time in prison, on average, than the world average and the European average. Among the 35 countries surveyed on this question in 1998, the average time actually served in prison was 32.62 months. Europeans sentenced to prison served an average of 30.89 months. Those in the United States served an average of only 28 months.
From "Adult Time for Adult Crimes: Life Without Parole for Juvenile Killers and Violent Teens"

[Update/Correction: two hour later]

I'm generally no fan of the conservative Heritage Foundation. In fact, just between you and me, I generally hate them and everything they stand for. But I wasn't going to bring that up because I like to be tolerant and forgiving by nature. And if two of their researchers can write a good report, I'm more than happy to read it and learn.

And though it's rare to catch people in all-out balls-to-the-walls lies (though I've caught the DEA red handed on the issue of drug prices), there's nothing too rare about academic and moral dishonesty.

I decided to do a little fact checking, since, well, I didn't really believe that our prison sentences were shorter. Plus I don't trust the Heritage Foundation.

[The actual Heritage report, by the way, is about why we should keep sentencing juveniles to life without parole. It seems like a strange cause to fight for. What do they chant at rallies? But that's neither here nor there. I'm interested in the time people spend behind bars.]

First read the above quote from the Heritage Foundation and think about what it means.

Stimson and Grossman are not two fresh-faced grad students to be treated with kid gloves for bad statistical analysis. One is a "Senior Legal Fellow" and the other a "Senior Legal Policy Analyst." And besides, they're trying to influence policy and get more kids locked up forever.

Plus their report claims to be all about getting the "facts" right. And much of their report resonated with the cop in me. And 10 pages of endnotes certain gives them the ersatz veneer of rigorous academic analysis.

I copied the data (“Table 18.01: Average length of time actually served in prison”) to SPSS and crunched the numbers just like they did. Indeed, the average US sentence length is listed as 28 months and the mean length of time for the all countries listed is 32.62 months.

But anybody who does basic stats--and if you can copy the data from a table into a stats program, crunch the numbers, and publish them, you had better know basic statistics--should see two red flags. First is the two-decimal result. The original data is rounded to the nearest month. Using two-decimal places implies a statistical precision but in fact is statistical nonsense. Besides, who really cares about 1/100 of a month (just about 8 hours)?

The second red flag is the use of mean and not median for "average." The difference between the two matters. "Mean" is the average in the sense of adding up all the numbers and dividing by the total number of numbers. The "median" is the point at which half the numbers are above and half the numbers below. Both "mean" and "median" are averages, but "median" is generally better for analyses of numbers that have a set minimum (often zero) on one side but are open-ended on the other side (as in, they can go up to a gazillion!).

Take income. Median income is always lower than mean income because the millionaires (the outliers at the high end) push the mean average way up. If next year everybody in the U.S. made $1,000 less but Bill Gates, one person, made a trillion dollars more, the mean American income would go up by $2,000 per person! But the median income would go down $1,000, just like the average income.

So if Stimson and Grossman used median, the average would go from 33 to 26 months and the U.S. would go from below average to above average. So if they're using means, they're either statistically ignorant of trying to pull a fast one. But no matter, I’m not going to spend time writing all of this for a difference of seven months.

But wait... there's more.

2) Statistical outliers: Malcolm Gladwell didn’t invent them just to sell books. You generally shouldn’t include them in statistical analysis. The outliers here, in terms of sentence length, are Colombia, Qatar, Moldova, Latvia, and Suriname (with a mean of 90 months). Remove these four countries and the mean goes down to 23.5 months and the median to 19 months.

Now sometimes "outliers" aren't outliers but rather extreme case. If you're talking about average world prison sentence length, you shouldn't ignore America because there are more two million prisoners in America. But who cares if prisoners in Qatar serve 74 months? There are only 520 people in prison.

Anyway, the difference between 19 months versus 32.6 months matters, but it's still not what gets my goat.

Oh, I'm just getting started.

3) The table only includes 35 countries. Looking at each of these countries as equal for the purpose of statistical analysis is crazy. You've always got to apply qualitative common sense to quantitative analysis.

Surinam? 665 prisoners in the whole friggin country!

Montserrat? Montserr-who?! Where the hell is Montserrat!? What I'm trying to say is, who give a flying f*ck about Montserrat? What happens in Montserrat sure as hell must stay there because I didn't even remember that the capital of this Caribbean island was buried in 39 feet of volcanic mud in 1995 and abandoned. The total population of this non-nation is less than 5,000!

Give me a f*cking break. For statistical purposes, these countries doesn't exist. The US has two-point-three-friggin-million people behind bars! Equating Montserrat with the United States is bullsh*t... and the authors of this report should know this.

You ain't seen nothing yet!

4) “European average,” they say.

Now call me crazy, or chauvinistic, or “Old-Europe,” but when I say “Europe” in terms of criminal justice policy, I mean--and I think most people understand me to mean--the rich civilized part of Europe that's now part of the European Union. (By my calculations, Greece only joined Europe about 5 years ago.)

It’s not just geography. It’s culture. This report counts Moldova as European. Technically, yes, Moldova is part of Europe. But technically Israel is part of Asia. And Egypt and Morocco are part of Africa. But I don’t see too many Arabs in my neighborhood calling themselves African-American.

To say "European average" and give equal weight to (ie: not adjust for population) to Moldova and Germany is crazy. Oh, but wait, Germany and France aren’t even included in the data! How can you have a "European average" without Germany and France? No offense to Botswana and Mauritius (they're on the list), but it's not a world average if you don't have Russia, China, Indonesia, or India!

If you want to be honest, say 10 years ago Moldovan prisoners served more time than U.S. prisoners. But who gives a flying f*ck" about Moldova?! (Poor Moldova. I'm sure they're very nice. In fact, it says right in their tourism website that Moldova is, "rich in fertile soil and in hardworking and caring people.")

And no matter which countries I count as European, I can’t duplicate the report's average of "30.89" months. Seems to me the mean average for European countries included would actually be 34 months. But I’ll assume that was was just bad work rather than intentional dishonesty, since the correction would be in their favor.

So let's get back to the original question: do European prisoners serve more time than the U.S. average of 28 months? Here are some of the European countries listed:

Denmark: 3
Netherlands: 4
Iceland: 5
Ukraine: (yeah, what the hell, I’ll count the Ukraine as European): 5
Finland: 8
England and Wales: 14
Portugal: 26
Spain: 29

I'd bet good money that Germany and France (which aren't included in the data) fall somewhere between the Netherlands and England, with France being higher than Germany. That tends to be the way it is with those countries and criminal justice issues.

So why all this type over something as minor as sentence length? Because I don't like being played for a fool. Because I posted a lie thinking it was true. I posted it because the numbers really surprised me. I posted it because it went against what I believed.

I don’t like it when ideological groups spread lies. When people believe lies, and people tend to believe what they hear and read, the liars win. And liars, at least the ones that aren't pathological, tend to have an agenda.

Mind you, this is just the one paragraph I actually fact-checked. But coming from the intellectually empty and morally counterproductive Heritage Foundation, it shouldn't have come as any surprise.


Adrian said...

Thanks for taking the time to fact check.

Unknown said...

Great write up. For one, I love any post that mixes "flying f*ck" with quick primer on the difference between median and mean. But more importantly, a good take down of some specious data...

PCM said...

Thanks. I'm sure I'm not the first person to face the dilemma: how am I going to write about statistics and not be boring?!

Anonymous said...

Thanks for the analysis, Peter. This "study" is disgraceful. Those who don't have the time, expertise, or inclination to study the data behind the conclusions are likely to form opinions that adversely affect policy-making decisions.

I just finished a grad course in stats for my MCJ degree, where we learned about the very issues you discuss. I'm forwarding a link to your blog so that my fellow classmates can read the post, too. Now that the arduous semester is over, perhaps they are in a better position to reflect positively upon all they have learned and how they can apply their newly acquired knowledge.

Anonymous said...

I think there is an even simpler explanation: the question was ambiguous and thus some countries counted probation (or time served) AS ZERO and some countries did not count them at all.

That's the only sane way to explain the MASSIVE difference between, say, Ukraine and Hungary. Anyone who think Ukraine is far softer on criminals than most euro countries must be on drugs.

Speaking of drugs, since they make up far larger fraction of convictions in the U.S. than in other countries, you end up, in essence, comparing the sentence for drug possession in the U.S. with a conviction for major assault in other countries.

PCM said...

Good points. Thank you.