Jumping To Conclusions

I sometimes (okay, often) criticize the New York Times for the way it frames stories. But I don't know that I've ever seen bias from them quite as blatant as this Agence France-Press report about a Stars and Stripes survey, headlined "Morale of US troops suffers in Iraq."

A survey of US troops in Iraq by the military newspaper Stars and Stripes has found that nearly three-quarters of those questioned said unit morale was low or average, and that nearly half did not plan to re-enlist.

So what's the problem? Where to begin.

1. Statistics without context: "nearly half did not plan to re-enlist." What does that mean? How many people normally don't plan to re-enlist? The article doesn't say. We have no idea if this is above or below average. (The article does cite Donald Rumsfeld for the claim that recruitment and retention remain good, but by not providing actual data, the article makes it sound like a he-said-she-said debate.)
   
2. "Of those questioned." The phrase glosses over the fact that the survey was actually non-scientific; the article admits, lower down, that the survey "was taken among soldiers who were available to answer questions and so was not a random sampling of troops." Which soldiers are more likely to answer questions about their morale? Disgruntled ones or ones who are content? You can't draw valid conclusions from a self-selected sample.
   
3. Both of those are just typical bad reporting; they're excusable. The big flaw, though, the one that tips the meter from "useless" to "biased," is the heart of that sentence: "Nearly three-quarters... said unit morale was low or average."

   What did the survey actually find?

   Some 34 percent of those surveyed said morale was low or very low, 27 percent said it was high or very high, and the rest said it was average.

   Yes, that's right. Only one third said that morale was low, not the "nearly three quarters" implied by AFP. They simply added up the people who said it was low with people who didn't say it was low, and pretended that they all gave the same answer. An equally valid story -- except that it wouldn't have been a story then -- would have been, "Nearly two-thirds of troops said unit morale was average or high."

So what we have here is a non-scientific survey in which 1/3 of respondents complain about their morale, and AFP tries to spin the story as 3/4 of troops having morale problems.

Moreover, the article provides no context: what percent of troops usually describe their morale as low? Presumably there are always some disgruntled people in the military, right? How much higher is this than normal? And wouldn't it be important to know whether their morale was low because they want better food or whether their morale is low because they think the mission is hopeless? It makes a big difference, don't you think?

Update: Incidentally, most newspapers appear to have framed the story in the negative (sample headlines: US forces in Iraq admit morale is low, US soldiers losing fight to keep up morale, Iraq briefs: Troops say morale low, and Survey: US Troops Suffering Low Morale). But I was amused to see Fox News' spin: Stars and Stripes Survey: Iraq Morale Could Be Better. That cup is half full, dammit.)