January 22, 2007  ·  Lessig

Jim Hightower has opened a site for his colorful, and often compelling, commentary. The flash animations are free for download under a CC BY-ND license. The first is depressingly apt (though the 600,000 figure is not, in my view, correct. But the correct number is still astonishingly high.)

  • http://insomnia.livejournal.com Mark Kraft

    “the 600,000 figure is not, in my view, correct”

    Actually, the latest Lancet Report estimate of excess morbidity above the statistical average in Iraq was NOT 600,000. It was 654,965, as of July 2006.

    Since that time, another half a year has transpired. That’s equivalent to about 15% of the total time period of the war thus far, and, given the increasing rate of violence, may quite possibly have been enough time for the morbidity figures the Lancet Report reached to go up by 20% or more.

    In other words, if you felt that the Lancet figures were way off, and that only about 500,000 Iraqis had died, then you have to also concede that perhaps another 20% — 600,000 — have died by now.

    Personally, I am curious as to what rationale has led you to conclude the Lancet figures were in error. Your living is based upon constructing rational arguments, so I would be interested in seeing whether your arguments in this matter are based strongly on reason, or simply disbelief or bellyfeel.

    The biggest stumbling block I’ve seen people face when they try to rationalize the large figures given by the Lancet Reports are that they are so much higher than other figures. The reason for this, however, is simple. What is being measured in the Lancet Report findings is fundamentally different than what is being measured in other body counts.

    In this case, what is being measured is excess morbidity above a previous statistical average. In pre-war Iraq, U.N. health officials were able to work with the Iraqis — often inside the country — to establish baseline statistics on how many people died each year from various causes.

    What the Johns Hopkins / Lancet Report has found since then is:
    1> More people are dying in Iraq from EVERY potential cause of death. Everything that could kill you has become increasingly lethal, due to a complete breakdown of medical facilities and Iraqi society.
    2> Violent causes of death have become the #1 killer of all Iraqis.

    Imagine if much of the U.S. had no electricity for most of the day, irregular phone service, frequently damaged roads, regular roadblocks and checkpoints, neighborhood turf battles making travel slow and potentially dangerous, carbombs and mass killings, educated individuals such as doctors fleeing the country, etc.

    If you had an immediate medical emergency, could you reasonably expect to get prompt lifesaving treatment? How many other risks — everything from disease, to malnutrition, to accidents, to unexploded ordinance — would your family potentially face that it never had to face in the past?

    I think a certain degree of historical comparison is useful in the case of this conflict. Between 3 and 5 percent of the Spanish population died in just three years of the Spanish Civil War, so how unusual would it be for approximately 2.3% of Iraqis to die during four years of this conflict?

    In other words, if the Lancet is correct, then conflict in Iraq is only about 55% as dangerous for the Iraqi population as compared to Spanish population during the Spanish Civil War.

    Now, some people respond to the claims of 655,000 dead Iraqis by saying “…but where are the bodies?” Clearly, Iraqi morgues are not signing off on this many dead people. The reason for these kinds of discrepencies are kind of obvious, however.

    1> Families and neighbors are burying their dead promptly, as their religion dictates.
    2> Iraqis are more likely to die in those areas where society — including hospitals and morgues — are the least functional. (i.e. Conflict makes going to the nearest hospital impossible.)
    3> Iraqi hospitals are often innundated with dead, and cannot / do not adequately account for all of them.
    4> Not every murdered Iraqi gets conveniently dumped off at the morgue. Many are presumably buried in mass graves somewhere, much like Saddam used to do. (Indeed, for those groups that round up their neighbors in the middle of the night and kill them, dumping the bodies in the middle of the street would present them with a greatly increased risk of getting caught. )

    The Iraqi government has given figures on bodies coming through their morgues, but they have NOT given figures of official death certificates issued, claiming to not have any centralized, accountable method in place. Under the current situations, it is possible to get a death certificate for a lost family member after the fact based on the word of a few witnesses, oftentimes without a body.

    What most people don’t realize is that there is a statistical way to determine the relative accuracy of the Johns Hopkins / Lancet report findings. Over 90% of those asked for proof of death for their family members in the Lancet survey provided a death certificate, so all that is necessary to determine the relative accuracy is to compare lists of dead from the Iraqi government vs. those Iraqis surveyed in the Lancet Report with proof of death. This would show that the Iraqi government “official” lists statistically undercounted the total verified deaths X percent… or it would show that the data used in the Lancet Survey was off. The Iraqi government has refused access to do this, however.

    I would suggest visiting the following sites if you want to know more about why the Johns Hopkins report is both important and the most accurate estimate of total Iraqi deaths that we have available to us:
    http://www.radioopensource.org/the-quantification-of-war/ (features an interview with the creators of the Lancet Survey, going into details on why their statistics are good.)

    http://commentisfree.guardian.co.uk/daniel_davies/2006/10/how_to_not_lie_with_statistics.html

  • http://www.aaronsw.com/ Aaron Swartz

    I have to ask: is there a reason you think your epidemiologial statistical judgment is better than the Lancet’s?

  • lessig

    I’d be happy (actually, I’d be VERY sad, but you know what I mean) to be proven wrong. But the analysis I’ve been following since the beginning of the war says this: http://www.iraqbodycount.net/press/pr14.php.

    Is this wrong?

  • http://insomnia.livejournal.com Mark Kraft

    I think the best way to respond to the Iraq Body Count’s assertions is to, first off, point out the fact that none of the authors of the IBC criticism had any experience whatsoever with calculating deaths from any war prior to Iraq — they were activists, basically — whereas the authors of the Johns Hopkins “Lancet Report” have decades of experience as epidemiologists, and have created the definitive epidemiological surveys for numerous world conflicts.

    By the IBC’s own admission, their tally of Iraqi civilians who have died are incomplete, reflecting only civilians whose deaths have been reported by the media. This is completely different than what the Lancet Report’s goals are, which is a largescale statistical study of increased morbidity above pre-war levels. The IBC numbers are only a relatively small subset of that larger result, which includes increased deaths from non-violent causes, deaths which aren’t reported by the media, deaths of combatants, etc. Clearly, the people at IBC are in no way an authority on epidemiological methods, or the statistical science behind it.

    So, basically we have an established, highly-developed, scientifically-established method of estimating increased morbidity in a large population, being called into question by people who have no experience with epidemiology, no credentials worth mentioning, and who are arguably biased in their desire to be the definitive source of information. Indeed, the team at IBC have fewer credentials relevant to epidemiology than most established global warming skeptics have relevant to atmospheric science.

    While IBC does a fair job of raising doubts, this isn’t a courtroom battle of reasonable doubt. It is a matter of whether what the epidemiologists at Johns Hopkins say is scientifically and statistically credible or not.

    Clearly, the staff of IBC have to show that either:
    1> The epidemiological methodology / established tenets of epidemiology are flawed
    or
    2> The data collected by the team at Johns Hopkins is so clearly outside the realm of standard probability as to be worthy of being dismissed.

    But look at the arguments made by IBC. NONE of them attack the methodology at all. Indeed, dozens of prominent epidemiologists from around the world have come forward, saying that the team at Johns Hopkins created a very solid epidemiological study, while exactly zero epidemiologists have come forward refuting the report that I could find. (You’re welcome to look youself.)

    Epidemiology is hardly a non-established science. In fact, it is accepted in the courtroom in order to determine probablity. The methodology of the Johns Hopkins/Lancet Report of 2006 was further refined to address critics of the first Lancet Report, in order to rule out possible concerns regarding samplings, bias, etc.

    So, the science and methodology is very strong, and was, infact, not refuted. Only the result reached is in question.

    Since the 2006 Lancet survey included the period of time contained in the 2004 survey, the researchers could directly compare the results of the two surveys for accuracy. In 2004, they estimated that somewhere in excess of 100,000 deaths had occurred from the time of the invasion until August 2004.
    Using data from the 2006 survey to look at the same time period, they estimated that the number of excess deaths during that time were about 112,000, entirely consistant with both survey’s margin of error.

    Either or both of surveys could be slightly in error, but well within statistical norms, basically… and considering that the first Lancet report threw out statistical “hot spots” such as Fallujah from their 2004 report because the deaths reported were so high that it would skew the overall results, I’d suggest that the first report underestimated total deaths and was responsible for most of the discrepency between the two.

    As Dr. Les Roberts said in an interview after the release of the first Lancet Report:
    “Please understand how extremely conservative we were: we did a survey estimating that ~285,000 people have died due to the first 18 months of invasion and occupation and we reported it as at least ~100,000.”

    Since the 2004 and 2006 Lancet reports come up with results well inside the established statistical variance, if the science/methodology are good, then what you’re essentially arguing is that BOTH the 2004 and 2006 Lancet surveys had results which were so inflated as to be meaningless, and that both were inflated by approximately the same level, so as to come up with consistant results.

    Frankly, that’s a bit like lightning striking twice in the same place. Not impossible, but definitely not likely.

    Over 92% of those asked from their sample were able to provide death certificates of those claimed to have died. The Iraqi Ministry of Health has previously admitted that their morgue counts are not consistant with the issuance of death certificates.

    I would suggest you read the sworn congressional testimony of Dr. Burnham and Dr. Roberts, the authors of the latest Johns Hopkins / Lancet report, as they go into great detail why their methodology is sound, and why their numbers are the best we can possibly get, under the circumstances.

    Also, below is a transcript of a key section of that recorded interview with Dr. Roberts, which directly answers their critics.

    “First of all, the UN numbers and the IBC numbers are 80% deaths in Baghdad, and it’s only 1/5th of the country. And our (most recent) data showed, and our first survey showed, and the Brooking Institutions data on attacks shows the violence rate in Baghdad is roughly the same as the national average.

    So if 80% are coming from Baghdad, we know there is gross, gross underreporting. There was one month according to the UN — the month of July 2006 — when in spite of all that fighting in Ramadi, there were exactly zero deaths reported from Anbar province.

    You know, it’s been pointed out “how could there possibly be 500 deaths a day going unnoticed?” Well, there has to be 400ish deaths a day from natural causes. Where are those bodies? What fraction of those have been reported into the Ministry of Health’s data? Well, I haven’t seen that data since 2005, but back then it was about 5%.

    I can’t think of any war, any war, where 10% of deaths were captured by the press. And the fact that the press is so focused in Baghdad, that all the deaths are so focused in Baghdad, should give us tremendous, tremendous pause.

    And also, I haven’t heard of any epidemiologist, I haven’t heard any people who have done this before, gone out and measured mortality during a time of war, say that our method is inappropriate, or that our results are implausible. It’s mostly folks who come from some other realm and this always happen that in times of war that a certain number are reported by the beligerents, and later on we realize that we had to multiply the number by several fold to come up, for example, in Vietnam, with the Fordham University demographic estimate of just about a million civilian deaths amongst Vietnamese. I believe that lots of people have trouble digesting this, but if we’re right, there have to be more deaths from violence than there are from natural causes, and that’s (consistant with) the evidence we see from Iraq. If we’re right, there has to be at least three times more bodies now than came in in 2002, and the evidence of that is overwhelming as well.

    We reported when our first survey was done in Eastern Congo that 1.7 million people died — about 15% of that from violence. That’s a rate that’s somewhat higher than we’ve seen in Iraq. And it went on the front page of the New York Times, it was just accepted as gospel, there was a UN resolution passed that week saying that all foriegn parties need to pull out.

    If you realize that our soldiers are attacked and injured at a rate that is higher than they were in Vietnam, and you start thinking about the levels of violence that are going on here, this death rate is not out of step with what is normal in times of war.”

    Now that I’ve given you all that information, I will try to refute the claims of IBC, point-by-point:

    “On average, a thousand Iraqis have been violently killed every single day in the first half of 2006, with less than a tenth of them being noticed by any public surveillance mechanisms”

    “Public surveillance mechanisms” meaning the press, in this case — i.e. the IBC numbers — which have documented more deaths than the official Iraqi government totals.

    In other words, IBC is complaining that for the Lancet report to be correct, then their tallies — which are only of a subset of the total deaths, as reported by the media — must be significantly less than the total increased morbidity figures for ALL Iraqis, from ALL causes, in ALL places in Iraq, including those regions where reporters cannot operate freely.

    The thing is, both IBC and the Lancet Report can be absolutely accurate in what they are counting, because they aren’t counting the same thing.

    The problem for the people over at the IBC is, if the standard scientific method for establishing these kinds of numbers win out, then what they are doing — and have done for years — is made largely irrelevant.

    As pointed out by Dr. Roberts, 80% of the deaths reported by Iraqi authorities are from within Baghdad, which is 1/5th of the country’s population. Given that both Lancet surveys and the Brookings Institution reports suggest Baghdad is roughly near the average level of deaths from violence within the country, this clearly suggests massive underreporting of deaths.

    Also, as the Lancet report points out, there is an undercounting of Iraqis dying from non-violent causes too, above the statistical average from before the war. (Imagine trying to get emergency medical care in a war zone with spotty phone service, long lines at roadblocks and checkpoints, neighborhood turf battles… every life-threatening emergency becomes more lethal in Iraq.)

    There are many other reasons offered by the creators of the Lancet report as to why excess Iraqi morbidity is higher than IBC account for in its limited counts, but the biggest reason is simple probability. With data consistent with the 2004 Lancet Report that mirrors the Brookings Institution’s findings about casualty percentages in Baghdad , as well as increased levels of Iraqi casualties seen in both IBC and Coalition figures, there is nothing about the Lancet survey results which is inconsistant with any of these other sources of data, assuming that these other sources do not report all increased sources of morbidity — which they don’t — and are not able to count all of the Iraqi dead, which they basically admit to.

    “Over 7% of the entire adult male population of Iraq has already been killed in violence, with no less than 10% in the worst affected areas covering most of central Iraq”

    Entirely believable. The Lancet Report indicates that the primary victims — and chief combattants — in the war have been adult males, and the percent who have died is certainly less than many other conflicts involving civil war or ethnic cleansing. Estimates for the Vietnam War are about 400% more than the Lancet Report, per capita. Indeed, the Rwandan genocide killed nearly 12% of the entire population, men and women, in only 3 months.

    “Half a million death certificates were received by families which were never officially recorded as having been issued”

    The authors of the Lancet Report specifically refuted this potential criticism in this document.

    Specifically, they said:
    “Even with the death certificate system, only about one-third of deaths were captured by the government’s surveillance system in the years before the current war, according to informed sources in Iraq. At a death rate of 5/1,000/year, in a population of 24 million, the government should have reported 120,000 deaths annually. In 2002, the government documented less than 40,000 from all sources. The ministry’s numbers are not likely to be more complete or accurate today.”

    “Some 800,000 or more Iraqis suffered blast wounds and other serious conflict-related injuries in the past two years, but less than a tenth of them received any kind of hospital treatment”

    Although Dr. Roberts and Burnham haven’t responded directly to this somewhat tangental issue, keep in mind what has already been pointed out:
    - This is a country that didn’t even keep accurate Death Certificate records during a time of peace. Now we are to presume they’re keeping accurate records of blast wounds, concussions, etc. during a time of war, when their medical facilities are most overwhelmed? Can’t we reasonably assume a similar — or larger — percentage of unreported cases today?

    - As pointed out by Dr. Roberts, 80% of the deaths reported by Iraqi authorities are from within Baghdad, which is 1/5th of the country’s population. Could Iraqi government records of treated victims also be similarly skewed, hiding massive underreporting of the actual numbers?

    - The Lancet Report found “hotspots” of extreme violence. In those situations, with roadblocks, snipers, neighborhood turf wars, lack of access to emergency medical treatment, etc. is is that unreasonable to assume a considerable amount of Iraqis have not recieved formal medical treatment?

    “The Coalition has killed far more Iraqis in the last year than in earlier years containing the initial massive “Shock and Awe” invasion and the major assaults on Falluja.”

    This claim is rather deceptively presented by the people at IBC, who seem to be trying to imply that the Coalition was responsible for more deaths in 2006 than in ALL prior years, as opposed to being responsible for more deaths in 2006 than in ANY previous year of the conflict.

    It says, right on the first page of the 2006 Lancet Report that “The proportion of deaths ascribed to coalition forces has diminished in 2006, although the actual numbers have increased”.

    Indeed, this statement is supported by the number of Iraqi deaths reported by the MultiNational Corps – Iraq. What’s more, the Coalition’s figures on increasing rates of Iraqi deaths are very much consistant with — and even higher than — the overall increase in the rate of death seen by the Lancet Report. See this graph for details.

    Ultimately, it all comes down to probability. Barring any truely compelling arguments that show that the 2006 Lancet report is wildly inconsistant with all the other data which has been gathered, you reasonably have to assume that 654.965 (as of July 2006) is the most likely estimate of the number of Iraqis who have died in Iraq above the statistical norm since the conflict began, because that is exactly what science tells us it is, given the data that was collected, which *IS* consistant with prior surveys and with known trends, and which is the absolute best data we’ve got to work with to determine this figure.

  • lessig

    You are a good and persuasive man, Mark. Thanks for the effort to make this all clear.

  • http://insomnia.livejournal.com Mark Kraft

    I wouldn’t go to such effort if I didn’t think the subject of that effort was worth persuading.

    Best to you –
    m.