Tyranny of numbers

Iran’s COVID-19 crisis has gotten lots of media attention because Iran was an early epicenter of the pandemic, and because of its geopolitical significance.  The crisis has also intensified pressure on the Trump administration not to heed calls by former world leaders, former US diplomats, and influential newspaper editorials, to ease sanctions against Iran.  For some proponents of regime change in Iran the epidemic is more than a human tragedy: it is also the proverbial straw that could break the camel’s back, which is why throwing some light on the intensity of the crisis is timely.

It is difficult to find a reference to Iran’s COVID-19 case without the words “disaster” or “catastrophe” in it.  Speculative numbers of deaths as high as 3.5 million estimated by Iranian academics have been widely cited. A simulation study by two researchers from Virginia Tech and MIT estimates the actual death count may be 11 times the official numbers.  There is no doubt that official data are underestimates, though we do not know by how much.  At the same time, I find it hard to believe that official numbers undercount true cases by a factor of 11, or that it is remotely possible that 3.5 million Iranians will die from the disease (though, to be fair, this number was cited as the worst-case scenario).

In any case, before discussing the veracity of the official data or the accuracy of the simulations, let us first see what the official data say about how bad the situation is in Iran.

The graphs below are based on data accessed on April 16 from the website of the European CDC, which updates the file daily.  I have normalized the number of daily new cases and daily new deaths by millions of people, which makes for a more sensible comparison across countries than comparing raw numbers from vastly different population sizes.

I compare Iran’s numbers with Spain, a country with a population 3/4 that of Iran. According to the first two graphs, Spain seems much harder hit than Iran, especially in the number of daily deaths.  The bottom graph shows the death rate from infections (the number of daily deaths divided by the total cases on that day). This is similar to the so-called “attack rate” which divides cases by the number at risk.

All three indices show improvement in both Iran and Spain, though Iran’s rates are lower.  In both countries, governments enforced social distancing and restrictions on travel.  If the undercounting in Iran and Spain is comparable, the lower number of daily deaths and cases suggest that as a society Iran has dealt with the crisis better than Spain (and most other countries known to be bad cases such as Italy and the US).  In addition, the lower death rates in Iran (lower graph) suggest that its health infrastructure has also performed relatively well in combating the epidemic.  The death rate in Iran was 1.2 per thousand on April 16, compared to 2.9 in Spain, 3.5 in Italy, and 7.7 in the US. This is consistent with the fact that, despite having half the GDP per capita of Spain, and one-third that of the US, Iran’s life expectancy is not too far lower from these advanced countries (76 years compared to 83 in Spain and 78 in the US).  Iran’s health sector is evidently better run that the rest of the economy.

Source: ECDC.

Source: ECDC.

These graphs undermine the general impression that Iran’s health crisis case is one of the bleakest.  But the crisis is far from over and it is a mistake to interpret these downward sloping lines as a sign that Iran (or Spain for that matter) is out of the woods.  Iranians are returning to work this week, so be prepared for Iran’s infection rates to turn back up in the coming weeks.  Under sanctions, Iran’s leaders are making a difficult calculation in balancing jobs and lives.  Sanctions have denied them the resources for an easier tradeoff.  Calls for regime change tied to sanctions and economic collapse also compel Iranian decision makers to place a greater weight on the economy than they otherwise might have– no one is pinning the survival of the “regime in Madrid” to Spain’s economic performance.  Most developing countries only have enough resources to sustain their people without work for a few weeks. Sooner or later, Iranians, especially the poor, have to balance the risk dying from hunger with the risk of succumbing to the virus.

How bad are the official counts? To doubt the veracity of data issued by any government is generally healthy, and in the case of Iran it is well advised.  But even unaccountable governments do not like to be caught in a lie.  Iran has a relatively vocal civil society in matters related to the economy and public health, which is why it is not easy to get away with lies for too long, especially about things like the rate of inflation or mortality.

For years, reporters from respected news organizations, such as the New York Times, who reported on Iran’s inflation felt compelled to undermine official rates by adding a sentence like, “according to experts, the actual inflation rate is twice as high.” But if officials who report inflation rates are aware of the compound formula, they would be very careful with lying about the increase in the price level on a routine basis.  After a while the errors compound.  For example, if each year officials under-reported the actual inflation rate of 40% by half, then after 40 years the observed prices — for labor, taxi, chicken, and the like — would be 476 times those indicated by the officials data (1.4^40 is 476 times larger than 1.2^40).

What does the government’s honesty about inflation have to do with veracity of COVID-19 data?  Simple: Producers and presenters of official health data in Iran are aware of the multiplicity of sources for mortality statistics and are therefore likely to avoid big lies.  Health ministry officials are sophisticated enough to know that a sizable underreporting of deaths from this epidemic will be revealed sooner or later, if not in the death registration statistics collected by the agency that issues birth and death certificates, then by the age structure that the 2021 census would reveal when it is published.

Big lies are hard to hide from the census, especially since Iran routinely publishes the 2-percent sample of each census in unit record (which record the age, education, and the location of residence of each individual), some of which are available on the IPUMS platform.  On average about 1200 Iranians, mostly elderly, die from various causes each day.  This is what gives the age structure its pyramid shape.  Until a week ago, official fatalities from the virus were about 200 per day, and this week they are 100 per day, increasing the death rate by 8-16 percent.  It would be hard to mistake the age structure resulting from these proportions on the 2021 census for those resulting from death counts many times as high.

Fear of later exposure rarely discourages politicians from making false claims, but the incentives of professionals and organizations tasked with collecting and analyzing data are generally more closely lined up with the truth — think Trump vs. Dr. Fauci — and Iran should be no exception.

Correction added on April 18, 2020: In the original post I had incorrectly reported the Virginia Tech-MIT study’s estimates of the number of deaths from the epidemic. They are 11 times higher than the official numbers, not 40 times as I had written in the original post. My apologies to the authors of the study and to readers.