Tyranny of numbers

The Statistical Center of Iran (SCI) has been gradually moving to make the micro data it collects available to a wider community of researchers.  Several years ago, SCI made the Household Expenditure and Incomes Survey (HEIS) available through their publications office in the Fatemi Ave. Anyone could walk in, sign a form so SCI would know who is using the data, and pay a nominal fee to buy a CD with the raw data on it.  Hundreds of researchers were able to work with the micro data, numerous Masters and PhD dissertations have been written inside and outside Iran as a result, adding to our understanding of household welfare, poverty, and inequality in Iran.  But to obtain the data, someone had to go to that office to pick up the CD.  Now, finally, SCI has taken a bold step to make these data available online and for free.  So far, there are 29 surveys online, 1984-2012.  (See below for how to access the surveys.)

This is a welcome move for which SCI should be applauded.  I am sure that somewhere someone is unhappy with liberalizing access to information.  I hope that this new policy is here to stay.  I recall a day, in the summer of 1988, just after peace had broken out between Iran and Iraq, when I showed up at Fatemi Ave, at the publications office of SCI, to pick up a copy of the SCI’s latest Annual Yearbook.  I was told I could not buy it because I needed a letter from my institution certifying that I was a trustworthy academic.  We have come a long way since then.  But free access to data will doubtless make many people in positions of power uneasy and even unhappy.

There are good reasons why producers of data are apprehensive of making them widely available.   There is, of course, the old argument that information is power, and governments prefer that ordinary people not have access to the same information as they do, that armed with the same information they might challenge government claims about the state of the economy and how things are changing.  We have gone beyond this for several years now, and Iranian papers are filled with not always accurate analyses of poverty and inequality by independent researchers.

There is also a more mundane reason, one that continues to seed doubt in the heart of those who make raw data freely available: That in the hands of people who do not know how to use them data can be a source of confusion rather than enlightenment.  This is specially serious when data are released without much information, as is the case with the HEIS raw data files.  When you download them, you get an compressed file that contains numerous tables in a database format and a 68 page questionnaire (all in Persian, so so far the data is not easily accessible by non-Persian speaking researchers).  There are keys you can ask for but even those do not tell you that making tables from the micro data you need to have the appropriate weights, which are not in the compressed file, but, again, you can get from the SCI staff if you know how to ask for them.  Without the right weights, you can get very misleading results.  There are other pitfalls along the way that experienced users of HEIS know how to avoid.

Of course, like everything else, if the information marketplace is competitive and there are no barriers to entry, the right information will eventually find its way to discerning readers, but that is not the public policy scene of Iran today.  Newspaper reporters are interested in sensational stories to increase their sales, often using misleading headlines and dubious content.  Careful studies take time, not to mention training and expertise, to prepare, things that are sadly lacking in Iran’s diploma-driven system of scientific inquiry.  The bad can drive out the good.  Waiting to make sure you are reporting accurate information and writing with ifs and buts may mean that you never make it to the front pages of newspapers.  I have seen plenty of such reports in Iran’s best dailies based on studies using the HEIS data but using shaky methodology and offering misleading conclusions.

One guard against low quality studies that make use of survey data is to rely on reputations based on publications with serious peer review procedures, but then economists and social scientists who offer their findings to the world through such venues are quite rare in Iran.

What to do?  There are two steps that the existing micro data portals take that help increase responsible use of micro data.  The SCI would do well to follow similar procedures.  The Luxembourg Income Study (LIS), and IPUMS at the University of Minnesota, the two best known such portals, check researcher credentials before releasing the data. They also produce streamlined data with uniform variable definitions and codes that minimize researcher errors.  The Economic Research Forum in Cairo, with which I am affiliated, has adopted a similar procedure for its data portal, which went live a few months ago.  The ERF catalogue now boasts some 40 user-friendly surveys from a dozen Arab countries that support research on the region globally.  As a country in the ERF region (Arab countries, plus Iran and Turkey) Iran could benefit from the ERF expertise in harmonizing survey data and making them available in a responsible way.

The SCI’s quiet but admirable move to release the raw data needs to be supplemented by steps to make HEIS data more user-friendly and the studies based on them less prone to researcher errors.  One way to do this is to invite users of data to engage in a dialogue about how to do this.  This could be a through a conference organized in Tehran or via web-based discussions.  I invite readers of this blog to offer their suggestions.

For now, here is how to access the HEIS micro data (but please use the data responsibly so new sets keeps coming).  Got to http://www.amar.org.ir, and under the tab “آمارهای موضوعی” find the Household Expenditures and Income Survey.

Speaking of micro data on Iran, the one other micro data set on Iran that I am aware of and is freely available on the web is the School to Work Transition Survey of 2005, jointly collected by the ILO and SCI.  You can download it from the ILO website.  Daneil Egel and I have a paper on youth transition in Iran, which is based on this survey, and can be downloaded from the Middle East Development Journal website (or here).