Measuring poverty and inequality: how to get it right

Two widely used survey methods for measuring household well-being in developing countries are recall and diary interviews on respondents’ consumption. This column reports evidence that collecting snapshots based on recall is less prone than diary data to producing erroneous indicators of people’s well-being. This is important for the accurate measurement of poverty and inequality on which effective assistance programs depend.

Trends in poverty and inequality are used to track progress in development within and across countries. These trends help us to understand which potential national growth strategies are more inclusive, how crises like Covid-19 have implications for well-being beyond health and mortality, and whether safety nets and social assistance programs are being targeted effectively.

In most developing countries, indicators of poverty and inequality are based on surveys of household consumption. These assessments are formed from snapshot measurements covering a short period of time, such as one or two weeks.

But the underlying metric in which policy-makers are most interested is a longer-term concept of well-being, anchored in households’ usual rate of consumption. The discrepancy between assessing a long-term concept using data for a shorter reference period can, however, lead to flawed measures of poverty and inequality.

Our research shows that assessments based on two of the most widely used methods for collecting snapshots – recall interviews and diary data – may deviate from the real underlying distribution of usual consumption. Nevertheless, we find that recall interviews are less prone to erroneous conclusions about the population’s overall well-being, which suggests that the higher costs of implementing diaries are not justified.

Beyond the implications for measuring trends in poverty and inequality, this finding suggests that results derived from recall data will identify a set of households for inclusion in a poverty alleviation program that is more likely to be truly needy compared to those identified using diaries.

Household well-being is a long-run concept

Economic theory anchors well-being in a household’s usual rate of consumption, which is the outcome of a long-run optimization that depends on preferences and expectations about future flows of income.

The only way to measure this usual rate (which we denote asY* is by following the same households for an extended period to compute their average consumption over time. But since long series of panel data or administrative archives on transactions are often unavailable, well-being is typically assessed from surveys using snapshot measurements of consumption with reference to a shorter time period.

Snapshot measurements of consumption

Evidence from a number of countries, including Niger and Tanzania, shows that households will report their consumption differently depending on how the data are collected. Diaries have become the standard for benchmarking comparisons of snapshot measurements from alternative collection modes.

But the assumption that diaries provide an error-free benchmark has little statistical justification. For example, consumption must be imputed from diaries after adjusting for stock inflows and outflows, which is prone to additional measurement problems. Respondent fatigue and low literacy may also affect the quality of the data.

The difficulty of respondents remembering the exact timing of events also raises concerns about the use of recall questions. The implications of changing the recall period and how many items to recall have been experimentally compared to measurements from diaries under close supervision. Guidelines on how best to measure consumption from snapshots have been derived from such comparisons.

Measuring well-being: what can go wrong?

Welfare measurement poses problems that are logically distinct, a fact that has been often overlooked. Indeed, the idea that even the best measurements collected over a narrow slice of time may provide a poor picture of Y* goes back to Friedman’s work in the 1950s.

Consider two households (the Smiths and the Johnsons) planning to spend the same dollar amount over four weeks and therefore having the same Y*. During this period, both households budget $200 for dining out. The Smiths treat themselves to a $50 pizza dinner every week, while the Johnsons prefer a $200 dinner once every four weeks.

Table 1 shows that these preferences can affect how surveys depict the households’ comparative consumption, when the survey measures a single snapshot of a week. Columns 1 and 2 show an example of possible spending patterns for a typical month, while column 3 demonstrates that the relative position of the two households depends on the snapshot week considered.

Differences in survey measurements for the Smiths and the Johnsons reflect preferences for consumption frequency, not a measurement problem. Thus, wrong conclusions about welfare may arise even when the survey snapshot is measured without errors.

The bottom of column 3 shows that we should expect no mistake, on average, in the welfare ranking of households like the Johnsons and the Smiths. But the variance of all possible differences in column 3 is positive and large, despite the same underlying Y*.

Thus, while snapshots can yield an accurate measurement of average well-being, they will provide flawed indicators of inequality and the population share below the poverty line.

How to get it right

Using diary and recall measurements collected from the same household, research on Iraq empirically tests which collection mode yields snapshots closest to Y*. Figure 1 shows histograms of diary and recall snapshots compared with the distribution of Y* (the continuous line).

The presumption that diaries outperform recall data to assess disparities in Y* finds no support, as deviations from Y* are substantial. Importantly, this research also shows that poverty statistics and several indicators of inequality, such as the Gini coefficient, better reflect the underlying distribution of Y* in Iraq when computed from recall data.

This finding challenges the belief that the higher costs of using a diary bring more accuracy for poverty or inequality measurement. Indeed, recall interviews may be less prone to delivering wrong conclusions about well-being than diaries. This provides the empirical rationale for using recall modules in fieldwork, a possibility already contemplated for several national expenditure surveys.