IJE Advance Access originally published online on September 26, 2007
International Journal of Epidemiology 2007 36(6):1285-1291; doi:10.1093/ije/dym176
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comparing health inequalities across time and place—rate ratios and rate differences lead to different conclusions: analysis of cross-sectional data from 22 countries 1991–2001
Department of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, UK.
*Corresponding author. Honorary Lecturer, Non-communicable Disease Epidemiology Unit, Department of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, Keppel Street, London WCIE 7HT, UK. E-mail: kathmoser83{at}hotmail.com
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
Background Socio-economic inequalities in health within countries are a key public health issue. It is important that we can effectively make international comparisons of the level of inequalities and assess trends over time. We investigate how the results of such comparisons can differ depending on whether inequality is quantified using the rate ratio or rate difference.
Methods We examine levels and trends in inequality in under-five mortality using data from 22 low/lower-middle income countries [Africa (11), Latin America/Caribbean (5), Asia (6)], each with two Demographic and Health Surveys between 1991 and 2001. Within-country inequalities are quantified using the rate ratio and rate difference.
Results Ranking countries by their level of inequality at one point in time differed, sometimes substantially, according to whether the rate ratio or difference was used (Spearman's rank correlation = 0.49). Similarly, ranking countries according to the magnitude and direction of change in inequality over time depended on the measure used. Importantly from a policy perspective, in five countries the direction of change was in the opposite direction (increase vs decline in inequality) when using the ratio compared with the difference measure.
Conclusions The results of comparisons of the magnitude of health inequalities between countries and over time depend upon whether the rate ratio or rate difference is used. When statements are made comparing the size of inequalities it should be made completely clear whether these are measured on an absolute or relative scale. If the substantive conclusions differ according to the measure used this should be clearly stated. In this situation emphasis should only be given to results based on one summary measure if this can be clearly and explicitly justified in the context.
Keywords Health inequalities, international comparisons, ranking, measurement, under-five mortality
Accepted 7 August 2007
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
Socio-economic inequalities in health within- and between-countries remain key public health issues. In 2005, the World Health Organisation established the Commission on Social Determinants of Health ‘to draw the attention of governments, civil society, international organizations and donors to pragmatic ways of creating better social conditions for health’.1 With this goes the need to be able to make international comparisons of the scale of health inequalities, and to assess how inequalities change over time, in order to monitor trends, inform policy and evaluate the effect of interventions. For national governments, and indeed the international community as a whole, there is the inevitable and understandable desire to rank countries according to the extent of inequality in health at any moment in time and, secondly, according to how successful they have been in reducing such inequalities over time. While there is no doubt that league tables are open to misuse and misinterpretation,2,3 they nevertheless provide informative summaries that may be used to identify priorities both internationally as well as at the level of individual countries.
Assessing progress in reducing health inequalities and comparing countries according to the magnitude of their inequalities raises the fundamental question of how best to quantify these inequalities. Much attention has been given to the difficulties in making meaningful comparisons when the socio-economic variables used (whether, for example, education, income or occupational social class) vary across countries.4 In the 1990s, a major systematic attempt was made by Mackenbach and colleagues to make meaningful comparisons of the size of health inequalities across European countries.5 This programme identified variations in study design and type of data collected as major challenges.6 It also made it very apparent that the way in which inequalities are measured has an important influence on how countries are ranked in terms of the magnitude of their health inequalities. For example, one can quantify the extent of inequality between death rates in one socio-economic group compared with another either by taking the ratio of the rates or their arithmetic difference.7 Using the rate ratio, a relative measure, Mackenbach et al.5 found that Sweden had larger health inequalities than most other European countries. However, as pointed out by Swedish researchers,8 if inequality is measured using the absolute difference in mortality rates, Sweden has one of the lowest levels of health inequality.
Despite the apparent simplicity and importance of the question of how inequalities should be measured there have been few attempts to focus directly on this issue and how it might influence conclusions in comparative analyses over place and time and, specifically, the ranking of countries at any moment in time or with respect to changes in inequalities over time. Whilst there are many ways in which the magnitude of health inequalities can be assessed,7 the most widely used measures are rate ratios and rate differences that compare mortality rates in those of highest socio-economic position with those of lowest socio-economic position, typically using the top 20% and bottom 20%. While these measures do not use all of the information available, due to the ease with which they can be calculated they are nevertheless extensively used and are hence the measures we consider here. It should be emphasized that in this article we are concerned with establishing whether in principle making comparisons of the size of inequalities depends on whether the rate ratio or difference is used. Other relative or absolute measures, such as the Relative or Slope Indices of Inequality, are not the focus of our investigations.
The ideal setting for an examination of the influence of different measures on comparisons of socio-economic inequalities in health would be one that used data from a substantial number of countries collected in an identical fashion in terms of study design, socio-economic measure and health outcome. Such an international data set does not exist for high-income countries. However, the Demographic and Health Surveys (DHS) conducted in many low- and middle-income countries come close to the ideal (http://www.measuredhs.com/). Using a standardized methodology and core questionnaire the DHS programme has carried out nationally representative household surveys in over 60 countries over the last 20 years.
We use DHS data from 22 countries to investigate how the conclusions of comparative analyses, specifically the ranking of countries according to the level of health inequality, and time trend in inequality, can be affected by whether the rate ratio or rate difference is used. Whilst under-five mortality is the health outcome used, our interest is in the methodological issue rather than the substantive inequalities in this particular health outcome in these particular countries.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
Using published data9 we examine levels and trends in inequality in under-five mortality within 22 low and lower-middle income countries each with two Demographic and Health Surveys between 1991 and 2001 (11 countries in Africa, five in Latin America and the Caribbean and six in Asia). In 2000 these countries accounted for 27% of the world's population. While clearly not representative of all low- and middle-income countries, they nevertheless encompass high and lower mortality regimes, and a range of socio-demographic conditions.
As described elsewhere in more detail,10 under-five mortality was estimated, using standard methods, from information on births in the 10 years preceding the survey derived from birth histories collected from women of reproductive ages. Socio-economic position is described using an index of household wealth calculated from information on ownership of household assets (e.g. radio, bicycle), housing characteristics (e.g. roofing, floor materials), source of drinking water, toilet facilities and availability of electricity. Households, ranked according to their wealth index, were split into five groups each containing 20% of individuals and representing the poorest up to the least poor population quintiles for the survey in question. This produces a measure of relative household wealth in the sense that, for example, the poorest quintiles in country A and country B will not necessarily have the same level of absolute wealth but each will represent the poorest fifth of their population. Under-five mortality (deaths under age 5 per 1000 live births) was calculated for each quintile.
As previously stated, we used two simple approaches to the measurement of the magnitude of inequalities: the rate ratio and rate difference. The former measures relative inequality, the latter absolute inequality. We compare these approaches in both a cross-sectional analysis, using data from the earlier of the two surveys alone, and in a longitudinal analysis investigating ranking of changes in inequality.
In the cross-sectional analysis we calculate rate ratios and rate differences that compare the mortality rate in the poorest and least poor quintiles using the ratio and difference of these rates, respectively. The 95% confidence intervals are calculated (see Appendix for formulae) using standard errors which took account of the DHS sample survey design. Countries are ranked according to their rate ratios and rate differences and the association between rankings assessed graphically and using Spearman's rank correlation coefficient. Rate ratios are plotted on log scales.
We also carry out an analogous analysis of time trends in inequality, specifically investigating whether the ranking of countries by change in inequality is the same according to the rate ratio and rate difference. Since the length of time between the two surveys differs across countries we standardize to a 5-year change, calculated from the difference between the years of the surveys. Where the survey date spans 2 years the later of the 2 years is used. Denoting the estimated mortality rates in groups defined by the lower and upper quintiles in the two surveys by l1, u1, l2 and u2 respectively, and the time (in years) between the two surveys by t, the 5-yearly absolute change in rate difference is 5[(u2 – l2) – (u1 – l1)]/t. On the relative scale the change in inequality over time is expressed as the 5-yearly percentage change in rate ratio. As is standard for relative measures, the 5-yearly relative change is calculated after logarithmic transformation {exp[5 log((u2/l2)/(u1/l1))/t]} and this is then converted to a percentage change (e.g. a relative change of 1.8 is an 80% increase). Data analysis was conducted using Stata Version 8.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
Under-five mortality in the 22 countries ranges between 28 and 252 deaths per 1000 live births (Table 1).
|
Using the data for the earlier of the two surveys for each country, Figure 1 displays rate ratios (panel 1A) and rate differences (panel 1B) between the poorest and least poor quintile groups, each in rank order. Inequality is diverse with rate ratios spanning 1.2 to 5 and rate differences 10 to 130 deaths per 1000 live births. The ranking of countries by their level of inequality in under-five mortality depends to a large extent on which measure of inequality magnitude is used. For example, Peru displays the highest inequality according to the rate ratio whereas it is ranked only 7th highest when the rate difference is used. Mali has the highest inequality according to the rate difference but is ranked only 14th highest when the rate ratio is used. The rankings of countries according to rate ratios and differences are not strongly correlated (Spearman's rank correlation coefficient = 0.488).
|
An analogous pair of graphs is shown in Figure 2, this time ranking countries according to the magnitude of the 5-yearly change in inequality. On the ratio scale (panel 2A) inequalities in some countries have reduced while in others they have got larger. Similarly, diversity is shown when changes in inequality are considered on the difference scale (panel 2B). The rankings of countries using the two measures of change are again imperfectly correlated (Spearman's rank correlation coefficient = 0.756). While a correlation of this magnitude would be considered high were this relating to two distinct variables, the expectation is that in this context it would be close to unity if the ranking of change in inequality was not dependent upon the measure used. Furthermore, and perhaps more importantly from a policy perspective, in five countries (Peru, India, Namibia, Vietnam and Nepal) the direction of change suggests increasing inequality on the ratio scale, but decreasing inequality on the difference scale.
|
The cross-sectional and time trend aspects of the data are combined in Figure 3. For each country, inequalities in the earlier and later periods are indicated by solid and open circles, respectively. The pair of points for each country is joined by a line. For countries where the direction of change in inequality over time is the same for both ratio and difference measures the connecting line is dotted, whereas where the direction of change differs on a ratio and difference scale a solid line is used. The figure shows that between the two survey periods there is substantial variation between countries in the size as well as direction of change in inequality. Most importantly, it makes clear that the relative size and direction of trends can depend upon the way in which inequalities are measured.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
To our knowledge this is the first systematic comparison of different measures of inequality magnitude across a large number of countries using a standardized measurement tool. It demonstrates that the conclusions of comparative analyses across time and place depend in part on which measure of inequality magnitude is used. Ranking countries according to their level of inequality produces a substantially different ordering depending on whether the rate ratio or difference is used to quantify inequality. Whether inequality is seen to increase or decrease over time, and the size of that change, also depends on the measure used. These findings, which are not confounded by different study designs, data collection methods, health outcome or socio-economic indicators, have important implications for policy, practice and research.
The existence of the Demographic and Health Surveys creates a unique opportunity to systematically evaluate how the choice of measure of inequality magnitude affects conclusions in comparative analyses over place and time. This analysis makes use of DHS data from 22 countries each with two surveys separated by a few years all collected using very similar survey designs. While the comparability of the index of household wealth may be questioned, it is at least classifying households using the same substantive principle and completely avoids the much more serious problems of comparing inequalities using conceptually completely different dimensions of socio-economic position (for example, education compared with income). This rich and comparable collection of data sets has no parallel in health inequality research in high-income countries. Consequently, this methodological work is feasible in a way it is not with the less uniform data sources available across high-income countries. Moreover, the wide ranging childhood mortality experienced by the countries included in our analysis has enabled us to develop the analysis further than would have been possible with the more homogenous mortality seen across high-income countries.
Our finding that there is a lack of agreement between rankings based on rate ratios and differences cannot be explained by sampling error. It is recognized in other settings that sampling error in performance indicators can lead to instability in rankings using any single measure.2 However here the issue is a more fundamental one, because the discrepancies we have identified are a consequence of the arithmetic properties of the measures rather than a consequence of random error. Even if all of the surveys were of sufficient size to effectively eliminate random error then discrepancies in the rankings of the order that we observe here would still occur.
It is likely that if the magnitude of inequality had been measured in yet other ways the rankings would have been different again. Indeed, rankings produced using the concentration index,11 while similar to the rankings using the rate ratio, are not by any means identical. The strength of our analysis, however, is that the rate ratio and rate difference make use of identical data but different arithmetical calculations. One would expect measures that use different data, for example from the whole distribution (as for the concentration index) rather than from the extreme quintiles (as for the rate ratio and difference), to produce different rankings.
Our focus on comparing inequalities between countries complements other work that has been done looking at inequalities within countries, over time and between ethnic or socio-economic groups. In a recent report on methods for measuring disparities in cancer in the USA12 it is elegantly shown that if differences in stomach cancer mortality between men and women are measured on a relative scale, then between 1930 and 2000 there has been a steady increase in gender disparity. However, if instead the differences are measured on an absolute scale then since 1930 there has been a steep decline in disparity between men and women. This finding is analogous to our results in that both underline the fact that choice of measure can give very different impressions about the nature of inequality or disparity. For the purposes of their particular report the authors suggest that quantifying disparity in absolute terms may be particularly appropriate as this provides an indication of the population health impact of the inequalities in cancer-related outcomes. They do, however, recognize that measurement in relative terms can also be appropriate depending upon the context. While their report stops short of recommending the universal use of either absolute or relative differences for summarizing the extent of inequality or disparity, it does underline the importance of presenting the underlying rates and not just one or more summary measure. Similar points are also made in another recent attempt to provide guidance on the methods that should be used to measure inequalities in health.13 These guidelines also refrain from suggesting that either absolute or relative measures have an intrinsic advantage that is appropriate in all contexts.
|
Conclusion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
There has been little awareness that the choice of measure of health inequality may affect assessments of the relative performance and achievements of different countries. Our findings clearly demonstrate that different measures can lead to very different conclusions. We believe these findings are not restricted to the setting used here but apply equally to other measures of health, data sources, and geographical levels and locations, and are as relevant to high- as to low-income countries.
While there cannot be any rigid recommendations that give precedence to either absolute or relative measures of inequality when comparing socio-economic inequalities in health, analyses of inequality should present and discuss the underlying rates and not just summarize them in single summary measures. Moreover, when statements are made comparing the size of inequalities it should be made completely clear whether these are measured on an absolute or relative scale. If the substantive conclusions differ according to the measure used this should be clearly stated. In this situation emphasis should only be given to results based on one summary measure if this can be clearly and explicitly justified in the context.
Conflict of interest: None declared.
|
Appendix: Statistical formulae |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
Denote the estimated mortality rates in groups defined by the lower and upper quintiles in the two surveys by l1, u1, l2 and u2 respectively, let t be the time interval (years) between the two surveys and let V() denote variance.
The variance of the rate difference and logarithm of the rate ratio at the first survey are given respectively by:
|
|
|
|
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
We are grateful to Eldaw Abdalla Suliman and Agbessi Amouzou for supplying us with the standard error data used in calculating the confidence intervals.
KEY MESSAGES
|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionConclusionAppendix: Statistical formulaeAcknowledgementsReferences |
|---|
1 World Health Organisation. Commission on social determinants of health. Available at http://www.who.int/social_determinants/resources/csdh_brochure.pdf (Accessed July 1, 2007).
2 Marshall EC, Spiegelhalter DJ. Reliability of league tables of in vitro fertilisation clinics: retrospective analysis of live birth rates. Br Med J (1998) 316:1701–04.
3 Sanderson C, McKee M. Commentary: how robust are rankings? The implications of confidence intervals. Br Med J (1998) 316:1705.
4 Murray CJL, Gakidou EE, Frenk J. Health inequalities and social group differences: what should we measure? Bull World Health Organ (1999) 77:537–43.[Web of Science][Medline]
5 Mackenbach JP, Kunst AE, Cavelaars EJM, Groenhof F, Geurts JJM, EU Working Group on Socioeconomic Inequalities in Health. Socioeconomic inequalities in morbidity and mortality in western Europe. Lancet (1997) 349:1655–59.[CrossRef][Web of Science][Medline]
6 Kunst AE, Groenhof F, Borgan JK, et al. Socio-economic inequalities in mortality. Methodological problems illustrated with three examples from Europe. Rev Epidemiol Sante Publique (1998) 46:467–79.[Web of Science][Medline]
7 Anand S, Diderichsen F, Evans T, Shkolnikov V, Wirth M. Measuring disparities in health: methods and indicators. In: Challenging Inequities in Health: from Ethics to Action—Evans T, Whitehead M, Diderichsen F, Bhuiya A, Wirth M, eds. (2001) New York: Oxford University Press.
8 Vågerö D, Erikson R. Socioeconomic inequalities in morbidity and mortality in Western Europe (letter). Lancet (1997) 350:516.[Medline]
9 Gwatkin DR, Rutstein S, Johnson K, Suliman EA, Wagstaff A. Initial Country-level Information about Socio-economic Differences in Health, Nutrition, and Population (2003) 2nd. Washington, DC: The World Bank.
10 Gwatkin DR, Rutstein S, Johnson K, Pande RP, Wagstaff A. Socio-economic Differences in Health, Nutrition, and Population (2000) Washington, DC: The World Bank.
11 Wagstaff A. Inequality aversion, health inequalities and health achievement. J Health Econ (2002) 21:627–41.[CrossRef][Web of Science][Medline]
12 Harper S, Lynch J. Methods for Measuring Cancer Disparities: A Review Using Data Relevant to Healthy People 2010 Cancer-Related Objectives (2006) Washington DC: National Cancer Institute.
13 Keppel K, Pamuk E, Lynch J, et al. Methodological issues in measuring health disparities. National Center for Health Statistics. Vital Health Stat Series 2 (2005) 141.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  J Macinko and I T Elo Black-white differences in avoidable mortality in the USA, 1980-2005 J Epidemiol Community Health, September 1, 2009; 63(9): 715 - 721. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||