IJE Advance Access originally published online on October 2, 2007
International Journal of Epidemiology 2008 37(1):194-200; doi:10.1093/ije/dym202
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Accuracy of child morbidity data in demographic and health surveys
1Department of Health Sciences, University of York, UK.
2Centre for Health Economics, University of York, UK.
* Corresponding author. Department of Health Sciences, Seebohm Rowntree Building, A/TB/165, University of York, Heslington, York, YO10 5DD, UK. E-mail: kp6{at}york.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
Background: The Demographic and Health Surveys (DHSs) have been used throughout the developing world for the last 20 years to provide data on the distribution of disease in order to inform planning. Data on child illness and death are reported by mothers and are susceptible to error.
Methods: We conducted an in-depth study of the Iranian DHS carried out in 2000–2001 and reviewed 110 DHS carried out around the world to check for bias by assessing the social gradient in reported child morbidity and mortality.
Results: We found that the reported under-5 child morbidity and mortality rates for the 28 Iranian provinces were inversely correlated (r = –0.592, P < 0.001) and that the adjusted social gradient of child morbidity implied increased illness in those who had literate vs illiterate mothers (OR = 1.26, 95% CI 1.20–1.32) compared with a decrease in mortality with increased literacy (OR = 0.52, 95% CI 0.46–0.59). Many of the other DHSs also show increased rates of reported child diarrhoea in households with higher levels of maternal education, access to piped water and urban (vs rural) dwellings, the reverse of what is found with mortality rates.
Conclusions: This suggests that there may be significant recall and reporting bias in under-5 childhood morbidity in DHSs. Caution should be used in the interpretation and use of data from DHSs and the survey methods should be reviewed.
Keywords Demographic and Health Surveys, child mortality, morbidity, bias (epidemiology), error sources
Accepted 30 August 2007
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
Childhood diarrhoeal diseases and acute lower respiratory infections are two of the leading causes of the world's burden of disease.1–2 Monitoring the distribution of these diseases in developing countries may usefully inform health policy. Demographic and Health Surveys (DHSs) are possibly the leading sources of maternal and child health data in the developing world with over 200 DHSs having been conducted in more than 75 developing countries over the last 20 years, often with financial support from the United States Agency for International Development (USAID).3–4 DHS provide data on key health indicators, including diarrhoea and acute respiratory infection, as well as geographic, demographic and socio-economic characteristics of the population.
Results of DHSs are intended to be used as baseline data for decision making in developing countries and to inform health planning and implementation as well as the monitoring and evaluation of health programmes. They are also used by international organizations in their reports on global trends and between- and within-country comparisons.5
Given the significance of DHS, it is important that the data are accurate. Since the data on child health are based on reports from mothers, there is a risk of inaccuracies due to errors in reporting and recall.6 Recall error has been well described in studies of self and proxy health reporting7–8 and some concern has been expressed in the past about potential biases in mothers’ recall of child diarrhoeal disease in DHS.9 However, there has been no systematic exploration of this potential problem.
This article reports a study exploring the evidence for bias in reported child morbidity in DHSs and considers the implications for their use in health planning.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
We analysed the results of the Iranian DHS survey which was carried out in 2000–2001 to examine the association between reported under-5 child morbidity and socio-economic factors known to be positively associated with child morbidity and mortality. We examined the association between mortality and morbidity to assess the degree to which social gradients were consistent. We also reviewed the reports of 110 DHS from other developing countries carried out between 1986 and 2002 to compare social gradients in under-5 child morbidity and mortality.
In 2000–2001, the Iranian Ministry of Health, in collaboration with the United Nations Children's Fund (UNICEF), the United Nations Population Fund (UNFPA) and the Statistical Centre of Iran (SCI), conducted its first DHS to provide an up-to-date source of data of demographic, socio-economic and health indicators of the Iranian population.10 This was a two-stage cluster sample of 10 households in 200 rural and 200 urban areas in each of the 28 provinces (with 2000 more households in the capital city, Tehran). In the 114 000 households in the original sample, 42 765 mothers with children under 5 years of age were interviewed and provided a reproductive history and a yes/no answer to the question: ‘has your under-5 child had any illness in the past two weeks?’
Analysis was carried out both at the provincial and individual level, with analyses weighted by the population of each province. Morbidity rates were for any reported illness in the previous 2 weeks/100 under-5 children. Under-5 mortality rates were calculated as the probability of a child dying between birth and the exact age of 5 years. In the provincial-level analysis, Pearson correlation coefficients (r) were calculated to estimate the association between the under-5 child morbidity rates of the 28 provinces and variables known to be related to child health—mothers’ literacy and the number of under-5 children in the household. The correlation between under-5 child morbidity and mortality rates was also calculated.
The individual-level analysis explored the morbidity and mortality rates stratified by a number of key social and demographic factors known to influence child health: mothers’ literacy, educational level, type of area of residence, indices of wealth or ownership of assets, household size, number of under-5 children and mothers’ age.11 We used logistic regression to explore further the relationship between mothers’ literacy and reported morbidity and mortality, adjusting for other social and demographic factors. A multi-level regression analysis was also carried out to assess the potential impact of clustering (at both household and provincial level) on the results.12
In addition, we examined the results of the other DHSs for childhood morbidity and mortality since surveys began in 1985. Diarrhoea morbidity data (stratified by socio-economic variables) from all 110 DHS undertaken between 1986 and 2002 were extracted from three comparative reports published by the Measure DHS project.13–15 Mortality data for the same surveys (where they were reported) were extracted from reports on child mortality.16–18 Where the data were available, we then compared the rates of under-5 child diarrhoea and under-5 child mortality between: mothers with different levels of education, rural vs urban residence, and households with and without piped water. We only examined rates of diarrhoea as a measure of childhood morbidity in these international analyses, because of research showing that data on diarrhoea are more consistent than other morbidities.9,19
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
Provincial-level analysis
There is a strong positive association between mothers’ literacy and reported child morbidity (r = 0.693, P < 0.001). Figure 1 shows that in nearly all of the provinces, the rate of reported child morbidity was higher amongst literate mothers than illiterate mothers. On the other hand, there is a strong negative correlation between rates of mothers’ literacy and mortality (r = –0.765, P < 0.001). Similar associations, counter to the expected direction, were found with other socio-economic variables at provincial level.
|
This inconsistency is also shown in the statistically significant negative association between under-5 child morbidity and mortality (r = –0.592, P < 0.001) (Figure 2).
|
Individual-level analysis
Stratified rates of childhood morbidity and mortality show that the social gradient for morbidity was generally in the reverse direction of that found for mortality and indicated that reported morbidity was higher in those who are better off (Table 1). For example, the rate of any reported morbidity in the previous 2 weeks for children of illiterate mothers is lower (35.2%) than the rate for children of literate mothers (42.6%) whilst the reverse was true for mortality, where the risk of an under-5 child having died in the last year being reported by illiterate mothers was almost twice that of literate mothers (Table 1).
|
This is also shown in the unadjusted and multi-variable regression of morbidity and mortality on mother's literacy, respectively (Table 2). The odds of reporting an under-5 child death by literate mothers was around half that by illiterate mothers; whereas the odds of reporting any child illness in the previous 2 weeks was 26% higher in literate compared with illiterate mothers, even after adjusting for a range of other social and demographic data. These results were not materially altered when estimated with multi-level regression models, accounting for the clustering of children within families, and children within provinces.
|
Other DHS reports
Unexpected patterns of childhood diarrhoea prevalence in relation to mothers’ education were identified in 65 of the 110 of DHSs around the world. In 44 of these 65 surveys, under-5 mortality is reported. In all 44, the rates of reported diarrhoea in the previous 2 weeks/100 and under-5 mortality/1000 stratified by mothers’ education move in opposite directions—higher rates of reported diarrhoea but lower mortality rates among the children of better educated mothers (Table 3). Similar unexpected patterns were found when stratifying by area of residence and piped water supply (Table 4).
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
Our analysis of the Iranian DHS shows that reported morbidity was higher in children of literate mothers, mothers with a higher level of education and other indicators of being socially and financially relatively better off. On the other hand, child mortality rates are positively associated with social deprivation. These unexpected results held in both provincial- and individual-level analyses. We also found similar counter-intuitive patterns in the results of many other DHSs carried out in developing countries.
These morbidity trends are inconsistent with the wealth of research evidence that maternal education is associated with improved child health;11,20 that in developing countries, urban residents have better health status than rural residents;21 and that unsafe water contributes to about 88% of child deaths from diarrhoea in developing countries.22
In the Iranian DHS, a general, filter question was asked about children's illnesses in the 2 weeks prior to the interview (which is not standard practice in the DHS). General questions are more vulnerable to inaccuracy than detailed questions about specific illnesses.23 However, the counter-intuitive patterns in the data of other DHSs where the general, filter question was not used, indicate that biased child morbidity data are not specific to the Iranian DHS, but are a problem in many of the DHS carried out in all parts of the developing world. Since these countries are at different stages of development, have different cultures, social structures and health systems and the surveys were carried out at different times from 1986 to 2002, no obvious socio-economic, regional, religious or cultural factors can explain the counter-intuitive results appearing in some countries rather than in others and it appears to be an intrinsic danger of the survey method.
Perception, recall and reporting of illness can be influenced by social and cultural factors which are likely to influence the reporting of morbidity more than mortality.23 The inverse relationship between reported child morbidity and social deprivation is likely to be the result of recall and reporting biases, where less educated, poorer mothers tend to report child illness less commonly than richer mothers.5,24–25
Although a previous review raised the issue of recall error in the reporting of child morbidity in the DHS,9 our study appears to be the first systematic attempt to explore the social gradients in reported morbidity and mortality in the DHS. This is perhaps surprising given the resources spent worldwide on the DHS and their use by policy makers.
|
Implications |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
The results of this study should encourage great caution when interpreting the results of reported child morbidity from the DHS. Even if the social gradients of morbidity and mortality are not in opposite directions, it does not mean there is no bias. The social gradient of reported child morbidity may be less steep than true morbidity due to social influences on reporting differences.
Improvements in the accuracy of such proxy surveys may require better training of interviewers,5,23 changes in the methods of eliciting proxy reports, shortening of the survey3,26 and detailed validation exercises in sub-samples of households to check for bias.27 Qualitative studies should be undertaken alongside such surveys in order to get a deeper understanding of the likelihood of reporting bias and the underlying causes.
Countries which have already conducted DHSs and may be using them to inform policy should undertake a more detailed analysis of their data and make appropriate adjustments. When there is doubt about the accuracy of morbidity data, mortality data may be a more reliable proxy for the pattern of ill health. This was the approach adopted by the National Health Service in England when first developing a rational formula for the allocation of resources.28
We conclude that child morbidity data collected by DHSs are susceptible to significant bias, probably due to socially patterned differential recall and reporting. Such data may produce misleading information on the distribution of health and disease in developing countries. Better methods need to be applied in these surveys to improve the quality of child morbidity data.
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
We would like to thank Dr Bahram Delavar, former Director General for Population and Family Health and Dr Mohsen Naghavi, former Director for Applied Research Centre, both in the Iranian Ministry of Health and Medical Education for access to the Iranian DHS data. Thanks are also due to Prof. Martin Bland in the Department of Health Sciences, University of York for advice on the life table estimation of under-5 mortality rates. AOM was supported by a scholarship from the Iranian Ministry of Health and Medical Education.
Conflict of interest: None declared.
Sources of funding: None declared.
KEY MESSAGES
|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionImplicationsAcknowledgementsReferences |
|---|
1 Murray JLM, Lopez AD. Evidence based health policy: lessons from the global burden of diseases study. Science (1996) 274:740–43.
2 Rudan I, Lawn J, Cousens S, et al. Gaps in policy-relevant information on burden of disease in children: a systematic review. Lancet (2005) 365:2031–40.[CrossRef][Web of Science][Medline]
3 Vaessen M, Thiam M, Le T. The Demographic and Health Surveys. Household Sample Surveys in Developing and Transition Countries. (2005) New York: United Nations, Department of Economic and Social Affairs, Statistics Division. 495–518. ESA/STAT/AC.85/23 (ed).
4 Macro International. Demographic and Health Surveys, Overview. (Accessed January 1, 2006). Available at: http://www.measuredhs.com/aboutsurveys/dhs/start.cfm.
5 Forsberg BC, van Ginneken JK, Nagelkerke NJD. Cross-sectional household surveys of diarrhoeal diseases - a comparison of data from the control of diarrhoeal diseases and demographic and health surveys programmes. Int J Epidemiol (1993) 22:1137–45.
6 Boerma JT, Sommerfelt AE. Demographic and Health Surveys (DHS): contributions and limitations. World Health Stat Q (1993) 46:222–26.[Medline]
7 Beckett M, Vanzo J, Sastry N, Panis C, Peterson C. The quality of retrospective data. J Hum Resources (2001) 36:593–625.[CrossRef][Web of Science]
8 Coughlin SS. Recall bias in epidemiologic studies. J Clin Epidemiol (1990) 43:87–91.[CrossRef][Web of Science][Medline]
9 Boerma JT, Black RE, Sommerfelt AE, Rutstein SO, Bicego GT. Accuracy and completeness of mothers’ recall of diarrhoea occurrence in pre-school children in demographic and health surveys. Int J Epidemiol (1991) 20:1073–80.
10 Iranian Ministry of Health. DHS Survey: Population and Health in the Islamic Republic of Iran. (2001) Tehran: Unicef, Iranian Ministry of Health and Medical Education. 1–167.
11 Cleland JG, van Ginneken JK. Maternal education and child survival in developing countries: the search for pathways of influence. Soc Sci Med (1988) 27:1357–68.[CrossRef][Web of Science][Medline]
12 Mplus Muthen B. (Accessed January 1, 2006). Available at: http://www.statmodel.com/index.shtml.
13 Boerma JT, Sommerfelt AE, Rutstein SO. DHS Comparative Studies, Child Morbidity and Treatment Patterns. (1991) 4. Maryland: Macro International. 1–39.
14 Rayland S, Raggers H. DHS Comparative Studies, Child Morbidity and Treatment Patterns. (1998) Maryland: Macro International Inc. 27.
15 Stallings RY. DHS Comparative Studies, Child Morbidity and Treatment Patterns. (2004) Maryland: Macro International Inc. 8.
16 Sullivan J, Rutstein S, Bicego G. DHS Comparative Studies, Infant and Child Mortality. (1994) 15. Maryland: Macro International. 1–57.
17 Bicego GT, Ahmad OB. DHS Comparative Studies 20, Infant and Child Mortality. (1996) Calverton, Maryland: Macro International Inc. 20.
18 Mahy M. DHS Comparative Report 4, Child Mortality in the Developing World. (2003) 4. Maryland: ORC Macro. 1–60.
19 Rousham EK, Northrop-Clewes CA, Lunn PG. Maternal reports of child illness and the biochemical status of the child: the use of morbidity interviews in rural Bangladesh. Br J Nutr (1998) 80:451–56.[Web of Science][Medline]
20 Basu AM, Stephenson R. Low levels of maternal education and the proximate determinants of childhood mortality: a little learning is not a dangerous thing. Soc Sci Med (2005) 60:2011–23.[CrossRef][Web of Science][Medline]
21 Wang L. Determinants of child mortality in LDCs: empirical findings from demographic and health surveys. Health Pol (2003) 65:277–99.[CrossRef]
22 Black RE, Morris SS, Bryce J. Where and why are 10 million children dying every year? Lancet (2003) 361:2226–34.[CrossRef][Web of Science][Medline]
23 Kroeger A. Health interview surveys in developing countries: a review of the methods and results. Int J Epidemiol (1983) 12:465–81.
24 Cogswell ME, Oni GA, Stallings RY, Brown KH. Sociodemographic and clinical factors affecting recognition of childhood diarrhea by mothers in Kwara State, Nigeria. Soc Sci Med (1991) 33:1209–16.[CrossRef][Web of Science][Medline]
25 Bruijnzeels MA, Foets M, Vander Wouden JC, Prins A, Vanden Heuvel WJA. Measuring morbidity of children in the community: a comparison of interview and diary data. Int J Epidemiol (1998) 27:96–100.
26 Cleland J. Demographic data collection in less developed countries 1946–1996. Popul Stud (1996) 50:433–50.[CrossRef]
27 Ross D, Vaughan P. Health interview survey in developing countries, a methodological review. Stud Fam Plann (1986) 17:78–94.[CrossRef][Web of Science][Medline]
28 Department of Health and Social Security. Sharing Resources for Health in England: Report of the Resource Allocation Working Party. (1976) London: HMSO.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||