IJE Advance Access originally published online on May 21, 2007
International Journal of Epidemiology 2007 36(4):858-864; doi:10.1093/ije/dym092
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
A field test of three LQAS designs to assess the prevalence of acute malnutrition
1Academy for Educational Development, FANTA Project, Washington DC, USA.
2The World Bank, Washington DC, USA.
3Independent Consultant, Formerly with Catholic Relief Services, Emergency Response Team, Nairobi, Kenya.
4Ohio State University, Center for Biostatistics, Columbus, Ohio, USA.
5Catholic Relief Services, Baltimore, Maryland, USA.
* Corresponding author. Academy for Educational Development, Food and Nutrition Technical Assistance (FANTA) Project, 1825 Connecticut Avenue NW, Washington DC, 20009. E-mail: mdeitchl{at}aed.org
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background The conventional method for assessing the prevalence of Global Acute Malnutrition (GAM) in emergency settings is the 30 x 30 cluster-survey. This study describes alternative approaches: three Lot Quality Assurance Sampling (LQAS) designs to assess GAM. The LQAS designs were field-tested and their results compared with those from a 30 x 30 cluster-survey.
Methods Computer simulations confirmed that small clusters instead of a simple random sample could be used for LQAS assessments of GAM. Three LQAS designs were developed (33 x 6, 67 x 3, Sequential design) to assess GAM thresholds of 10, 15 and 20%. The designs were field-tested simultaneously with a 30 x 30 cluster-survey in Siraro, Ethiopia during June 2003. Using a nested study design, anthropometric, morbidity and vaccination data were collected on all children 6–59 months in sampled households. Hypothesis tests about GAM thresholds were conducted for each LQAS design. Point estimates were obtained for the 30 x 30 cluster-survey and the 33 x 6 and 67 x 3 LQAS designs.
Results Hypothesis tests showed GAM as <10% for the 33 x 6 design and GAM as 
10% for the 67 x 3 and Sequential designs. Point estimates for the 33 x 6 and 67 x 3 designs were similar to those of the 30 x 30 cluster-survey for GAM (6.7%, CI = 3.2–10.2%; 8.2%, CI = 4.3–12.1%, 7.4%, CI = 4.8–9.9%) and all other indicators. The CIs for the LQAS designs were only slightly wider than the CIs for the 30 x 30 cluster-survey; yet the LQAS designs required substantially less time to administer.
Conclusions The LQAS designs provide statistically appropriate alternatives to the more time-consuming 30 x 30 cluster-survey. However, additional field-testing is needed using independent samples rather than a nested study design.
Keywords Acute malnutrition, assessment, emergency, Ethiopia, lot quality assurance sampling, LQAS, wasting
Accepted 3 April 2007
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In emergency settings, humanitarian workers must rapidly detect areas where a population's health and survival are at risk. Due to the close association between acute malnutrition prevalence and crude mortality rates,1 the prevalence of acute malnutrition among children 6–59 months can be used as an indicator of elevated risk. Governments, donors and relief agencies often use a predetermined threshold of acute malnutrition to judge whether a response is justified and to guide the level of aid provided. WHO's classification scale defines the severity of malnutrition in a community based on the prevalence of wasting [Weight-for-Height-Z-(WHZ)-score <–2 Standard Deviations (SDs)]: Acceptable: <5%; Poor: 5–9%; Serious: 10–14%; Critical:
15%. These thresholds can be used to guide the implementation of selective and general feeding programmes.2,3 Global Acute Malnutrition (GAM) is defined as WHZ-score <–2 SDs and/or bipedal oedema.4–6 In emergency settings, governments and international agencies often apply the WHO thresholds to GAM in lieu of wasting. In these situations, the primary concern is to detect areas with a GAM prevalence 10% or above in order to determine where food aid is needed. In emergency settings, the prevalence of GAM needs to be assessed quickly, accurately, and often times repeatedly, to determine when and where to start and stop humanitarian aid. The most common approach for assessing GAM is a two-stage 30 x 30 cluster-survey7—which can be time-consuming and expensive. Alternative, less time- and resource-intensive sampling approaches are needed to rapidly identify areas requiring humanitarian assistance, thereby saving lives and resources. One of the most frequently used quality control statistical methods in international health is Lot Quality Assurance Sampling (LQAS).8
This article examines three adaptations of LQAS to assess GAM and other child-level indicators in food-insecure settings. Earlier sample-size and precision limitations of LQAS for GAM assessment are addressed.9 A field-test of the LQAS designs was conducted in the Siraro woreda of Ethiopia in June 2003 during a period of heightened food insecurity, at a time when the government used GAM thresholds to help prioritize woredas for food aid. The LQAS designs were administered simultaneously alongside a conventional 30 x 30 cluster-survey, using a nested sampling approach. This article describes the LQAS designs and compares the results of those designs with the 30 x 30 cluster-survey—in terms of point estimates and 95% CIs for GAM and other child-level indicators, classification results for GAM thresholds of 10, 15 and 20%, and the amount of time required for data collection.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
LQAS principles
Cumulative binomial probabilities are used in LQAS analyses to detect if a critical threshold has been reached for an indicator. To design an LQA sampling plan, the threshold of interest for an indicator (e.g. GAM prevalence), and tolerable statistical error (alpha and beta) are defined a priori. LQAS is an hypothesis test to determine whether the outcome is
or < the defined threshold. The alpha error is the probability of incorrectly classifying an area as not being at risk when the true GAM prevalence is the threshold of interest. The beta error is the probability of incorrectly classifying an area as being at risk when the true GAM prevalence is < the threshold of interest.
LQAS uses two thresholds, an upper and a lower threshold, to define the alpha and beta errors. The upper threshold is a GAM prevalence at which an area is at risk (e.g. 15%). This is the threshold the data are tested against. The lower threshold is the GAM prevalence at which an area would not be considered a priority for an intervention (e.g. 
10%). The alpha error is calculated for the upper threshold and the beta error for the lower threshold.
The null hypothesis assumes the GAM prevalence is the upper threshold. To classify GAM prevalence as or < the upper threshold, the number of children with GAM is counted and then compared against a Decision Rule (DR) determined using binomial probabilities.8,10 We judge GAM prevalence as the upper threshold if the number of children with GAM in the sample is > the DR and judge GAM prevalence as < the upper threshold if the number of children with GAM is the DR.
To illustrate how LQAS is applied in international health settings, assume a design with a sample size of 198, a 15% upper threshold, 10% lower threshold, alpha error 0.10 and beta error 0.20. If more than 23 (DR) children are malnourished, the area is judged to be at risk (GAM 15%). If 23 or less children are malnourished, we conclude the area is not at risk (GAM <15%).
GAM sampling design and protocol
GAM prevalence is estimated using a specific sampling protocol. Traditional LQAS requires a simple random sample (SRS) of households and one randomly sampled interviewee per household.11 In contrast, the sampling protocol to obtain an unbiased estimate of GAM in emergency settings requires that every child 6–59 months in a sampled household be measured.5,6,12 A two-stage 30 x 30 cluster-survey is most commonly used.7 The standard method involves selecting 30 clusters by Probability Proportional to Size (PPS) and the first household within each cluster by a random method such as spin-the-bottle. Subsequent households to be sampled within a cluster are typically selected by proximity to the first random household selected. 5,6,12 Data collected by the 30 x 30 cluster-method are clusters of children within households as well as clusters of households within clusters. In short, this means the children selected to obtain an unbiased estimate of GAM are not a SRS—they are clusters. Therefore, one cannot assume that LQAS can be used for analysing GAM.
By using computer simulations, we investigated whether the DRs derived from binomial probabilities were accurate when GAM was assessed using data collected in small clusters. Different population sizes as well as different intra-cluster correlations (0, 0.25 and 0.5) were used in the simulations. These intra-cluster correlation levels represent the GAM correlations that could occur from sampling multiple children in a household as well as from sampling multiple households within a cluster. Results from the simulations revealed that for an intra-cluster correlation of 0.25, the alpha error ranged from 0.045–0.057 and the beta error ranged from 0.258–0.287, when data were collected in households having one to six children 13 (Table 1). In other words, small cluster-sizes of size two to six had errors similar to a SRS of the same size for GAM thresholds of 10, 15, and 20%.
|
Three LQAS designs were developed from those simulation results: (1) 33 x 6 (33 clusters, six children in each cluster); (2) 67 x 3; and (3) a 67 x 3 sequential design. The designs use the smallest sample-sizes possible to detect GAM upper thresholds of 10 and 15% (with lower thresholds of 5 and 10%, respectively), while maintaining alpha errors
0.10 and beta errors 0.20. The designs can also detect a GAM upper threshold of 20% (with lower threshold of 15%) with slightly larger errors (alpha 0.13, beta 0.24). 13 The latter threshold was relevant to Ethiopia, given the emergency classification scale used by the Government.14 In addition, the 33 x 6 and 67 x 3 designs provide point estimates and 95% CIs for GAM and other child-level indicators.
The sequential design uses a multi-stage sampling plan which allows GAM thresholds to be detected before the full sample has been collected. The design can be used when the GAM threshold is defined a priori and point estimates for GAM and other child-level indicators are not desired. The full sample plan is based on the 67 x 3 design (67 clusters, three children in each cluster), however, the design allows for data analysis once a minimum sample-size of 25 is attained. To allow for multiple hypothesis testing, the sequential design uses lower errors (alpha 0.01, beta 0.05) for DRs at the smaller sample-sizes.13 Once the clusters are selected by PPS, the clusters can be sampled in any order. Sampling stops when one can make a reliable classification about the GAM threshold of interest. If the number of malnourished children is neither sufficiently high nor sufficiently low, a classification cannot be made about the threshold and sampling continues. Up to four hypothesis tests can be conducted over the course of the sampling while still conforming to overall error limits of alpha 0.10 and beta 0.20.
Assessment site
Siraro woreda, located in the Oromiya Region of Ethiopia, comprises 62 geographically defined Peasants Associations (PAs) and has a population of about 200 000. The main source of livelihood is subsistence agriculture. During 2002–2003, the Famine Early Warning System Network (FEWS NET) estimated that approximately 4 million in the Oromiya region of Ethiopia were affected by drought-related effects of food insecurity.15 At the time of this study, 22 of the 62 PAs in Siraro had lost 60–70% of their crops, 23 had lost 40–60% and 17 had lost 25–40%.16 Agencies working in the area suspected elevated GAM prevalence and that a priority situation existed.
Sampling methodology
To assess the situation in Siraro, Catholic Relief Services (CRS), a relief and development agency implementing programs in Ethiopia, in collaboration with the FANTA Project managed by the Academy for Educational Development (AED), administered a conventional 30 x 30 cluster-survey in accordance with the Government of Ethiopia's emergency nutrition survey guidelines.14 The 33 x 6 and 67 x 3 designs were administered simultaneously using the same questionnaires and a nested design. The data collected for the 67 x 3 design were also used for the sequential design.
The clusters (PAs) in the 30 x 30 cluster-survey and the 67 x 3 design were selected independently from the same sampling frame of PAs using PPS sampling. The first 30 clusters of the 33 x 6 design were the 30 clusters selected for the 30 x 30 cluster-survey. The last three clusters were selected using PPS from among the clusters sampled for the 67 x 3 design. During data collection, two of the PAs selected for data collection were inaccessible. Replacement clusters were selected from PAs on the basis of close proximity and similar population size.
All children 6–59 months in a household were included in the study and measured using standard anthropometric methods.14,17 To randomly identify the first household to be sampled in each cluster, the spin-the-bottle method was used.12,14 Subsequent households in a cluster were selected by geographic proximity. This sequence of household selection continued until the minimum prescribed sample-size for the cluster of each design was attained.
If a PA was selected as a primary sampling unit by more than one survey design, the data were shared across survey designs, until the sample-size required for the cluster of a given design was attained.
Data management
To collect data for this study, 15 interviewers (five interview teams) were hired. The teams received 35 h of training. During data collection (June 4–11 2003), three technical advisors provided supervision and technical support.
For the time study, the teams used an odometer to record the distance travelled from the base camp to the first cluster for each work-day and used a stopwatch to record the time required to complete each survey. Additional data collected by one technical advisor included the amount of time required to: (i) arrive at the first randomly selected household in each cluster, (ii) walk the distance between households, and (iii) complete each segment of travel. To calculate the time required for data collection, average measures of time expenditure were applied to the cluster and sample-size specifications of each design.
Data were checked for out-of-range and missing-values and entered (June 5–13 2003) using SPSS V7.0. Identical data cleaning and transformation procedures were applied to the 30 x 30, 33 x 6 and 67 x 3 data sets. Anthropometric Z-scores were calculated using the 1978 NCHS growth references and the EpiNut sub-routine of Epi-Info 6.04. Children with out-of-range anthropometric values flagged by Epi-Info were excluded from analysis, unless bipedal oedema was present.18 The data cleaning resulted in some clusters having less than the prescribed sample-size. Also, certain clusters were larger than the planned design since the sampling protocol called for data to be collected on all children 6–59 months in a sampled household. Oversized clusters were treated by randomly eliminating the excess number of cases. The final sample-size of the 30 x 30, 33 x 6 and 67 x 3 designs was 871, 195 and 194, respectively. To treat unequal cluster-sizes within designs, data were weighted inversely proportional to the achieved sample-size during point estimate calculations.
The 30 x 30 cluster-survey data were analysed using the CSample sub-routine of Epi-Info 6.04 which calculated the design effect for GAM (DE = 2.175) and other child-level indicators. Because computer simulations revealed the errors associated with GAM to approximate those of a SRS for the 33 x 6 and 67 x 3 designs,13 accounting for design effects for GAM was not essential. The primary analysis of GAM prevalence for the 33 x 6 and 67 x 3 data sets was therefore conducted using SPSS V12.0. For this analysis, the data were not weighted. To perform a test of the SRS assumption, a secondary analysis of GAM prevalence was also conducted on the 33 x 6 and 67 x 3 data, using CSample-the same analysis method applied to the 30 x 30 cluster-data. All other indicators for the 33 x 6 and 67 x 3 designs were analysed using the Epi-Info CSample sub-routine.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Point estimate analysis
The prevalence of GAM from the 33 x 6 and 67 x 3 designs found estimates of 6.7% (CI = 3.2–10.2%) and 8.2% (CI = 4.3–12.1%), respectively. The results are similar to the 30 x 30 cluster-design which estimated a GAM prevalence of 7.4% (CI = 4.8–9.9%). The CSample analysis of GAM prevalence for the 33 x 6 and 67 x 3 data produced results similar to those which assumed a SRS (Table 2). As evidenced by the overlapping CIs, indicators related to child morbidity and vaccination status also yielded similar results across all designs (Table 3).
|
|
GAM threshold analysis
Table 4 shows the DRs used to test the 33 x 6 and 67 x 3 designs for the 10, 15 and 20% GAM thresholds. GAM was found in 13 children in the 33 x 6 design (n = 195) and in 16 children in the 67 x 3 design (n = 194). With DR = 23, both designs indicate the GAM prevalence is <15% (Table 4). The assessment of whether the 10% threshold had been reached (DR = 13) yielded mixed results across designs. The 33 x 6 design indicates the GAM prevalence is <10%, whereas the 67 x 3 design indicates the GAM prevalence is
10%.
|
The sequential design shows GAM classification results consistent with those of the 67 x 3 design. Our analysis indicates that judgments could be made before all the data were collected. Analysis by cluster reveals that sufficient information was available to judge GAM prevalence <20% after cluster 13 (n = 39). After cluster 38 (n = 113) GAM prevalence could be judged <15%. The judgment of GAM prevalence
10% was possible after cluster 61 when 14 of 177 children exhibited GAM (Table 5).
|
Data collection time
Analysis of the time expenditure data showed, on average, two clusters in the 30 x 30 design could be completed by one interview team (three people) in a 12 h work day (30clusters/2 per day x 3 people = 45 person days). A 12 h work day was therefore assumed for all time estimation calculations. Our analysis shows the time required for the 30 x 30 cluster-survey and the three LQAS designs ranged from 5.22 to 45.00 person-days (Table 6). These time expenditure results are valid for this study only. Results would vary according to the geography, infrastructure and prevalence of malnutrition in the area sampled.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This study compared three new LQAS designs with the 30 x 30 cluster-design commonly used for emergency nutrition surveys. Across designs the following results were compared: point estimates and 95% CIs for GAM and other child-level indicators, classification results for GAM thresholds of 10, 15 and 20%, and time required for data collection. Similar point estimates and comparable CIs for GAM and other child-level indicators were found from the 30 x 30, 33 x 6 and 67 x 3 designs. The CIs for the 33 x 6 and 67 x 3 designs are only slightly wider than those of the 30 x 30 design for most indicators tested; yet, the LQAS designs required substantially less time when compared with the 30 x 30 cluster-survey. The CSample analysis of GAM prevalence showed the 33 x 6 and 67 x 3 designs to conform generally to the SRS assumption. The slightly inflated design effect (>1.000) for the 33 x 6 design is likely related to the reduced sample-size. Although this study showed inconsistent classification results for the 33 x 6 and 67 x 3 designs at the 10% threshold, this is probably due to the normal amount of error associated with surveys. Computer simulations have shown the statistical error associated with the 33 x 6 and 67 x 3 designs to be similar across designs at each GAM threshold.13
The most critical limitation of the study is the nested sample design. Ideally, data for each design should be sampled independently to allow for stricter comparison of results between designs. In addition, comprehensive time expenditure data would have been preferable. In this study, only one person (who rotated teams) collected time data related to driving, spin the bottle and walking household to household; with five teams working, one sample of time data may not represent all their efforts.
Despite these limitations, the LQAS designs described here can contribute to the methodological toolkit of humanitarian agencies. The 33 x 6 and 67 x 3 designs provide alternatives to the more time-consuming 30 x 30 cluster-survey. While alternative sampling methods may allow for a more precise CI for estimating GAM prevalence, the trade-off is a greater sample-size and more time for data collection than required by the 33 x 6 and 67 x 3 designs. The results from this study indicate the CIs for the LQAS designs are only slightly wider than the 30 x 30 design. However, because LQAS decision rules are based on binomial probabilities, in cases where the CI of a GAM point estimate overlaps a threshold for determining humanitarian action, the LQAS designs can be used for hypothesis testing, to make a decision about action. The LQAS decision rules minimize the alpha error so that the risk of not detecting a GAM threshold is small (0.10), providing programme managers with a useful tool to decide whether resources should be moved into an area. Furthermore, in comparison with a 30 x 30 cluster-survey, the LQAS designs are faster to implement, requiring fewer resources to obtain the information needed for action.
|
Acknowledgments |
|---|
The authors thank Stacy Hoshaw-Woodard for conducting the computer simulation work. The authors are also grateful to Cherinet Abuye for training the interview teams and supervising data collection, and to the country offices of CRS and Christian Children's Fund (CCF) in Ethiopia for the administrative and logistic support provided during the fieldwork. The authors also acknowledge Eunyong Chung, Bruce Cogill, Nadra Franklin, and Anne Swindale for their review of the manuscript. Special thanks are due to Gilles Bergeron who reviewed an earlier draft of the manuscript and provided valuable editorial comments. Financial support for this project was provided to CRS from the Andrew W Mellon Foundation and to the Food and Nutrition Technical Assistance (FANTA) Project by the Office of Health, Infectious Diseases and Nutrition of the Bureau for Global Health and the Ethiopia Mission at the US Agency for International Development (USAID), under terms of Cooperative Agreement No HRN-A-00-98-00046-00 awarded to the AED. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of the Andrew W Mellon Foundation or USAID.
Conflict of interest: None declared.
KEY MESSAGES
|
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1 Mason JB. Lessons on nutrition of displaced people. J Nutr (2002) 132:2096S–2103S.
2 World Health Organization (WHO). Management of Nutrition in Major Emergencies. (2000) Geneva: World Health Organization.
3 World Health Organization (WHO). Physical status: the use and interpretation of anthropometry. In: Report of a WHO expert committee. (1995) Geneva: World Health Organization.
4 University of Nairobi. Report of a Workshop on the Improvement of the Nutrition of Refugees and Displaced People in Africa. (1995) Geneva: Standing Committee on Nutrition (SCN).
5 Standardized Monitoring and Assessment of Relief and Transitions (SMART). In: Measuring mortality, nutritional status and food security in crisis situations. Version 1, 2006. Available at: http://www.smartindicators.org/.
6 Medicins Sans Frontieres (MSF). Nutrition Guidelines. (1995) 1st. Paris: Medicins Sans Frontieres.
7 Spiegel PB, Salama P, Maloney S, van der Veen A. Quality of malnutrition assessment surveys collected during famine in Ethiopia. JAMA (2004) 292:613–18.
8 Valadez JJ. Assessing child survival programs in developing countries: testing lot quality assurance sampling. (1991) Cambridge: Harvard University Press.
9 Reinke W. Applicability of industrial sampling techniques to epidemiologic investigations: examination of an underutilized resource. Am J Epidemiol (1991) 132:1222–32.
10 Valadez JJ, Weiss B, Leburg C, Davis R. Assessing Community Health Programs: using LQAS for baseline surveys and regular monitoring. (2003) London: Teaching-aids at Low Cost.
11 Roberston SE, Valadez JJ. Global review of health care surveys using lot quality assurance sampling (LQAS), 1984–2004. Soc Sci Med (2006) 63:1648–60.[CrossRef][Web of Science][Medline]
12 Save the Children (SC) UK. Emergency Nutrition Assessment: Guidelines for Field Workers. (2004) London: Save the Children.
13 Hoshaw-Woodard S. The use of LQAS sampling to assess malnutrition. (2003) Ohio State University. Version II, Final Draft.
14 Ethiopia Early Warning Department, Disaster Prevention and Preparedness Commission (DPPC). (2002) Ethiopia Early Warning System (EWS): Guideline on emergency nutrition assessment. Addis Ababa: Early Warning Department/Disaster Prevention and Preparedness Commission.
15 Famine Early Warning System Network (FEWS NET). (2002) Dec 9. Washington, DC. Ethiopia: unmistakable pre-famine conditions. Ethiopia monthly food security update. Available at: http://www.fews.net.
16 Ethiopia Disaster Prevention and Preparedness Commission. (2003) June. Siraro: Unpublished data.
17 Cogill B. Anthropometric Indicators Measurement Guide. (2003) Washington, DC: FANTA Project/Academy for Educational Development.
18 Dean A, Dean J, Coulombier D, et al. Epi Info: A Word Processing, Database, and Statistics Program for Public Health on IBM Compatible Microcomputers. (1996) 6th. Atlanta: Centers for Disease Control and Prevention.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  P. Garner and G. D. Smith Information for decision making: lot quality assurance sampling in the spotlight Int. J. Epidemiol., February 1, 2010; 39(1): 5 - 6. [Full Text] [PDF]
|
|
|
|
|
|
S. Morris Commentary: Learning to Love Lot Quality Assurance Sampling Int. J. Epidemiol., August 1, 2007; 36(4): 864 - 865. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||