International Journal of Epidemiology, Vol 28, 10-18, Copyright © 1999 by International Epidemiological Association
JP Habicht, CG Victora and JP Vaughan
The question of why to evaluate a programme is seldom discussed in the
literature. The present paper argues that the answer to this question is
essential for choosing an appropriate evaluation design. The discussion is
centered on summative evaluations of large-scale programme effectiveness,
drawing upon examples from the fields of health and nutrition but the
findings may be applicable to other subject areas. The main objective of an
evaluation is to influence decisions. How complex and precise the
evaluation must be depends on who the decision maker is and on what types
of decisions will be taken as a consequence of the findings. Different
decision makers demand not only different types of information but also
vary in their requirements of how informative and precise the findings must
be. Both complex and simple evaluations, however, should be equally
rigorous in relating the design to the decisions. Based on the types of
decisions that may be taken, a framework is proposed for deciding upon
appropriate evaluation designs. Its first axis concerns the indicators of
interest, whether these refer to provision or utilization of services,
coverage or impact measures. The second axis refers to the type of
inference to be made, whether this is a statement of adequacy, plausibility
or probability. In addition to the above framework, other factors affect
the choice of an evaluation design, including the efficacy of the
intervention, the field of knowledge, timing and costs. Regarding the
latter, decision makers should be made aware that evaluation costs increase
rapidly with complexity so that often a compromise must be reached.
Examples are given of how to use the two classification axes, as well as
these additional factors, for helping decision makers and evaluators
translate the need for evaluation--the why--into the appropriate design-
-the how.
ARTICLES
Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact
Division of Nutritional Sciences, Cornell University, USA.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  A. J. Barros and A. D Bertoldi Out-of-pocket health expenditure in a population covered by the Family Health Program in Brazil Int. J. Epidemiol., August 1, 2008; 37(4): 758 - 765. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Menon, M. Mbuya, J.-P. Habicht, G. Pelto, C. U. Loechl, and M. T. Ruel Assessing Supervisory and Motivational Factors in the Context of a Program Evaluation in Rural Haiti J. Nutr., March 1, 2008; 138(3): 634 - 637. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Brooker, N. B Kabatereine, F. Fleming, and N. Devlin Cost and cost-effectiveness of nationwide school-based helminth control in Uganda: intra-country variation and effects of scaling-up Health Policy Plan., January 1, 2008; 23(1): 24 - 35. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Wells Pence, P. Nyarko, J. F. Phillips, and C. Debpuur The effect of community nurses and health volunteers on child mortality: The Navrongo Community Health and Family Planning Project Scand J Public Health, December 1, 2007; 35(6): 599 - 608. [Abstract] [PDF]
|
|
|
|
 |
|
 M-C Boily, C M Lowndes, P Vickerman, L Kumaranayake, J Blanchard, S Moses, B M Ramesh, M Pickles, C Watts, R Washington, et al. Evaluating large-scale HIV prevention interventions: study design for an integrated mathematical modelling approach Sex. Transm. Inf., December 1, 2007; 83(7): 582 - 589. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. C. Kim, C. H. Watts, J. R. Hargreaves, L. X. Ndhlovu, G. Phetla, L. A. Morison, J. Busza, J. D.H. Porter, and P. Pronyk Understanding the Impact of a Microfinance-Based Intervention on Women's Empowerment and the Reduction of Intimate Partner Violence in South Africa Am J Public Health, October 1, 2007; 97(10): 1794 - 1802. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. G Victora Commentary: Epidemiology and futurology why did Rothman get it wrong? Int. J. Epidemiol., August 1, 2007; 36(4): 712 - 713. [Full Text] [PDF]
|
|
|
|
|
|
A. E Pettifor, C. MacPhail, S. Bertozzi, and H. V Rees Challenge of evaluating a national HIV prevention programme: the case of loveLife, South Africa Sex. Transm. Inf., August 1, 2007; 83(suppl_1): i70 - i74. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Bonvecchio, G. H. Pelto, E. Escalante, E. Monterrubio, J. P. Habicht, F. Nava, M.-A. Villanueva, M. Safdie, and J. A. Rivera Maternal Knowledge and Use of a Micronutrient Supplement Was Improved with a Programmatically Feasible Intervention in Mexico J. Nutr., February 1, 2007; 137(2): 440 - 446. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. F. Killeen, M. Tanner, W. R. Mukabana, M. S. Kalongolela, K. Kannady, S. W. Lindsay, U. Fillinger, and M. C. de Castro HABITAT TARGETING FOR CONTROLLING AQUATIC STAGES OF MALARIA VECTORS IN AFRICA Am J Trop Med Hyg, April 1, 2006; 74(4): 517 - 518. [Full Text] [PDF]
|
|
|
|
 |
|
 L. Ross, P. Simkhada, and W. C. S. Smith Evaluating effectiveness of complex interventions aimed at reducing maternal mortality in developing countries J. Public Health Med., December 1, 2005; 27(4): 331 - 337. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. Robitaille, S. Laforest, M. Fournier, L. Gauvin, M. Parisien, H. Corriveau, F. Trickey, and N. Damestoy Moving Forward in Fall Prevention: An Intervention to Improve Balance Among Older Adults in Real-World Settings Am J Public Health, November 1, 2005; 95(11): 2049 - 2056. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M Chopra, S Patel, K Cloete, D Sanders, and S Peterson Effect of an IMCI intervention on quality of care across four districts in Cape Town, South Africa Arch. Dis. Child., April 1, 2005; 90(4): 397 - 401. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. Allen and R. Shrimpton The International Research on Infant Supplementation Study: Implications for Programs and Further Research J. Nutr., March 1, 2005; 135(3): 666S - 669S. [Full Text] [PDF]
|
|
|
|
|
|
N C Grassly, M Morgan, N Walker, G Garnett, K A Stanecki, J Stover, T Brown, and P D Ghys Uncertainty in estimates of HIV/AIDS: the estimation and application of plausibility bounds Sex. Transm. Inf., August 1, 2004; 80(suppl_1): i31 - i38. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. A. Rivera, D. Sotres-Alvarez, J.-P. Habicht, T. Shamah, and S. Villalpando Impact of the Mexican Program for Education, Health, and Nutrition (Progresa) on Rates of Growth and Anemia in Infants and Young Children: A Randomized Effectiveness Study JAMA, June 2, 2004; 291(21): 2563 - 2570. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. K. Stanton Methodological Issues in the Measurement of Birth Preparedness in Support of Safe Motherhood Eval Rev, June 1, 2004; 28(3): 179 - 200. [Abstract] [PDF]
|
|
|
|
|
|
C. G. Victora, J.-P. Habicht, and J. Bryce Evidence-Based Public Health: Moving Beyond Randomized Trials Am J Public Health, March 1, 2004; 94(3): 400 - 405. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Bryce, C. G. Victora, J.-P. Habicht, J. P. Vaughan, and R. E. Black The Multi-Country Evaluation of the Integrated Management of Childhood Illness Strategy: Lessons for the Evaluation of Public Health Interventions Am J Public Health, March 1, 2004; 94(3): 406 - 415. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. H. Pelto, I. Santos, H. Goncalves, C. Victora, J. Martines, and J.-P. Habicht Nutrition Counseling Training Changes Physician Behavior and Improves Caregiver Knowledge Acquisition J. Nutr., February 1, 2004; 134(2): 357 - 362. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. J. Conrey, E. A. Frongillo, J. S. Dollahite, and M. R. Griffin Integrated Program Enhancements Increased Utilization of Farmers' Market Nutrition Program J. Nutr., June 1, 2003; 133(6): 1841 - 1844. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E.-C. Ekstrom, S. Z. Hyder, A M. R Chowdhury, S. A Chowdhury, B. Lonnerdal, J.-P. Habicht, and L. A. Persson Efficacy and trial effectiveness of weekly and daily iron supplementation among pregnant women in rural Bangladesh: disentangling the issues Am. J. Clinical Nutrition, December 1, 2002; 76(6): 1392 - 1400. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. W. Bloem, L. Kiess, and R. Moench-Pfanner Process Indicators for Monitoring and Evaluating Vitamin A Programs J. Nutr., September 1, 2002; 132(9): 2934S - 2939. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. J. Stoltzfus and G. Pillai Measuring Performance: A Strategy to Improve Programs J. Nutr., April 1, 2002; 132(4): 845S - 848. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S Abdulla, J. A. Schellenberg, O Mukasa, and C Lengeler Usefulness of a dispensary-based case-control study for assessing morbidity impact of a treated net programme Int. J. Epidemiol., February 1, 2002; 31(1): 175 - 180. [Abstract] [Full Text] [PDF]
|
|