IJE Advance Access originally published online on March 15, 2006
International Journal of Epidemiology 2006 35(3):593-596; doi:10.1093/ije/dyl008
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Author's Response |
On stones, wands, and promises
1 Department of Anthropology, Penn State University, University Park, PA 16802, USA
2 Department of Biology, Penn State University, University Park, PA 16802, USA
3 Department of Medical History and Ethics, University of Washington School of Medicine, Seattle, WA 98195, USA
* Corresponding author. Department of Biology, 409 Carpenter Building, Penn State University, University Park, PA 16802, USA. E-mail: KenWeiss{at}psu.edu
Accepted 5 January 2006
The remarkable advances in molecular biology during the past two decades have given man an understanding of the basic processes that shape his life and have placed within the realm of possibility medical achievements undreamed of a scant few years ago.The way is being opened not only for permanent cures of genetic disease but also for drastic changes in man's genetic makeup.
These quotes, from an article in Time magazine, sound themes repeated again and again by those who believe that the major causes of complex diseases are to be found in our genes. Justifications for continuing to pour taxpayer and investor money into the search for genetic causes of complex disease include: (1) that we stand at the threshold of great discovery, with major advances in health right around the corner as a consequence; (2) we are not doing this for ourselves, or even for our children, but for our children's children; (3) although the odds are long on actually unravelling the causes of disease with current approaches, we might make an accidental discovery along the way; (4) science works incrementally and discoveries are made in fits and starts, and expecting quick results is asking too much (see number 1); (5) there have been notable successes and we risk missing others if we change course now; (6) science progresses as much by rejecting hypotheses as by confirming them, so the inconsistent results so common in complex disease research justify an ever expanded search effort; or (7) we do this because nothing else is working.
These are some of the regularly offered justifications for current genetic epidemiological research, but they are matched by far fewer cautions. We appreciate the thought respondents gave to our paper.1–7 We are well aware that we provided a large target, and are pleased that, for the most part, our cautions were not dismissed out of hand. Nor did the respondents gainsay our attempt to explain the epistemological reasons for the problems that we face today. We find it telling that even the respondents who question our views neither refute them specifically nor mount any kind of vigorous defence of the status quo beyond stating what essentially amounts to a belief that extending current methodology with ever larger, longer studies and more technological and biostatistical sophistication, and, of course, moving to genetics will do the trick. Indeed, several respondents sound even more dire warnings than our own. Some proffer solutions, but they generally do not take into account the changing biological landscape facing both epidemiology and human genetics now that ‘easy’ diseases have been explained (responsible genes or environmental risk factors identified for essentially single cause diseases).
All the respondents would no doubt agree with Dr Weed2, as we do, that epidemiology has a venerable history. Its practitioners surely share Dr Weed's belief that ‘health is a fundamental social good’, and enhancing this social good is why most people are in this field and why we undertook to write the kind of paper we wrote: when a discipline faces fundamental problems, it does no one—particularly students—a service to deny them. When we have been feeding the public exuberant promises of enormous return on investment, from personalized medicine to near-immortality, it is wrong to evade accountability for how their investment is spent. And, arguably the stakes are higher now than they have ever been—many of the proposed fixes for complex diseases require greater resources, for potentially smaller returns, than ever before.
As several respondents noted about our metaphor, alchemy was not entirely in vain. Indeed, it occupied leading scientific thinkers for centuries, including Isaac Newton himself. Some threads ultimately morphed into modern chemistry. But in this context one respondent objected to our raising the subject of vested personal and professional interests as an obstacle to progress—that we refer to as an ‘informal Bayesian’ effect on the way we make inferences—so it is entirely in order for us to note that alchemists were trying to turn base metals into gold, not oatmeal (and Newton finished his career in charge of the Royal Mint!). At least they were doing it largely with their own venture capital.
Funding, institutional, and career dependencies are substantial inertial factors in many areas of science, including complex disease research. These things are widely acknowledged privately, but rarely publicly, for the obvious reason that public support is required for research and our society works by lobbying and public persuasion. It would be highly disingenuous to deny that funding availability has a major effect on research directions and, hence, the way data are collected and inferences made. Given the problems we raise, ignoring these facts is a disservice to the future of the science itself. However, there may be many practicable ways to encourage new mutations in the idea-world. These might include increasing the scope for creative thinking by reducing the concentration of large vested interests, reducing the conservative pull of institutional demand for indirect costs and soft-money salaries; increasing incentives for innovative and creative thought by providing independent support for more investigators, on a more stable basis, even if with lower per capita funding; and insisting on accountability for results that is proportionate to investment, for example by rewarding the funding bureaucracy for the effectiveness of its investment decisions rather than the size of its portfolios. These kinds of systemic changes would not lead directly to new methods for parsing complex diseases, but they might do so indirectly by opening the system and nurturing young independent investigators whose innovative thinking will produce those solutions.
Aside from these generalities, we also need to address some of the specific points raised by the respondents. Dr Ioannidis3 writes that the problems in epidemiology, as well as human genetics, may be exacerbated by poor study quality, and he properly suggests the value of detecting and discounting poorly designed studies (although he does not say so, this might include excluding from meta-analysis the inflating effect of initial discovery and mapping studies that are typically affected by many sources of bias). Eliminating poor studies would help clear the fog and reduce the number of candidate risk factors that need to be considered.
Dr Ioannidis further suggests that we need to dissect a complex disease into hundreds of genetic risk factors of small effect because parsing the causes of these diseases into a limited number of macrofactors simply has not worked. However, this assumes we can reliably identify these many risk factors and estimate their relative risks, and that they would be usefully stable and unaffected by comparably numerous microscale environmental factors. This reductionist solution sounds wonderful in principle, but think about its implications: Many if not most of these risk factors are likely to be context-dependent, because common complex chronic diseases generally have highly variable prevalence, both spatially and temporally. Thus, gene–environment (or inter-environment) interactions must be involved if environmental factors do not, as Dr Ioannidis correctly says, account for risk on their statistical margins—that is, by themselves. We use the term epistemological to refer to the profound empirical problems we face, beyond identifying the factors involved, in attempting to reliably quantify the myriad 2-, 3-, ... and higher-order interactions among hundreds of genetic as well as environmental factors, not to mention their variation among populations and sub-populations. This means attempting thousands of independent, individually rare risk estimates and predictions even for a single disease. Yet, this kind of causal dissection is just what is being widely promised in human genetics today, and more than one of the respondents seem sanguine about its promise, even if they offer little or no supporting justification. We think what is needed instead are new conceptual approaches to understand and deal with the aggregate of effects that are individually ephemeral or even basically inestimable.
The usefulness of the unquestioned technological and statistical wizardry of science rests on the further assumptions that risks estimated in current studies will be valid prospectively, meaning that all important confounders have been identified, that substantial new factors will not enter the mix, and that future environmental exposures will be predictable. This is a major rationale behind using huge prospective ‘biobanks’ to get around some of the problems in retrospective studies. But, realistically, we will not know all risk factors affecting complex diseases, nor their numerous confounders, and those that are identified in the future will have to be ascertained retrospectively or remeasured with ever-changing criteria even in biobank subjects, just as they are today. So again what sounds totally reasonable in principle is, on closer inspection, less than transparent, to say the least, and this does not include the large costs indefinitely encumbered by such decades-long studies. This is another reason we argue that new conceptual thinking, not just scaled-up technology, is needed.
Dr Coggon4 points out the long history of successes in occupational epidemiology, and indeed, this field has had many. These are mainly risk factors with large replicable effects, the kinds of risk factors we describe in our paper as just those for which current methods have worked very well, so it is not surprising that the list is long. While he points out that the promise of genetic epidemiology has not yet been realized, he suggests that genetic studies have ‘less potential for bias and confounding than other areas of epidemiology, and lower relative risks can be interpreted with greater confidence’. In fact, in a subtle way, the opposite is often the case for genetic mapping (gene identification) studies, where confounding (among genetic markers in the genome) and bias (due to heavy multiple testing) are actually maximally designed into standard sampling strategies.8,9 The resulting bias in the estimated effect sizes contributes to the frequent disappointment we have experienced in replication attempts.
Drs Schwartz and Susser5 write a sobering commentary, though they are cautiously optimistic and offer what they hope are the beginnings of a framework for a more robust epidemiology. They note that it was the discovery of microbes that led to infectious disease epidemiology with its many successes, and that the discovery of the link between smoking and lung cancer ushered in the age of chronic disease epidemiology, but that there has been no equivalent discovery to usher in a new and equally successful subset of epidemiology for our new age. We suggest that, in fact, what is widely viewed as the new, new epidemiology is genetic epidemiology, which was ushered in by the discovery of single-gene diseases such as cystic fibrosis, PKU and the small ‘Mendelian’ subsets of complex traits, such as familial hypercholesterolaemia. As we wrote, we believe that the discovery of genes for Mendelian diseases was so successful that it lured many epidemiologists, not to mention human geneticists now dealing with complex chronic diseases, to believe that these diseases will be tractable in the same way. But we tried to explain in terms of inferential criteria and known aspects of genetics why extrapolating Mendelian concepts into areas where the signal-to-noise ratio approaches unity may not be effective.
As the authors of a paper detailing many thorny problems for which we ourselves do not claim to have the solutions, we appreciate the demonstration by Drs Schwartz and Susser of one potential strategy. But we caution that the solution is going to have to go beyond refined methodology, beyond ensuring that we are testing the correct hypothesis, beyond tighter study design and ever improved technologies, to more biologically grounded approaches. Among other things, these approaches should take into account the fact that nature works via natural selection that screens overall phenotypes regardless of their individual underlying genotypes, which selection cannot ‘see’,10 so epidemiologists should not expect to be able to detect clearer genotype-phenotype relationships either.
Dr Millikan6also makes useful and constructive suggestions, though he is more optimistic about epidemiology's embrace of genetics than we are. We would only like to note one area in which, ironically, the truth is more likely to be the reverse of his argument. He doubts that the oft-promised individualized medicine based on genotyping at birth—Your Life on a Chip—will ever get off the ground but feels that genetic data will make positive contributions on a population-scale. While we share his scepticism with respect to the widespread use of individualized risk assessment, we argued that if we stay within the fraction of complex traits that are truly genetic, like certain BRCA1 and 2 mutations, FH mutations, and the like, that this is just the area in which genetics may be the legitimate, indeed the preferred, approach to detection, prevention, and ultimately to therapeutic intervention as well. In this restricted sense, Dr Millikan might agree.
It is difficult to know what to make of Dr Weed's commentary2. Overall, he presents a satiric defence of the status quo, and in blank verse at that. He denigrates our critique of epidemiology, but, as we wrote, we are well aware of the field's long history of major contributions to Public Health and we are by no means the first to point out the problems in the field, as he and other respondents said, and as readers of this journal are well aware. The turn towards human genetics as a potential corrective has been, at least in part, in response to the frustrations of the status quo where other approaches have not worked. It is to sound a cautionary note about this turn to genetics that we wrote our paper: the classical approaches of both genetics and epidemiology are stymied for similar reasons by the very nature of the diseases that have become the focus of both fields.
Dr Weed is further distressed by what he sees as the pressure on epidemiologists to choose between biology and sociology as causes of disease. As we asked no such choice of epidemiologists, he seems to be voicing a more generic unhappiness of his that goes beyond our particular paper. We would just point out that the many issues of determining causation, such as we discuss, are as applicable to social risk factors as they are to the biological. But, and again we are by no means the first to write this, when the correlation between risk of disease and poverty, ethnicity, low level of education, proximity to hazardous industry or waste dumps, and the like is high, it is at least fair to ask whether the larger contribution to public health would not be to put resources disproportionately into addressing underlying social and political issues rather than to continue to look for genetic—or even environmental—‘causes’ of associated diseases. We do not address this in our paper, but as Dr Weed does in his commentary, we will note that it touches on the larger question of how best to contribute to Public Health, which we believe should be a central part of the larger debate. Surely there is more than one answer, and addressing the association of ethnicity, unequal distribution of resources and risk of disease may be one—perhaps Dr Weed would even agree with this.
Drs Merikangas, Low and Hardy7 write that we are preaching to the choir, that our message is already well known to all serious epidemiologists, and presumably the lessons learned and well established in practice. But they seem to want things both ways, by then dismissing those same arguments as nihilistic and unhelpful. And, they write strongly in defence of simply scaling up, using traditional methods and approaches, in spite of what ‘everybody knows’. If these issues truly have already been hashed out and resolved, then it would seem that a large number of people are ignoring the lessons. Merely paying lip service to a lesson is effectively the same as not having learned it at all.
Merikangas et al. suggest that we do a disservice to the understanding of the role of gene–environment interaction in disease and fail to recognize the importance of traditional case–control studies, but traditional case–control family studies of diseases with possible genetic and environmental causation are a classic case in point when it comes to ignoring the role of the environment. As we discuss in the paper, such studies tend to assume genetic causation and ignore alternative explanations for lack of disease in controls, which can be a real problem when the disease is a common one.
Merikangas et al. further state that failure is as important a path of progress in science as success. That is highly debatable from a historical point of view, but in any case, an important point we tried to stress is that it is not always clear what constitutes failure or how to recognize it. When should an epidemiological hypothesis be rejected? When one study contradicts it? Two? Even after Dr Ioannidis's guidelines have given bad grades to all the poor studies, contradictory results will remain. They aren't all the result of poor research. How precisely and under how many different conditions do we have to keep measuring the same risk, and what do those estimates mean? What are our stopping rules? When is a case closed?
Merikangas et al. end their commentary with a declaration of faith that we can ramp up our studies to identify and measure many causal variables, and avoid confounders, implying that we will not increase heterogeneity and statistical noise or overdetermination faster than we can get adequate samples—these same problems pertain to Dr Ioannides' suggestion of increasing the number of variables we collect. We ask readers not just to react to the optimistic platitude of the Merikangas et al. closing quote but to think about its epistemological implications.
The operant question is where most complex traits lie on the scale from infinitesimal to unitary causation. A wealth of evidence, and century-old genetic theory that is also relevant to environmental exposures, suggest that most such traits lie toward the former. To the extent this is so, the epistemological problems apply formally, in that the bulk of risk factor regimes could not even in principle be individually identified, each person being unique in causally uncharacterizable ways. It is the diametric opposite of nihilism to suggest that science really does understand the basis of complex traits, even if that implies no golden panacea. That knowledge constitutes progress if we but use it!
To conclude, our paper is a groping attempt to understand and explain at a deeper level the reasons for problems we see in both epidemiology and human genetics as both fields wrestle with the causes of complex disease. We do not claim even to have all the questions, much less answers to them. And we certainly do not aim to discourage prospective students from going into these fields. Quite the contrary: we hope it is precisely the people who are motivated to go into Public Health in order to make a difference who will read our comments—and those of the authors who precede us in thinking about problems in this field—and decide that they want to take on the challenge of making epidemiology a discipline whose discoveries are robust and credible. That is the best way to honour an honourable calling.
Clearly, long-term investment in basic science is important, but we already know, from a Public Health perspective, that the preponderance of cases of complex chronic disease are owing to exogenous experience rather than endogenous genetic susceptibility, in the sense that people are not born with complex late-onset disease. If a major fraction of the billions spent on technological research were spent instead on simpler things like, yes, early health education to improve diet and promote exercise, the benefits could grossly dwarf even the greatest plausible genetic successes, especially with regard to late-onset disease. And then focusing our technological abilities on traits that really are genetic, and there are many of them, could eventually lead to their treatment and even eradication.
And the quotes from Time with which we opened this response? They are from the cover story of the April 17, 1971 issue titled The New Genetics: Man into Superman. The long history of over-promising, of equivocal result after equivocal result, and behavioural and dietary recommendations that change faster than we can change our behaviour and menus, has already turned much of the funding public, the stakeholders, into non-believers. How many more times can we tell them that genetics will achieve permanent cures for almost all diseases, and heretofore undreamt-of medical achievements, before they have every right to transfer their support to other endeavours?
 |
References |
|---|
 Top References |
|---|
1 Buchanan AV, Weiss KM, Fullerton SM. Dissecting complex disease: the quest for the philosopher's stone? Int J Epidemiol 2006;35:562–71.
2 Weed DL. Commentary: rethinking epidemiology. Int J Epidemiol 2006;35:583–86.
3 Ioannidis JPA. Commentary: grading the credibility of molecular evidence for complex diseases. Int J Epidemiol 2006;35:572–78.
4 Coggon D. Commentary: Complex disease—responding to the challenge. Int J Epidemiol 2006;35:581–83.
5 Schwartz S, Susser E. Commentary: What can epidemiology accomplish? Int J Epidemiol 2006;35:587–90.
6 Millikan RC. Commentary: The human Genome: philosopher's stone or magic wand? Int J Epidemiol 2006;35:578–81.
7 Merikangas KR, Low NCP, Hardy J. Commentary: Understanding sources of complexity in chronic diseases: the importance of integration of genetics and epidemiology. Int J Epidemiol 2006;35:590–92.
8 Goring HH, Terwilliger JD et al. Large upward bias in estimation of locus-specific effects from genomewide scans. Am J Hum Genet 2001;69:1357–69.[CrossRef][Web of Science][Medline]
9 Terwilliger JD, Weiss KM. Confounding, ascertainment bias, and the blind quest for a genetic ‘fountain of youth’. Ann Med 2003;35:532–44.[CrossRef][Web of Science][Medline]
10 Weiss KM, Buchanan A. Evolution by phenotype. Perspect Biol Med 2003;46:159–82.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  W. Maziak The triumph of the null hypothesis: epidemiology in an age of change Int. J. Epidemiol., April 1, 2009; 38(2): 393 - 402. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. J KHOURY and M. GWINN Genomics, epidemiology, and common complex diseases: let's not throw out the baby with the bathwater! Int. J. Epidemiol., October 1, 2006; 35(5): 1363 - 1364. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||