Commentary: Lack of scientific influences on epidemiology
Department of Health Sciences, University of Alberta, 114 St–89 Ave, Edmonton, Alberta, T6G 2E1, Canada.
E-mail: carl.v.phillips{at}ualberta.ca
Accepted 4 September 2007
Unethical, anti-scientific, politically motivated attacks on health research and researchers by wealthy organized interests can be a nightmare. I know this from experience.
In ‘Corporate Influences on Epidemiology’,1 Neil Pearce recounts a bit of what it is like to have one's scientific contributions attacked because someone does not like the worldly implications of the results, and suggests that the problem is sufficiently pervasive to be a threat to epidemiology. The concerns, and especially the advice, Pearce offers seem to fit my own experience well. I have developed a thick skin, focused on the long run, and avoided imitating those who attack me by overstating or misrepresenting the evidence. I find strength in the fact that, as Pearce notes, most of the time science—not political correctness, not well-funded advocates—wins in the end.
The twist on Pearce's message in my case is that he might consider me to be on the wrong side (others who are less tempered and thoughtful than Pearce might go further in their condemnation), since I have done a few critical scientific analyses of epidemiological claims as part of industry consultancies. More significantly, the anti-scientific attacks on epidemiology that I have been a victim of have come not from corporations, or even government, but from those who are thought by most people to be public health advocates. The players and specific areas of research are different, but as with corporate influence, influential organized interests are willing to damage science and even sacrifice people's health to further their goals.
As a brief summary, much of my work focuses on the possibility that the health impact of nicotine use might be considerably reduced by encouraging smokers to switch to an alternative source of nicotine, oral smokeless tobacco being the most promising, since ample epidemiological evidence clearly shows that smokeless tobacco causes about 1/100th the total disease risk from smoking.2–4 This approach, known as ‘tobacco harm reduction’5—analogous to other harm reduction strategies such as promoting condom use, needle exchange or seatbelts—is opposed by the wealthy and influential non-profit and governmental organizations that control almost all of the funding and discourse on tobacco and health. Those organizations have effectively misled the public and policy makers about the epidemiology,2,4,6,7 overstating the health risks from ST use by orders of magnitude. Thus, there is little realization of the potential of harm reduction by smokers or even most health professionals. (Why organizations that claim to be concerned with people's health prefer to let them die from smoking, rather than learn of a very good alternative, is beyond the scope of this commentary.)
Tobacco harm reduction involves substantial potential for disease and mortality reduction and several genuine uncertainties (though the underlying epidemiology of comparative health risks is not among them), making it an important area for research. However, the organizations that control most of the agenda and funding for studies of tobacco and health actively block research that might undermine their abstinence-only (a.k.a. ‘quit or die’) activist positions. Those organized interests have used their power to try to de-fund me and my students, terminate my faculty position and censor the presentation of information about tobacco harm reduction by me and others. They have been successful at some of these to a disturbing extent, and may yet succeed at all of them. I provide some detail about the actors (non-corporate entities that include advocacy organizations, the administration of the new University of Alberta School of Public Health, and others) and their actions (sufficiently shocking that I am concerned that mentioning them would distract from the main message of this commentary) in a recent article.8
As outlandish as my situation has been, epidemiologist Jim Enstrom has suffered even more at the hands of anti-tobacco activists for publishing a single result from his research (which has not been challenged on substantive scientific grounds) that they did not like.9,10 Some of the ad hominem attacks on Enstrom and on me are facilitated by the fact that those of us who do research that the anti-tobacco orthodoxy does not like, and thus will not fund, need to seek grants from industry. (For both Enstrom and me, those grants have been completely hands-off money: no restrictions on its use, no involvement of the funders with research decisions, no pre-publication access to the work.) Those who control the competing funding use this as an excuse to attack us without addressing the scientific merit of our work. But this is clearly just a convenient rationalization for the attacks, given that every researcher I know who dared depart from the anti-tobacco orthodoxy—whether or not they ever accepted grants from the industry—has suffered similar attacks.
Pearce and others have implied that an association between funding sources and results means that industry funding causes researchers to reach certain conclusions. Many who invoke this commit the classic epidemiologic error of reversing cause and effect. In controversial areas of public health research, academics who have previously published conclusions or want to do research that might be seen as positive for the industry typically find that they are refused funding from sources other than industry. Pearce, unlike some others, recognizes that cause and effect often flow in this direction, but he fails to carry through that analysis to note that such causation is driven by the behaviour of the non-industry funders, not the industry or the researchers. Moreover, the situation is remarkably symmetrical: a history of conclusions that support the agendas of particular government and advocacy organizations increases the chance of getting funding from them.
What is less symmetrical is that it is difficult to imagine any corporate actor attempting (let alone getting away with) anti-scientific attacks on epidemiology remotely like those made by anti-tobacco activists. Some damage done by these attacks is overt and immediate, including misinforming the public and policy makers and directly interfering with research. But less visible damage includes erosion of the overall quality of the science and discouraging researchers from doing innovative or controversial work. My students have noticed that when they do research on harm reduction, they are harassed at every turn (e.g. the human subjects ethics board has yet to approve any of our projects, even when there is no risk to the subjects—when they did not merely sit on our requests for inexcusable periods they imposed literally impossible requirements; other professors demanded that one student withdraw from authorship of her papers to please outside constituents), whereas if they were to adhere to the abstinence-only orthodoxy they could tap into seemingly limitless funding, employment opportunities and uncritical approval.
Thus, I concur with Pearce that political interests have influences that pose a threat to epidemiology, though he was overly narrow in focusing on corporations. A few of the many other non-corporate distortions of epidemiology include: Advocates of cancer screening have often overstated the scientific evidence of its benefits and successfully established apparently inefficient standards of care. Officials working on HIV/AIDS have been convincingly accused of intentionally overstating disease prevalence.11 The US government has been repeatedly cited for making epidemiologic claims that are based only on puritanical religious doctrine.12
Of course, some of the behaviour Pearce cites represents inexcusable violation of scientific and academic ethics. Putting your name on a paper that someone else wrote without your involvement is indefensible. Perhaps, it is even worse when it is ghostwritten by an organized interest, compared with the more common variant, wherein a senior researcher ‘authors’ numerous papers thanks to his students and employees. When someone (corporation or otherwise) suggests the idea of doing a particular analysis (whether they pay for it or not), this should be acknowledged, to give due credit for the idea and to make clear that the work may have been done in pursuit of someone else's worldly agenda. And, of course, if that suggestion came with an attempt to dictate the outcome of the analysis, then it is no longer a suggestion, but is again indefensible ghostwriting.
 |
Rules and organizations will not provide a solution |
|---|
 Top Rules and organizations will... The need for more...Can epidemiology ever be...References |
|---|
Enforcement of these simple rules might be aided by clearer codes of conduct. But Pearce's sketches of solutions that involve codes, professional organizations and getting the ‘right’ people on influential committees offer little hope of addressing the fundamental challenges. Trying to thwart powerful interests by creating more power elsewhere has a very unfortunate history. The influence of powerful organized interests on government public health policy that Pearce alludes to is not some anomaly that can just be fixed once it is recognized. It is an example of the political tendencies toward agency ‘capture’13,14 and ‘iron triangles’ (e.g. the military-industrial complex)15 that are always present in government and that offer strong cases for maintaining pluralism as a counter to centralization.
It is easy to fall into a utopian belief that if some right-thinking individuals are put in a position of sufficient power, that a problem would be solved. But the annoying thing about positions of sufficient power (whether they be government offices or the Committee of the Grand Arbiters of Epidemiology) is that once they exist, we honest brokers seldom fill them. Usually the power ends up in the hands of an organized interest (often industry, though sometimes their antagonists).
My phrase ‘we honest brokers’ is, of course, a sarcastic allusion to the fact that most of us who seek to influence public policy believe we are doing the right thing in the right way. This is true for those who advocate distributing condoms or banning them, eliminating hydrogenated oils or giving consumers a choice, free HPV vaccines or none, more screening mammograms or fewer, etc. Or to phrase it in terms of the central question of pluralistic society, what should we do when there is disagreement about what constitutes the public interest?
A less fundamental but more immediate question is how can there be useful rules about disclosing conflicts of interest when there is disagreement about what constitutes a conflict of interest? My copy of Pearce's manuscript explicitly declares that he has no conflicts of interest. Yet his paper recounts fights that involved him personally, which surely must affect his motives. He offers thoughts about who should serve on advisory committees when his career has benefited from being chosen to serve on several such committees. These omissions probably reflect the widespread misperception that conflicts of interest result only from finance, which is only one of many possible influences facing a researcher, and usually not even the greatest.
Even what seem to be clear financial conflicts are often unacknowledged. Pearce's failure to note that he will profit from sales of the book he plugs in his second paragraph is of little consequence, since it goes without saying that self-promotion is part of the motive of most authors. But other common omissions likely affect the interpretation of research. For example, most of the articles that are highlighted as arguments against tobacco harm reduction were written by employees of politically active organizations that are committed to abstinence-only tobacco policies, but the authors never identify this as a conflict of interest. In several recent articles, such authors have included vehement anti-harm-reduction editorializing and policy conclusions (that in no way followed from their research results) without disclosing their employer's position. You can almost hear them saying, ‘but we are employed as activists on the right side of the issue, so we do not have a conflict! Only those on the other side have a conflict!’
Possibly worse than the challenge of competing visions about what is right for society is that people who actually secure power, for various reasons, come disproportionately from those whose primary goals do not include what is right for society. Some of the greatest thinkers in human history have struggled with these conundrums, which one summed up with the passage that begins, ‘But what is government itself, but the greatest of all reflections on human nature?’, and goes on to point out that both the governed and those granted the power of governing will be people, not angels.16
That author, James Madison, went on to design a pluralistic, multi-voiced, hypercritical process that addressed the conundrum in that passage. Those of us who study the history of the United States and the creation of modern governments-of-the-people—and who are watching the worst nightmares of America's Founding Fathers play out in our lifetimes—are very skeptical about trying to solve problems by giving the ‘right’ people the power to make sure things are done right. Remember that fascism was originally a utopian intellectual movement, backed by people like us, and we idealists who are tempted by its theoretical potential must remember what often happens when authority is proposed as a solution to pluralistic disagreements.
Less dramatically, it is well known that discussions of governance process are almost always stalking horses for discussions of policy. That is, particular individuals are always going to control those ‘neutral’ committees, write the codes of conduct, etc., and those individuals are going to have worldly agendas. My own story (and many others) show that anti-corporate activists can be every bit as narrow-minded, anti-scientific and anti-free-speech as the worst corporate actors. When those individuals influence epidemiological codes of conduct, serve on committees or write IARC or Surgeon General reports (and they do!), science suffers because the process is used to score points for one side of a worldly political battle, as would be the case if corporations had that control. Pearce and others who advocate vesting more power in committees and arbiters might find that within a few years whoever is ambitious enough to secure these positions does not support their vision of the public interest (inevitably some people will feel this way).
Of course, we ought to be able to do better than Madison's advice that ‘ambition must be made to counteract ambition’,16 which is necessary in areas that are entirely venal or about competing visions of the good. Epidemiology is supposed to be a science, largely based on a realist or positivist view that there are physical truths which it can reveal, regardless of their political implications.
|
The need for more scientific influence in our science |
|---|
|
TopRules and organizations will...The need for more...Can epidemiology ever be...References |
|---|
It might be too glib to say Pearce should consider himself lucky to have had his work critically read and analysed. After all, many epidemiological reports are never carefully read by anyone other than half of the authors (though the conclusions asserted in the abstract might still influence policy), and only a tiny minority are ever carefully analysed. But what Pearce portrays as persecution is what in most areas of science is called, well, science.
Pearce discusses how some industry-funded analysts are ‘hypercritical of others’ work’ as if scientists should not be hypercritical. He proposes an archetype of a critic who doubts the evidence that smoking causes lung cancer or that tampons cause toxic shock, implicitly equating scientific skepticism with anti-scientific nihilism. In fairness, this is probably not an intentional strawman, but rather results from the relative emptiness of the vast intellectual space between naive acceptance of epidemiological claims and active pursuit of flaws in particular analyses motivated by not liking the worldly implications. When there is little culture of critical analysis motivated by the concern that our results should be correct, it is easy to forget that the embarrassment for epidemiology is not that a few results are critically re-analysed, but that most results are not.
As I have noted elsewhere, epidemiology is characterized by the cranking out of thousands of new research reports per month, with little attempt at critical analysis.17 Standard practice involves ignoring well-known uncorrected errors18 and allows for substantial discretion in how results are analyzed and presented.19 A useful description of the methods and the data are usually unavailable to researchers who might want to conduct a critical analysis. In this environment, authors of all but a few major landmark papers can make claims with little concern that they will be challenged. Thus, authors with an unscientific agenda (whether it be corporate profits, supporting an activist position or just teasing out the bigger relative risk that gets a paper into a ‘better’ journal) have a fairly free hand to pursue it.
Pearce asks, ‘Why is epidemiological research so often full of controversy?’ Part of that causal pie is that it has substantial practical implications. But that is not a sufficient cause, since cellular biology, chemical engineering and informatics all have major practical implications, but findings in those fields are not assessed based on who paid for them. The other key slice of the pie is that if someone makes a claim that is nonsense in those other fields, then further research or careful analysis of what was published will demonstrate the error and the authors will probably pay a price. In epidemiology, researchers almost never suffer the deserved embarrassment for unsound analysis.
Some of us who attended graduate school in multi-disciplinary fields in the 1990s observed that the influence of politics-of-identity (often discussed under the rubric ‘diversity’) in a field was inversely related to how much the field was about physical reality. In computer science or geology, there was hardly a hint of politics-of-identity. In scientific fields with some worldly politics, like economics, it began to appear, and more so in departments of government or sociology. Continuing on that spectrum, in fields lacking empirical grounding, like critical literary studies, it seemed that politics-of-identity dominated the discourse.
The similarity to conflict-of-interest is striking. It is difficult to imagine a result in physics being dismissed or censored because the authors work for a weapons laboratory. Even studies in fields that generate public policy battles, like climate change or species loss, are seldom dismissed because of the politics of the authors. Why? Because these scientific claims transcend the preferences of the authors; data is available for critical analysis, methods are clearly described, errors and uncertainty are addressed in a useful manner, and if the result matters, critical analysis and replication (or failure to be able to replicate) is likely. It remains possible to skew conclusions a bit toward preferred results, but no matter how much one might advocate for the existence of cold fusion, when it does not work, it is pretty embarrassing to claim otherwise.
The implications of this represent a fundamental challenge for epidemiology: When research must stand up to scrutiny, authors have little room to indulge their political bias. But corporations, activists, governments and others exercise substantial influence over epidemiology because there is too little critical analysis and accountability to physical reality. Criticism based on identity is so common because critical science is so rare.
I had the good fortune to be mentored in epidemiology by George Maldonado, whose attitude toward the field was (my paraphrase) ‘wow, this is an incredibly complicated mess, and I cannot imagine how anyone can believe that they get correct answers with the standard analyses that ignore much of the complexity.’ And yet despite the complicated mess, epidemiological publications are usually presented with a sense of certainty and finality (notwithstanding the pro forma confessions that association is not causation, or recitations of ‘study limitations’ that effectively say something like, ‘this study probably suffered from massive uncontrolled confounding’). I find reading outside of health science to be refreshing because most authors give serious attention to potential errors, make confessions of uncertainty which they seem to take seriously, and present results of multiple analytic models. In contrast, it often seems that most researchers publishing in epidemiology (notably including the majority who have had no substantial training in epidemiology) have not thought about what they are doing hard enough to realize that it is an incredibly complicated mess. They learn what button to push to get an answer, and since it is the only answer that pops out when that button is pushed, it must be the answer.
Pearce quips that the job description of a consultant epidemiologist includes reciting ‘there are many "fatal flaws" in the study and it is therefore uninterpretable’. Perhaps he was suggesting such claims invoke flaws that do not exist, but perhaps this was a recognition that nearly every study in epidemiology has such substantial flaws that we should be unwilling to believe it provides much precision.
Critical analysis not only identifies errors, but inspires insight. Much of my best analysis has been in the context of energetic disagreements, including being contracted to provide critical re-analysis (a rare opportunity to spend sufficient time on one analysis and sometimes get subpoenaed data that would otherwise be kept secret) or responding to someone else's claims that we had done an incorrect analysis. An example of the latter is playing out in these pages20,21 though our next chapter has been delayed, in part because the first round of reviewer feedback consisted entirely of scolding us for concluding, based on a carefully articulated analysis, that our challengers employed faulty logic and calculations; the reviewer characterized this as childish bickering. This strikes me as a serious symptom: Perhaps if we claimed someone had a conflict of interest it would have been taken more seriously than was our mathematical analysis. Far too many in the field think that an ad hominem attack about someone's finances is legitimate scientific criticism, but actually criticizing the substance of published research is strange and rude.
Until there is widespread realization that some epidemiological methods, analyses and claims are scientifically wrong and teach an understanding of what makes them so, we will have no shortage of accusations about the influence of organized interests, and no shortage of actual influence by organized interests, and by researchers' own worldly preferences.
|
Can epidemiology ever be a science while in the shadow of public health? |
|---|
|
TopRules and organizations will...The need for more...Can epidemiology ever be...References |
|---|
In his second paragraph, Pearce implicitly measures his success in terms of the policy changes that his research brought about. Success in policy advocacy is certainly a legitimate way to assess one's work, but it is no measure of good science. Pearce had a political motive and was seeking to persuade people to act, just like those critics who he felt inappropriately attacked his research. Careful scrutiny and vigorous debate are good for science, but can interfere with the best laid plans of advocates.
Public health units in universities and government, to say nothing of many advocacy groups, are proverbial ‘liberal hotbeds’, usually acting as if their particular concerns trump any values and preferences that others have, and typically biased toward particular interventions. The engine that drives most population health research is this politicized environment, with its insular lack of self doubt, and the associated political goals are the greatest outside influence on the science. When anything that ‘reveals’ suspected hazards and justifies favoured interventions is ‘good science’, then the main threat to good science ceases to be bad scientific methods; instead the threat is considered to be something that is quintessential to real science, doubt. In this context, fixing epidemiology becomes a matter not of better methods, but of defending the preferred answers, using tactics that include minimizing the scientific contributions and influence of anyone who might disagree about what the preferred answers are.
Assessments of epidemiological methods, studies, conduct and influences that are motivated by public health activism—whether they are wholly tactical and cynical or are genuinely well-meaning—interfere, perhaps fatally, with improving the science of epidemiology. To reduce the anti-scientific influences of organized interests, epidemiologists need to focus on developing a better science, including developing methods to minimize political manipulation of results, demanding more complete publication of methods and data, and establishing a culture of quantifying errors and critical re-analysis. Only with an improved science that is not the tool of one group of organized interests will it be possible to establish a professional identity that defends the science and the scientists against manipulation and political threats from advocates of all stripes.
Conflict of interests
The article text includes references to some personal experiences and viewpoints that undoubtedly influence my writings on this subject (e.g. I have been attacked by anti-scientific interests; my reputation in the field is built on attempting to argue for greater critical analysis and a major overhaul of methods; my view of governance is influenced by my political views on American history and political science; I am extremely frustrated by the attitude toward ‘conflict of interest’ in epidemiology, which I believe reflects a combination of political manipulation and naivety; and I have engaged in the activity that Pearce criticizes but which I consider a very useful intellectual contribution). The identity of the various governmental, corporate, and other entities that have financed my work over the years is of relatively minor import. Anyone intent on understanding my interests by following the money—incidentally, a phrase that was popularized by a story about accumulation of governmental power22—would gain little insight. Indeed, the financial support for this article came from an organization (the University of Alberta School of Public Health) which has tried to censor my communication of some of what is included here, and would likely forbid me from publishing this if they could.
|
References |
|---|
|
TopRules and organizations will...The need for more...Can epidemiology ever be...References |
|---|
1 Pearce N. Corporate influences on epidemiology. Int J Epidemiol (2008) 37:46–53.
2 Phillips CV, Wang C, Guenzel B. You might as well smoke; the misleading and harmful public message about smokeless tobacco. BMC Public Health (2005) 5:31.[CrossRef][Medline]
3 Phillips CV, Rabiu D, Rodu B. Calculating the comparative mortality risk from smokeless tobacco versus smoking. Am J Epidemiol (2006) 163:S189.
4 Rodu B, Godshall WT. Tobacco harm reduction: an alternative cessation strategy for inveterate smokers. Harm Reduct J (2006) 3:37.[CrossRef][Medline]
5 Phillips CV. TobaccoHarmReduction.org. [Online]. (2006) [cited 6 August 2007]; Available from URL: http://tobaccoharmreduction.org/index.htm.
6 Kozlowski LT, Edwards BQ. "Not safe" is not enough: smokers have a right to know more than there is no safe tobacco product. Tob Control (2005) 14(Suppl 2):ii3–7.
7 Kozlowski LT, O’Connor RJ. Apply federal research rules on deception to misleading health information: an example on smokeless tobacco and cigarettes. Public Health Rep (2003) 118:187–92.
8 Phillips CV. Warning: anti-tobacco advocacy can be hazardous to epidemiology. Epidemiol Perspect Innov (2007) 4:13.[CrossRef][Medline]
9 Enstrom J. Defending legitimate epidemiologic research. Epidemiol Perspect Innov (2007) 4:11.[CrossRef][Medline]
10 Enstrom J. Defending legitimte epidemiologic research. [Online]. (2006) [cited 6 August 2007]; Available from URL: http://www.scientificintegrityinstitute.org/defense.html.
11 McNeil D. India, said to play down AIDS, has many fewer with virus than thought, study finds. New York Times. 8 June 2007; Available from URL: http://www.nytimes.com/2007/06/08/world/asia/08aids.html?ex=1186286400&en=19211028adbe6f9e&ei=5070.
12 United States House of Representatives Committee on Government Reform – Minority Staff Special Investigations Division. Politics and science in the bush administration. (2003) Available from URL: http://oversight.house.gov/features/politics_and_science/pdfs/pdf_politics_and_science_rep.pdf.
13 Wilson J. Bureaucracy:. In: What Government Agencies Do and Why They Do It. (1989) New York: Basic Books.
14 Wilson J. The Politics of Regulation. (1980) New York: Basic Books.
15 Vernon R, Spar D, Tobin G. Iron Triangles and Revolving Doors: Cases in U.S. Foreign Economic Policymaking. (1991) New York: Praeger.
16 Madison J. The structure of the government must furnish the proper checks and balances between the different departments. In: Independent Journal. (1788) (The Federalist No. 51). Available from URL: http://www.constitution.org/fed/federa51.htm.
17 Phillips CV, Goodman KJ, Poole C. Lead editorial: the need for greater perspective and innovation in epidemiology. Epidemiol Perspect Innov (2004) 1:1.[CrossRef][Medline]
18 Phillips CV. Quantifying and reporting uncertainty from systematic errors. Epidemiology (2003) 14:459–66.[Web of Science][Medline]
19 Phillips CV. Publication bias in situ. BMC Med Res Methodol (2004) 4:20.[CrossRef][Medline]
20 Goodman KJ, O’rourke K, Day RS, et al. Dynamics of Helicobacter pylori infection in a US-Mexico cohort during the first two years of life. Int J Epidemiol (2005) 34:1348–55.
21 Perry S, Parsonnet J. Commentary: H. pylori infection in early life and the problem of imperfect tests. Int J Epidemiol (2005) 34:1356–58.
22 Pakula A. All the President's Men. (1976) Warner Brothers.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  G. Davey Smith Big business, big science? Int. J. Epidemiol., February 1, 2008; 37(1): 1 - 3. [Full Text] [PDF]
|
|
|
|
|
|
N. Pearce Response: The distribution and determinants of epidemiologic research Int. J. Epidemiol., February 1, 2008; 37(1): 65 - 68. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||