IJE Advance Access originally published online on September 19, 2006
International Journal of Epidemiology 2006 35(5):1130-1135; doi:10.1093/ije/dyl196
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Commentary |
Commentary: The ‘bibliographic impact factor’ and the still uncharted sociology of epidemiology
1 Institut Municipal d'Investigació Mèdica, Barcelona, Spain
2 School of Medicine, Universitat Autònoma de Barcelona, Spain
3 School of Public Health, University of North Carolina at Chapel Hill, USA
4 Institut Català d'Oncologia, L'Hospitalet de Llobregat, Barcelona, Spain
5 Universitat Pompeu Fabra, Catalonia, Spain
6 School of Medicine, Universidad de Alcalá, Madrid, Spain
* Corresponding author. School of Medicine, Universitat Autònoma de Barcelona, Carrer del Dr Aiguader 80, E-08003 Barcelona, Catalonia, Spain. E-mail: mporta{at}imim.es
It is 1955, a time of ‘mechanical devices’ and ‘punched cards’,1 before microcomputers, of course, before the ‘impact factor’. 1955: Bill Gates and Steve Jobs are born (Box 1). It is only 50 years ago: Eugene Garfield first proposes a bibliographic ‘indexing’ or ‘citation’ system for scientific literature.1 His paper advocates a new citation index—as opposed to traditional subject indexing—based on a clever and innovative concept—today not much en vogue—: the association-of-ideas. An association-of-ideas index. Of course, ‘nothing could substitute for extensive reading, but . . . ’ (page 31).
| Box 1 The ‘calendar year’ of 1955, some events Pentagon announces plan to develop intercontinental ballistic missiles armed with nuclear weapons United States President Dwight D. Eisenhower sends the first advisors to South Vietnam Winston Churchill resigns as Prime Minister of the United Kingdom The Salk polio vaccine is introduced McDonald's first franchised fast food restaurant opens Imre Nagy, Premier of Hungary, is forced to resign and expelled from the Communist Party by hardline comrades West Germany becomes a sovereign state and joins NATO The Warsaw Pact is formed by the communist states of Eastern Europe and the USSR Disneyland opens The first atomic-generated electrical power is sold commercially In Algeria the National Liberation Front (FLN) continues the guerrilla war of independence against the French armed forces Vladimir Nabokov's Lolita is published in Paris by Olympia Press President of Argentina Juan Peron is ousted in a military coup In the US racial segregation is forbidden on trains and buses in interstate commerce. Rosa Parks (a Montgomery, Alabama seamstress), refuses to give up her bus seat to a white man and is arrested. Martin Luther King, Jr and other ministers coordinate a Black boycott of city buses General Motors becomes the first American corporation to make over $1 billion in a year 1955 is the year of birth of: Bill Gates (cofounder of Microsoft) and Steve Jobs (CEO of Apple Computer), Isabelle Adjani, Whoopi Goldberg, Kevin Costner and Bruce Willis (actors), Simon Rattle (conductor), Greg Norman (golfer), Alain Prost (race car driver), Yo-Yo Ma (cellist) . . . 1955 is also the year of death of: Albert Einstein and Alexander Fleming (scientists), Charlie Parker (saxophonist), Alfred Radcliffe-Brown (anthropologist), Thomas Mann (writer), Carlos Gardel (tango singer), James Dean (actor) . . . Source: Wikipedia. http://en.wikipedia.org/wiki/1955 In epidemiology, the tobacco - lung cancer controversy is raging. Berkson has one more critical paper in the Mayo Clinic Proceedings; Wynder edits a book trying to build a consensus with contributions from many epidemiologists; Doll publishes his groundbreaking work on the relation of asbestos to lung cancer (Br J Ind Med); the previous year, Doll and Hill have published a new article on mortality of doctors in relation to smoking (BMJ), and Armitage and Doll theirs on the multi-stage theory of carcinogenesis (Br J Cancer). Times these 4 papers have been cited: 121 (Berkson), 466 (Doll), 212* (Doll & Hill), and 621 (Armitage & Doll). Do we care much? *The 212 citations in the ISI-Thomson database exist in spite of a typical mistake by ISI-Thomson: the database contains an error in the primary reference of the article (BMJ 1954; 1 (4877): 1451-1455): in the database the first page is wrong. The article may hence have "lost" (in the database) some of the citations it actually received. Furthermore, the "Times cited" figure for the article in ISI's "Web of science" "Full record" of the "General search" is said to be zero (i.e., never cited). Several other papers by Doll & Hill on the same study have each been cited over 400 times, even more than one thousand times. So much for rankings of highly cited articles: meaningless without knowledge on context.
|
| Box 2 Main reasons why the ‘bibliographic impact factor’ is often not the scientometric indicator of choice or Why the ‘bibliographic impact factor’ is usually a much poorer indicator than the total number of citations or Why is it sort of unbelievable that we care about the ‘impact factor’
We would be lying if we said that our journal's impending first impact factor is not of interest to us. For a number that is so widely used and abused, it is surprising how few people understand how a journal's impact factor is calculated, and just how limited it is a means of assessing the true impact of an individual publication in that journal. A journal's impact factor cannot tell us anything about the quality of any specific research article in that journal, nor of the quality of the work of any specific author. It is well known that editors at many journals plan and implement strategies to massage their impact factors. Editors may decrease the denominator [of the BIF] by attempting to have whole article types removed from it. The rules of the game are unclear—editors can, for example, try to persuade Thomson Scientific to reduce the denominator, but the company refuses to make public its process for choosing ‘citable’ article types. During discussions with Thomson Scientific over which article types in PLoS Medicine the company deems as ‘citable,’ it became clear that the process of determining a journal's impact factor is unscientific and arbitrary. After one in-person meeting, a telephone conversation, and a flurry of e-mail exchanges, we came to realize that Thomson Scientific has no explicit process for deciding which articles other than original research articles it deems as citable. We conclude that science is currently rated by a process that is itself unscientific, subjective, and secretive During the course of our discussions with Thompson Scientific, PLoS Medicine‘s potential impact factor—based on the same articles published in the same year—seesawed between as much as 11 (when only research articles are entered into the denominator) to less than 3 (when almost all article types in the magazine section are included, as Thomson Scientific had initially done—wrongly, we argued). At the time of writing this editorial, we do not know exactly where our 2005 impact factor has settled. But whatever it turns out to be, we feel the time has come for the process of ’deciding' a journal's impact factor to be debated openly.31 Finally, PLoS Medicine's BIF was 8.389.
|
|
|
The core of his starting point is not that new (centuries old, actually): the ideas expressed by authors in articles (‘the micro unit of thought’, page 11) can be found and followed-up in the citations to those articles. Every time an author cites a previous article (i.e. the ideas within the article), he or she is indexing it from his/her experience. This emphasis on the ideas, uhm . . .
Among the fascinating projects that Eugene Garfield has been able to make real, to us this is the one that nowadays remains most valuable—immensely useful and influential in this era of information contamination: the ease with which we can conduct orderly, structured, thoughtful, insightful (and computerized) retrievals of papers, that is, associations of ideas, relationships among findings, among hypotheses . . . just by clicking at ‘Cited References’, ‘Times Cited’ . . . and thinking.
Useful and influential to the point that the impacts of Garfield's Science Citation Index are immeasurable and invaluable. At least the most important impacts: upon ‘thought processes’ and the generation of new ideas, upon ‘critical appraisal and interpretation’ (page 3),1 to ‘eliminate the uncritical citation of fraudulent, incomplete or obsolete data’ (opening sentence, page 1).1 The genuine, true impact upon hundreds of thousands of scientists who have used Garfield's tools ‘to evaluate the significance of a particular work and its impact on the literature and thinking of the period’ (page 3)1.
In his article1 Garfield shows an amazing vision, imagination, creativity, and ambition. And he anticipates that the new system ‘would provide a complete listing, for the publications covered, of all the original articles that had referred to the article in question’. Moreover, this article contains the first seed—if not formulation—of the idea of an ‘impacfot factor’ (page 2),1 as acknowledged in the opening sentence of a recent article in JAMA: ‘I first mentioned the idea of an impact factor in Science in 1955.’2 Today there can be no doubt about the influence not exactly of the paper itself but of the work that put the paper ideas in motion: influence on the ways researchers search, retrieve, and use the scientific literature, certainly, but also on how and where scientists publish the results of investigations, on how—and how much—research is funded, academic positions assigned...
‘Citation Indexes for Science’1 is just one of the early papers by Eugene Garfield. His vast production spans some 1524 articles. Although most of them are available from his own website,3 let us cite some favourites,4–16 since someone is surely counting citations to his papers. Surely, as we all know only too well, citations are often redundant, superfluous, vehement, self-serving. Eugene Garfield knows that we have long acknowledged our intellectual respect and debt to him17–23—but we know for sure that someone is counting our citations too. More than incidentally, today the same technologies that make it possible for us to enjoy the results of Garfield's dreams are also providing new tools for scientists (this is a reality). Information technologies are also promising new ways to assess the influence of scientific work (this being yet rather imperfect). And new tools for the sociology of science,24–26 including a still uncharted sociology of epidemiology.13–15,17,27–30
We could indeed follow the path walked by the idea of the ‘citation index’ using the Science Citation Index itself: the article1 has been cited in—or ‘by’—248 articles, mostly in the field of information science and library sciences; e.g. in 28 articles published in Current Contents and in 12 articles published in—or ‘by’—Scientometrics. The article has seldom been cited (just 2 cites) in journals of epidemiology and public health. Remarkably, some 200 different researchers have cited it. Almost a quarter of the citations to the paper come from the author himself. Also, the paper is presently cited more than ever before: it was cited in 96 articles (38.7% of the total) in the period 2000–2006 and in 50 papers (20.2%) in 1990–1999. Is it that the article has been rediscovered in recent times? Does the citation trend reflect an increase in the interest on the ‘impact factor’? Why would that be? (Box 2).
Presumably to acknowledge that an article may have a wide spectrum of impacts,20–23,31 the revered ‘impact factor’ 32–34 has recently begun to be referred to more precisely as ‘journal impact factor (JIF)’ by Garfield himself,2 while it has been named as ‘bibliographic impact factor (BIF)’ by others for rather long. For reasons that few of us would pretend to comprehend,22 the JIF/BIF has become not only the more widely used and misused scientometric indicator but, surprisingly—given the alternatives—, the most popular indicator of the ISI—Thomson galaxy (http://www.isiwebofknowledge.com and http://scientific.thomson.com/knowledgelink/). Surprisingly?
 |
Acknowledgments |
|---|
Our units are supported in part by grants from ‘Red temática de investigación cooperativa de centros en Cáncer’ (C03/10), by ‘Red temática de investigación cooperativa de centros en Epidemiología y salud pública’ (C03/09), and by CIBER de Epidemiología, Instituto de Salud Carlos III, Ministry of Health, Spain. We thank Alfredo Morabia for historical advice.
|
References |
|---|
 Top References |
|---|
1 Garfield E. Citation indexes for science. A new dimension in documentation through association of ideas. Science 1955;122:108–11. (Reprinted Int J Epidemiol 2006;35:1123-27.)
2 Garfield E. The history and meaning of the journal impact factor. JAMA 2006;295:90–93.
3 Garfield E. Essays of an information scientist. Available at: www.garfield.library.upenn.edu (Accessed July 28, 2006).
4 Garfield E. Citation analysis as a tool in journal evaluation. Science 1972;178:471–79.
5 Garfield E. Significant journals of science. Nature 1976;264:609–15.[CrossRef][Medline]
6 Garfield E. Caution urged in use of citation analyses. Trends Biochem Sci 1977;2:84.[CrossRef]
7 Garfield E. Is citation analysis a legitimate evaluation tool? Scientometrics 1979;1:359–75.[CrossRef]
8 Garfield E. How to use citation analysis for faculty evaluations and when is it relevant? Part 1. Curr Contents Life Sci 1983;44:5–13.
9 Garfield E. How to use citation analysis for faculty evaluations and when is it relevant? Part 2. Curr Contents Life Sci 1983;45:5–14.
10 Garfield E. Uses and misuses of citation frequency. Curr Contents 1985;43:3–9.
11 Garfield E. Which medical journals have the greatest impact? Ann Intern Med 1986;10:313–20.
12 Garfield E. The epidemiology of knowledge and the spread of scientific information. Curr Contents 1980;35:5–10.
13 Garfield E. The 101 most-cited papers from the British Medical Journal highlight the important role of epidemiology in medicine. Curr Contents 1987;7:3–11.
14 Garfield E. Mapping the world of epidemiology. Part 1. The disease detectives. Curr Contents 1988;35:3–11.
15 Garfield E. Mapping the world of epidemiology. Part 2. The techniques of tracking down disease. Curr Contents 1988;35:3–8.
16 Garfield E. Prestige versus impact: established images of journals, like institutions, are resistant to change. Curr Contents Life Sci 1987;38:3–4.
17 Porta M, Fernandez E, Puigdomènech E. Citations received by books: an approach to the influence of epidemiologic thought within the academic communities? Rev Saúde Pública 2006;40 (Suppl):50–56.
18 Porta M, Fernandez E, Murillo J, Alguacil J, Copete JL. The bibliographic ‘impact factor’, the total number of citations and related bibliometric indicators: The need to focus on journals of public health and preventive medicine. Sozial- und Präventivmedizin 2004;49:15–18.
19 Porta M. Quality matters and the choice of indicator matters too. BMJ 2003;326:931.
20 Fernandez E, Plasència A. Tell me how often we are cited and I will tell you . . . the bibliographic impact factor of Gaceta Sanitaria. Gac Sanit 2003;17:179–80.[Medline]
21 Porta M, Copete JL, Fernandez E, Alguacil J, Murillo J. Mixing citations to journals, articles and authors, and other catches in the bibliographic ‘impact factor’. Cadernos Saúde Pública 2003;19:1847–62.
22 Porta M. The bibliographic ‘impact factor’ of ISI, Inc.: how relevant is it really for public health journals? J. Epidemiol Commun Health 1996;50:606–10.[ISI][Medline]
23 Porta M. Factor de impacto bibliográfico (Science Citation Index y Social Sciences Citation Index) de las principales revistas de medicina preventiva, salud pública y biomedicina. Algunas cifras, algunas impresiones. Revisiones en Salud Pública 1993;3:313–47.
24 de Almeida-Filho N. La ciencia tímida. Ensayos de deconstrucción de la epidemiología. Buenos Aires: Lugar, 2000.
25 Morabia A, editor. A history of epidemiologic methods and concepts. Basel: Birkhäuser/Springer, 2004. Available at: http://www.epidemiology.ch/history (accessed july 28, 2006).
26 Amsterdamska O. Demarcating epidemiology. Sci Technol Human Values 2005;30:17–51.[Abstract]
27 Porta M, Alvarez-Dardet C. Epidemiology: bridges over (and across) roaring levels. J Epidemiol Commun Health 1998;52:605.[ISI][Medline]
28 Bolúmar F, Porta M. Epidemiologic methods: beyond clinical medicine, beyond epidemiology. Eur J Epidemiol 2004;19:733–35.[CrossRef][ISI][Medline]
29 Saracci R. Introducing the history of epidemiology. In: Olsen J, Saracci R, Trichopoulos D (eds). Teaching Epidemiology. 2nd edn. Oxford: Oxford University Press, 2001. pp. 1–19.
30 Porta M. Things that kept coming to mind while thinking through Susser's South African mémoir. J Epidemiol Commun Health 2006;60:559–61.
31 PLoS Medicine editors. The impact factor game (editorial). PLoS Med 2006;3:e291, 1–2.
32 Seglen PO. How representative is the journal impact factor? Res Eval 1992;2:143–49.
33 Seglen PO. Citation frequency and journal impact: Valid indicators of scientific quality? Eur J Epidemiol 1991;229:109–11.
34 Joseph KS. Quality of impact factors of general medical journals. BMJ 2003;326:283.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  M Porta Publishing has become a "Russian roulette" J. Epidemiol. Community Health, August 1, 2007; 61(8): 722 - 722. [Full Text] [PDF]
|
|
|
|
 |
|
 S. EBRAHIM Entelechy, citation indexes, and the association of ideas Int. J. Epidemiol., October 1, 2006; 35(5): 1117 - 1118. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||