IQ - Journal Impact Factor (Dirk Lewandowski)

Journal Impact Factor
Dirk Lewandowski © October 2006
 

The Journal Impact Factor (or simply Impact Factor) describes how much of an impression a scientific journal makes. This is done by measuring how often articles from the journal in question are cited in other scientific journals.

The Impact Factor consequently provides information on how well articles in the journal are received by other researchers who themselves publish. Journals that tend to be read by researchers working in industry or by practitioners (i.e. who take note of them, but themselves publish to a much lesser extent) consequently generally have a lower Impact Factor. However, this does not mean that publishing articles in journals like these as well is necessarily disadvantageous for researchers.

 
1. Data basis

The Journal Impact Factor is calculated on the basis of the journals contained in the Web of Science (a commercial product marketed by Thomson Scientific) and is annually published in the Journal Citation Reports. These consider all citations from journals that are themselves listed in the Web of Science. Generally, these are journals with international scope that have been selected on the basis of certain criteria and only publish articles that have already been peer reviewed.

Although the data basis for the Journal Citation Reports is large with more than 7,500 journals, the depth of coverage across various subjects differs quite substantially. For example, the natural sciences and medicine are covered very extensively, while the social sciences, arts and humanities only have a low degree of coverage. This is additionally affected by the fact that these disciplines also publish extensively outside the scope of journals (e.g. in monographs, proceedings and collections), while the Web of Science exclusively catalogues articles published, above all, in journals with international scope.

 
2. Calculation basis

The Impact Factor is calculated on the basis of the articles published over the past two years. So if a journal's Impact Factor is to be calculated for 2005, this is done by considering the articles published in 2003 and 2004.

The second value included in the calculation is the number of times an article from an indexed journal has been cited over the past two years. To calculate the Impact Factor, the number of citations is divided by the number of source articles.

So the formula for calculating the Impact Factor is:

 
C
IF (Z) = 
 
S1+S2

whereby J is the indexed journal, C = the number of citations received in the current year for the articles of the previous two years, and S1 and S2are the number of source articles published over the two previous years.

Two examples may serve to illustrate the calculation:

  • Nature published 859 articles in 2003 and 878 in 2004, producing 1,737 articles in total. In the same two years, the journal was cited 50,848 times. Dividing the number of citations by the number of articles produces a Journal Impact Factor of 29,273.
  • The American Journal of Bioethics published a total of 53 articles in 2003 and 2004; in the same period, the journal scored 133 citations. This produced an Impact Factor of 2,509 for that journal.

These two examples alone show how different the Impact Factor can be, even if - as in this case - they are both leading journals in their respective areas. The great difference above all results from the size of a subject category and the differing citation habits practised there. This is why, in terms of Impact Factor, journals may only be compared with other journals from their own subject category, but not (as would be the case for Nature and American Journal of Bioethics) in a cross-disciplinary comparison.

 
3. Methodological distortions

When comparing journals within a subject category, it is still necessary to proceed with caution. Besides research articles, as such, scientific journals also publish reviews and other more secondary materials (editorials and news articles, proceedings, technical notes, letters). In principle, review articles are cited more often than original articles, although the latter are generally considered of higher value, because they give first-hand reports on research done. The Impact Factor prefers journals that publish a high proportion of review articles.

Another problem of the various document types lies in their differing treatment in Journal Citation Reports. Only original articles, review articles, proceedings, and technical notes are seen as source articles. However, citations are also taken from other types of articles. In effect, this means that items such as letters are cited, but not as source articles. This in turn increases the Impact Factor of journals that publish a high number of "other" document categories.

The third problem area involves the two-year observation period on which the Impact Factor is based. This relatively short time span for scientific communication gives preference to current journals. The varying "maturation processes" for scientific information remain unconsidered

Fig. 1 illustrates how the distribution of citations differs at the level of journal categories. While rheumatology largely cites recent literature, citations in the field of mathematics are more evenly spread across the volumes. The Impact Factor calculation, however, excludes all citations referring to articles published more than two years before the considered year. Journals (and subject categories) that publish articles quickly are therefore at an advantage when calculating the Impact Factor

 

Source: ISI Journal Citation Reports, own calculation

 

4. What can the Impact Factor tell us about individual articles or researchers?

Since the Impact Factor describes a journal's bearing, it is also often used when deciding which journal to publish in and when evaluating the publication performance of individual scientists or research groups.

Publication in a journal with a high Impact Factor does not, however, guarantee frequent citations. Each journal contains articles that are not or only rarely cited. In fact, this also happens in frequently-cited journals like Science and Nature. Even considering original articles only, Nature 2004 (Vols. 432 to 427) contains 11 articles that have not yet been cited even once; around one half of all articles achieve a citation score below the journal's Impact Factor. This means that the Impact Factor is unable to say anything about individual articles; all it can do is make a collective statement about all the articles in a journal.

In medicine, in particular, the evaluation of individual researchers is based on Impact Factors, for example, when assessing a candidate's publication performance in a postdoctoral qualification (habilitation) procedure. In such cases, the Impact Factors of the journals in which the scientist has published are added together or are calculated by means of a weighting system. This is problematical for the above-mentioned reasons, and has also been also rejected by the Association of the Scientific Medical Societies in Germany (Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften - AWMF). Impact Factors "are unsuitable for evaluating individuals when deciding on habilitations and professorships."*

When judging the publication performance of individual researchers on the basis of citations, only the citations actually achieved should be counted. Weighting systems should be used for more recent articles that have not yet been able to score an appropriate number of citations.

 
5. Conclusion

Journal Impact Factors are a good method for assessing the short-term scientific influence of journals. Journals with similar profiles within a subject category can be compared on the basis of this value (possibly complemented by other values, such as the Immediacy Index and the Journal Cited Half-Life). However, Journal Impact Factors are not suitable for evaluating the publication performance of researchers.

 
References

Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften (AWMF), 2000:
AWMF-Vorschlag zur Verwendung des "Impact Factor". http://www.uni-duesseldorf.de/AWMF/bimet/impa-emp.htm [Retrieved 31.10.2006]
Brähler, Elmar, Oliver Decker and Manfred E. Beutel, 2004:
Deep Impact. Evaluation in the Sciences. Sozial- und Präventivmedizin 49(2004)1: 10-14. http://www.uni-leipzig.de/~decker/spm.pdf [Retrieved 31.10.2006]
Deutsche Forschungsgemeinschaft (DFG), 2004:
Empfehlungen zu einer »Leistungsorientierten Mittelvergabe an den Medizinischen Fakultäten. Stellungnahme der Senatskommission für Klinische Forschung der Deutschen Forschungsgemeinschaft. http://www.dfg.de/aktuelles_presse/reden_stellungnahmen/2004/download/stellungnahme_klinische_forschung_04.pdf [Retrieved 31.10.2006]
Garfield, Eugene, 1979:
Citation Indexing. Its Theory and Application in Science, Technology and Humanities. New York: John Wiley & Sons.
Garfield, Eugene, 2005:
The Agony and the Ecstasy. The History and Meaning of the Journal Impact Factor. http://garfield.library.upenn.edu/papers/jifchicago2005.pdf [Retrieved 31.10.2006]
Moed, Henk F., 2005:
Citation Analysis in Research Evaluation. Dordrecht: Springer. [Kapitel 5: Citation Analysis of Scientific Journals]

* "Für eine Bewertung von Einzelpersonen bei Habilitationen und Berufungen sind sie [die Impact Factors] nicht geeignet."
The Journal Immediacy Index states how often on average articles in a journal are cited in the same year as their publication.
The Journal Cited Half-Life value stands for the median age of articles cited from a journal in the current year, i.e. half of all citations come from articles that were published within the cited half-life.