OBSERVARE

Universidade Autónoma de Lisboa

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

THE IMPACT FACTOR AS A LEGITIMATOR OF THE SCIENTIFIC KNOWLEDGE

PRODUCED: A REVIEW OF THE LITERATURE

Tiago Lima Quintanilha

Tiago.lima@obercom.pt

PhD student in Communication Sciences at ISCTE–IUL (Portugal) and doctoral scholar of the

Foundation for Science and Technology (FCT). Quintanilha works in the area of Social Sciences,

with emphasis on communication sciences, open science and journalism studies. He was the

manager of an international scientific journal, from 2009 to 2016. He attends the third year of

the PhD in Communication Sciences at ISCTE-IUL. He received two prizes of academic merit and

is the author and co-author of three books, 12 book chapters, ten articles in international and

national academic journals, and more than 60 research reports in the area of media and

communication. He also collaborated with the Observatory of Communication, the SOCIUS,

Center for Research in Economic and Organisational Sociology at ISEG-UL, the Regulatory Entity

for Social Communication and the Reuters Institute for the Study of Journalism with the project

Digital News Report Portugal (2015 & 2016).

Gustavo Cardoso

Gustavo.cardoso@iscte-iul.pt

Cardoso is a full Professor at ISCTE – Instituto Universitário de Lisboa (Portugal), researcher

integrated in the CIES-IUL, associate researcher in Le collège d'etudes mondiales and editor of

the journal OBS*Observatory.

Abstract

The impact factor (IF) is perceived by all researchers and scientists as the main criterion in

their publication strategies, in order to respond to the demands of the academic reward

system that defines career progression and the allocation of funds for research. Through a

review of published literature we aim to explore different limits and ranges associated with

impact factor, according to academia. We also try to understand in which way the state of the

art is evolving, as well as the features that point to ambiguous criteria inherent to the

construct on which impact factor was established as the legitimising criteria of scientific

knowledge production.

Keywords

Impact factor; legitimacy; scientific knowledge production; citation metrics

How to cite this article

Quintanilha, Tiago Lima; Cardoso, Gustavo (2018). "The impact factor as a legitimator of the

scientific knowledge produced: a review of the literature". JANUS.NET e-journal of

International Relations, Vol. 9, N.º 2, November 2018-April 2019. Consulted [online] on the

date of the last visit, https://doi.org/10.26619/1647-7251.9.2.3

Article received on February 2, 2018 and accepted for publication on October 13, 2018

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

THE IMPACT FACTOR AS A LEGITIMATOR OF THE SCIENTIFIC KNOWLEDGE

PRODUCED: A REVIEW OF THE LITERATURE

Tiago Lima Quintanilha

Gustavo Cardoso

The quest to quantify everything undermines higher education

(Muller, 2018)

Introduction

At a time when the main challenges to the Open Science model (Priem, 2012; Bare,

2014; Quintanilha, 2015; Berghmans, 2015) are being discussed, or the derivation of a

phenomenon of scientific res publica (Cardoso et al, 2009) for skewing logics of the Open

Access movement from its appropriation by the parallel and parasitic industry

(Quintanilha, 2015), it is important to reflect on one of the crucial points that are part of

this derivation.

Within a framework of “certification independent of the means of registration” (Borges,

2006: 72), where belonging to the main indexers (Thomson Reuters, Scopus) relates to

the most important criteria for accreditation of scientific journals, impact factor (IF)

(Garfield, 1955; Borges, 2006; Johnstone, 2007; Saarela, 2016; Seglen, 1997;

Greenwood, 2007) – appropriately named as factor of influence by Sygocki and

Korzeniewska (2018) – is assumed as the great criterion defining the panorama of the

worldwide scientific publication. Other authors, such as Muller (2018), call it the tyranny

of metrics, whereas Garfield (2006), five decades after the concept was created, uses

the neologisms “scienciometry” and “journalology” to define the time. Impact factor is a

sort of filter that separates almost indiscriminately the relevant knowledge produced from

the rest. Thus, it constitutes itself as the great and current legitimating factor of the

scientific knowledge produced at the same time and, directly or indirectly, contributes to

the imprisonment of the Open Science model. Thomson Reuters/Web of Science/Clarivate

Analytics – through its journal citation reports – and Scopus Elsevier – through its SJR

(SCImago Journal Rank) and SNIP (Source Normalised per Paper) – control the great

metrics associated with the publication of scientific knowledge. Due to the way they were

established, these metrics end up contributing to factors of higher impact and influence

The translation of this article was funded by national funds through FCT - Fundação para a Ciência e

a Tecnologia - as part of OBSERVARE project with the reference UID/CPO/04155/2013, with the aim of

publishing Janus.net. Text translated by Thomas Rickard.

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

in scientific journals which are mostly of Anglophone origin and controlled by the major

publishers, such as Sage, Wiley, Taylor & Francis, Routledge, etc. These publishers

impose pay-per-query dynamics and help to pervert and strongly compromise the idea

of democratising a more inclusive Open Science model, forcing an almost tacit acceptance

of legitimating structures in a large number of highly requested journals which antagonise

the Open Science model due to their close access and how they operate. An article in

Science Magazine

considers these as exorbitant amounts applied by the major

publishers, which own the scientific journals with the greatest impact factor, a sort of

disruptive body to the continuity of the Open Access model in the sense that they create

tremendous pressure on the future of academic publishing.

Impact factor, depending to a large extent on the number of citations generated by an

article or publication, is determined beforehand by the hegemonic position of the

Anglosphere and the holders of greater symbolic capital in the academia – an adaptation

of Bourdieu's (1994) concept. In other words, an article published in English will have,

for example, a greater possibility of being cited, in a context of cumulative scientific

construction (Quintanilha, 2015), when linking texts with other texts, blocks of

signification and units of reading or lexia (Barthes, 1972).

Similarly, Saarela (2016: 699) noted that the top journals or core journals of large

disciplines typically have more citations, embodied in larger impact factors, compared to

the ones of smaller disciplines (Saarela, 2016). Whilst Howard (2009) calls them the “A-

list” of academic publication, Adler, Ewing and Taylor (2009) remind us that in some

scientific fields, such as biomedical sciences, most articles are often cited shortly after

their publication; and in other disciplines, such as math, most citations begin to emerge

two years after the articles are published. Some institutions, such as the University of

Western Australia

, warn of the fact that the metrics could be beneficial for validating

knowledge produced by journals in natural sciences, medicine and social sciences, and

totally irrelevant in the arts and humanities.

On the other hand, entry in this reading system of the impact factors is only possible,

therefore, by the aggregation of the journals to the main global indexes, some of them

controlled by the biggest publishers in the world – such as the case of Scopus, owned by

Elsevier. Thus, the process that defines the impact factor of a publication depends, in the

first instance, on the entry and association of it with the two main world indexes

(Thomson Reuters’ Web of Science and Elsevier’s Scopus). The most evident sign of this

bias associated with the organisations that have the main impact factors (i.e. Reuters –

Web of Science/Clarity Analytics – Journal Citation Reports) is given when the journals

themselves apply to the main indexers. One of the great criteria used by the Web of

Science, for example, in accepting journals is the publication of articles exclusively in

English, to the detriment of the quality that may be associated with articles published by

the proponent journal. The reason seems obvious: articles exclusively published in

English have a superior scope and potential for citation than an article published, for

example in Portuguese, and are restricted to the speakers of this language.

This controversial process, which regulates the panorama of the world scientific

http://www.sciencemag.org/news/2017/08/bold-open-access-push-germany-could-change-future-

academic-publishing

https://guides.library.uwa.edu.au/c.php?g=325233&p=2177836

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

publication, inaugurates a certain institutionalised way of looking at a scientific journal

based on its legitimation or delegitimation according to the impact factor attributed to it.

Therefore, it is important to reflect on the construction of the concept itself, observing

the criteria for the evolution and redefinition of the dimensions that have been on the

basis of the construction of this concept in systems of academic reward, which

significantly contributes to access to entry exams and progression in teaching and

research careers. It is also fundamental to think about how these criteria have been

decisive for the granting of postdoc fellowships and, to a certain extent, for PhD

fellowships as well as for the financing of research projects based on the merit of the

candidates and their research centres to which they are associated. Currently, impact

factors have increasingly played a crucial role in the allocation of resources for scientific

research purposes (Saarela, 2016) in many countries and constitute the primary criterion

for academics’ prestige and survival (Pirmez, Brandão & Momen, 2016: 543).

Framework of the concept

Maria Manuela Borges (2006) introduces the historical roots of the term “impact factor”,

first used by Eugene Garfield (1955). This quantitative concept “seeks to express the

intellectual influence or the contribution of a work in a given area of knowledge”, and

“other functions can be added to these ones, such as the increasing degree of

communication between authors or bringing up for them all the references related to

their work (Borges, 2006: 55).

Others, such as Seglen (1997), use the term “citation rate” as a complement to the term

impact factor in order to define the average of citations contained in a given journal and

in a certain period of time. Impact factor is usually defined by calculating the number of

citations in a given year in relation to the number of items published in that year (Seglen,

1997). Following the same reasoning, Greenwood (2007) states that “impact factor is

thus calculated as the number of citations a journal has received in the last complete

year for articles published in the two preceding years, divided by the total number of

articles the journal published in the two preceding years”. According to the author, this

calculation expresses the average number of citations of published articles, without

discriminating positively or negatively the larger or more frequently published journals

(Greenwood, 2007: 1).

Traditionally, impact factor was used to determine the most important sources of

knowledge to be acquired by universities (Saarela, 2016). Currently, impact factors are

carefully consulted by researchers from around the world, who thus define the scientific

journals to which they submit their articles (Greenwood, 2007). Johnstone (2007) argues

that impact factor is being used as an absolute measure to define the quality of scientific

journals. In addition, she points out that impact factor is also increasingly used as a tool

to measure the academic performance of researchers and to evaluate the feasibility of

departments and universities (Johnstone, 2007).

For Garfield (1955), the calculation of impact factor was done based on the number of

citations received by the scientific journals in a period of two years. “If one intends, for

example, to calculate the impact factor of a journal in 2005, one will use the data from

2003 and 2004, i.e. the total number of citations in 2005 for articles published in 2003

and 2004 divided by the total number of citable items in 2003 and 2004” (Borges, 2006:

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

56).

Figure 1: Calculation of impact factor, example 1

Source: Maria Manuela Borges (2016). A Esfera. PhD thesis: 56.

Figure 2: calculation of impact factor, example 2

Source: The University of Illinois at Chicago University Library Website.

https://researchguides.uic.edu/if/impact

It should be noted that, parallel to the two-year criterion, there is also the five-year

impact factor, which is identical to the two-year impact factor, but with a naturally greater

time interval, allowing us to observe much subtler variations in citation counts.

The cumulative construction of the term

The enormous plasticity of meanings attributed to the universe of the concepts involved

in the phenomenon of open science collaborates for some parallel definitions, which also

determine variations within the interpretation of the impact factor itself, giving it sub-

dimensions that, together, allow us to look to the concept in a more maximalist and

structured way, as if it were layered.

As an example, Bauer and Bakkalbasi (2005) introduce two concepts: co-citation

analysis, which is the number of times two documents are cited simultaneously in later

publications; and bibliographic coupling, which consists in a forecast that two articles that

cite a previous work may have something in common (Borges, 2005: 55).

Other secondary indicators, such as the “h-index”, for example, already quantify scientific

productivity and the apparent scientific impact of a scientist or researcher through the

number of citations that their cited articles reach. “The h-index measures the total

number of papers a scientist has authored and the number of citations those papers have

received, may be more acceptable to some. If, over a lifetime of a research career, you

have authored 50 papers that have been cited 50 or more times, your h-index equals 50”

(Kupiec-Weglinski, 2015: 482). In this way, the h-index can more accurately measure

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

the micro side associated with individual scientific production.

The “g-factor” quantifies, on the other hand, the scientific productivity of a researcher

and its calculation is based on the distribution of the citations received in the publications

of this researcher. The g-factor was firstly developed to respond to the

underrepresentation of European scientific journals in the bibliographic databases of

Thomson ISI. The “y-factor” results from a simple combination of the impact factor

available on the ISI databases and the weighted “page rank” in order to consider and

adjust the impact factor according to the greater or lesser popularity of scientific journals

(Satyanarayama, 2008).

Thomson also publishes the influence index of the article and the immediacy index, which

appears as a measure of the time (speed) that elapses between the moment a given

content is acquired and the moment of its reference/citation.

Eugene Garfield (2006) acknowledges that the creation of the impact factor in 1955 was

based on the need to select additional journals and sources of research. With the

legitimacy conferred upon him as the creator of the term impact factor, Garfield argues

that the “term ‘impact factor’ has gradually evolved to describe both journal and author

impact” (Garfield, 2006: 1). He also states the impact factor of scientific journals usually

involves large amounts of articles and citations, and individual authors generally produce

fewer articles, although some of them have published an admirable number. Garfield

gives the example of a transplant surgeon named Tom Starzl, author of more than 2,000

scientific articles; and Carl Djerassi, who invented oral contraceptives and published more

than 1,300 articles.

The ineffectiveness of the indicator and its repercussion in the skewing

of the evaluation of the scientific knowledge produced

One of the problems associated with impact factor has to do with a kind of appropriation

of the indicator by major indexers, such as Scopus

or ISI Web of Science, resulting in

evidence of the most prominent journals indexed in them, to the detriment of others, or

even of other documents, such as monographs and dissertations (Borges, 2006). This

evidence leads us to a snowball effect with repercussions on the motivations and

strategies of researchers and scientists whose annual evaluation and career progression

depends to a large extent on the volume of publications and their impact factor.

New researchers need to publish in the most prestigious scientific journals as well as

senior and associate researchers, who need to do so to maintain their fellowships and/or

research funding or to progress in teaching careers. “The researchers submit the

manuscripts to prestigious journals” (Borges, 2006: 275), entering the databases of the

great world indexes, which institutionalise the calculation of the impact factor. “In the

case of the scientific article, recognition of its quality is linked to the journal in which it

is published” (Borges, 2006: 36). It depended previously on the editorial board and today

depends on the impact factors of core journals.

However, “Ever since its discovery in 1960s, impact factor continues to be in the news,

often for the wrong reasons” (Satyanarayama & Sharma, 2008: 4). Some authors point

Scopus uses two metrics: SJR (SCImago Journal Rank) and SNIP (Source Normalised per Paper).

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

out that there are significant dangers and limitations associated with the calculation of

impact factors, such as the lack of quality of citations and the existence of journal self-

citation (Archambault & Larivière 2009; Falagas et al, 2008; Vanclay, 2012) – scientific

journals that tend to value proposals for the publication of articles that refer to other

articles published by them – as well as the informal combinations of researchers who

take the modus operandi of citing each other, raising the impact factor of their articles.

For Seglen (1997), assessing scientific quality without a universal solution is an extremely

complicated task. The author argues that, ideally, all processes to validate scientific

knowledge produced should be defined by academic certification based on the scrutiny

and verification of real experts in a given area. In practice, the so-called “peer-review”,

usually carried out by committees that have too generalist competences, conditions

everything else, from the simplistic verification of the knowledge produced to the

processes that legitimise journals (Seglen, 1997). This author, who is a critic of the model

of legitimating scientific knowledge produced based on impact factors, argues that this

impact factor, or citation rate, does not represent individual scientific production as it

does not define its quality.

The author lists the problems at the basis of impact factors:

1) the impact factors of scientific journals are determined by technical procedures that

are not related to the production of scientific knowledge per se;

2) the impact factors of scientific journals depends on the research area. Higher impact

factors are usually associated with scientific journals that cover vast areas of

exploratory research, fast-growing and short-lived literature that typically involves

several references per article;

3) the fact that the citation rates of scientific journals determine the impact factor of

the publication, and not the other way around;

4) citations of non-citable items are usually included in the same databases;

5) review or survey articles are strongly cited and inflate the impact factors of some

scientific journals;

6) the larger articles result in higher citation levels, also inflating the impact factors of

journals;

7) printed works (i.e. books) are not included in the databases as sources of citation;

8) the bibliographic databases are oriented to the English language and dominated by

US publications;

9) the impact factors depend on the dynamics (expansion or contraction) of the different

academic areas;

10) small areas of academic research tend to have fewer journals with higher impact

factors;

11) the relationship between research fields also determines the impact factors of

scientific journals (i.e. differentiated fields interconnected in the area of health

compared to smaller fields of research);

12) the limitations of databases or the example given by the Science Citation Index,

which covers a limited number of scientific journals around the world (Seglen, 1997).

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

Ultimately, Seglen (1992) points out that it is the large variability in citation

processes that renders the criterion of impact factor little accurate, which means that

it should not be used to evaluate scientific production.

The article “Hate journal impact factors? New study gives you one more reason”,

written

by John Bohannon (2016) and published in Science Magazine, said that scientists have a

love-hate relationship with the impact factor of scientific journals. This measure, used to

classify scientific journal prestige, is seen by many as a destroyer of the scientific

community.

Regarding the need to rethink all metrics and impact factors as legitimators of the

knowledge produced, authors such as Adler, Ewing and Taylor (2009) suggest new

multiple criteria to validate these metrics separately for each discipline, whilst being

calibrated and adjusted according to the specificity of each discipline, as well as according

to the attributes of each classification. In other words, metrics and impact factors should

be as diverse and rich as possible.

Adler, Ewing and Taylor (2009), with a fundamentally optimistic view of the Open Science

model, particularly due to its ability to provide greater variability of metrics that is

contrary to structures of manipulation and misuse of themselves – possibly because, at

the date of the respective article, they had not contemplated yet some bias of the model

itself

– list some reasons, close to Seglen’s model (1997), that explain their opposition

to the panorama of metrics as a criterion of scientific validation:

1) the meaning of a citation may be even more subjective than the peer-review itself;

2) unique trust in citation metrics provides, at best, an incomplete and superficial

understanding of research;

3) the fact that the validity of statistics for the impact factor and h-index are not well

known or even well studied;

4) the fact that the citation metrics provide only a limited and incomplete view of the

quality of research, and the statistics resulting from these metrics can be

misunderstood and misused;

5) the possibility that exclusive trust in citation-based metrics may replace the criterion

of subjective peer-review as the preponderant element to validate research;

6) the idea that impact factor cannot be used to compare scientific journals between

disciplines;

7) the certainty that impact factor does not accurately reflect the range of citations in

some disciplines, as not all scientific journals are indexed; and

8) the possibility that impact factor can be easily influenced by the (high) frequency with

which some authors are wrongly identified.

http://www.sciencemag.org/news/2016/07/hate-journal-impact-factors-new-study-gives-you-one-more-

reason.

Model captured by commercial interests, with an evident crisis of reproducibility and with questionable

research practices. https://opensciencemooc.eu/.

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

Recurring to Moed, Borges (2006) points out other limitations, such as: “when measuring

the impact of citation in the second or third year after its publication, may be there is a

biased towards magazines that have a faster IF maturation or decline”.

One of the reasons given for the uselessness of the impact factors is that they have no

validity as a predictive measure, since they are vaguely calculated.

Paulus, Cruz and Krach (2018) attempted to illustrate the fallacies inherent in the use of

metrics to evaluate journals and scientists’ works. For the authors, the simple fact that

scientific quality is judged by the impact factors of the journals shows that we are being

driven by weak and invalid arguments. And the uncertainty regarding the quality of a

work is exceeded by its impact factor rather than the quality of the work itself.

Kupiec-Weglinski (2015) argues, in turn, that impact factors do not reflect the quality or

reliability of science nor the scientist’s ability, valences and creativity. In addition, the

author says that the most important thing is to publish quality and innovative knowledge

whilst maintaining a continuous record of publications and good productivity. “The bottom

line is, you need to publish research that is reproducible, impactful and moves your field

forward” (Kupiec-Weglinski, 2015: 482).

For Satyanarayama (2008), impact factors can even skew the direction of scientific

investigation, as the scientists themselves tend to direct their investigation to

mainstream areas that are more easily financed. On the other hand, those researchers

who are dedicated to less-mainstream areas, although important, have greater difficulty

in getting funds for research and being recognised. “It is well known that funding follows

what is considered ‘significant’ in science, which is usually science driven by citation hype

and publications in high impact journals” (Satyanarayama, 2008: 4).

Towpik (2015: 465), in contrast, points out a mania associated with the impact factor

that persists and inflicts a pernicious effect on science and on some scientific conducts

and practices.

Weglinski (2015) goes further and argues that high impact factors are often mere

conveniences that money can afford. The higher the impact factor of a scientific journal,

the higher the costs of publication addressed to the authors who decide to publish in that

same scientific journal and/or to the institutions/researchers that intend to acquire these

publications. As a consequence, the entire democratising root of the open science

movement is also perverted, with its own impact factor contributing to this scenario.

Moustafa (2015) considers that impact factor has become the worst enemy of scientific

quality, placing great pressure on authors, editors, stakeholders and funders. Worst of

all, in a significant number of countries, the allocation of budget and government funds

is entirely channeled into the only journals with so-called high impact factors, leaving

everyone else out.

Referring to the case of research in the area of health, more precisely in nursing,

Johnstone (2007) argues that this obsession with impact factors endangers the

sustainability and viability of scientific journals in nursing and its academic texts. In

nursing, the author explains, researchers abandon their publishing agendas in order to

publish only in elite scientific journals, some of them outside the area of nursing. The

author argues that other ways of assessing the quality and impact of scientific journals

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

in nursing should be planned, and that books and book chapters should also begin to be

included in the metrics.

Ironside (2007) is peremptory in stating that impact factors are not useful and should be

abolished, although they may provide some useful information for the review process if

used in a judicious and conscientious manner.

Finally, it should be noted that the arguments that give a more stated defence of impact

factor focus, above all, on 1) a kind of tacit acceptance of metrics by researchers and

academics, and 2) the defence of initial reductionist thinking of Eugene Garfield who

conceived the impact factor as a way to evaluate science and scientific journals per se.

Some authors, such as Oppenheim (2008), recognising the importance of metrics,

consider that impact factor is strongly correlated with other scientific quality criteria, such

as peer-review. Oppenheim (2008) cannot foresee, for example, that impact factor can

easily replace the criterion of the peer-review in the structures that validate the

knowledge produced. Essentially, because the second is mostly a closed and silent

criterion, it is restricted to the author-mediator-reviewer relationship and not available

for consultation, which makes it vulnerable to the easy accessibility and enormous reach

of the impact factor as a validation criterion of knowledge produced, as considered by

most of the academic structure. For authors who celebrate the existence of metrics and

impact factors as legitimators of knowledge produced, articles of proven quality are

necessarily the most cited, since citations and other metrics correlate strongly with the

most subjective side of the peer-review.

On the other hand, Kampourakis (2018) points out that the most beneficial side of the

impact factor, albeit indirectly, is related to the very idea of dissemination and running of

the knowledge produced because of two reasons. First, researchers and scientists have

realised that the greater the dissemination of their work (i.e. ResearchGate), for example,

amongst their networks, the greater the likelihood that they could be cited – besides

generating greater scrutiny and verification of the knowledge produced. Second, the

journals that reach higher impact factors end up having an acknowledgment that, in the

short-term, allows them to benefit from a larger number of articles for publication,

managing their needs from this bigger influx of articles.

Publications often tend to celebrate (with notifications to their subscribers) and to

congratulate themselves

when entering major indexers and consequently accessing

impact factors (i.e. Scopus – Scimago Journal & Country Rankings), since they perceive

that being part of this system is, in the first instance, a crucial criterion for their validation

and accreditation in the academic community in a much more decisive way than the

degree of quality that may be associated with the material published. This is because,

ultimately, impact factor is today understood as the first criterion to define the value of

a scientific journal and published articles.

Final considerations

In general terms, the discussion in the scientific community about whether or not impact

factors are reliable in measuring knowledge produced results in a theme that has been

https://onlinelibrary.wiley.com/doi/full/10.1002/admt.201800285.

https://www.advancedsciencenews.com/celebrating-first-impact-factor-advanced-science/.

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

associated with the very mutability and evolution of the Open Science movement.

Although academic debate focuses less on the tacit aspect of the analysis and reading of

the values of the impact factors and more on the contradictions and problems associated

with the phenomenon – with repercussion on the skew of the processes to legitimate the

knowledge produced – there is still room for the use and discussion of the main ideas

listed in this article.

An example of the importance of a broad reflection on the limits of the impact factor

concept can be defined by the attempt to make it increasingly efficient by introducing

new sub-dimensions that give it greater scope.

The g-factor, y-factor as well as h-index are good examples that aim to make the

definition criteria less dependent on the characteristics described by academia as being

detrimental and little recognisable for researchers’ work and the activity of scientific

journals that are not part of the so-called A-list of scientific publication (Howard, 2009).

Eugene Garfield (2006) suggests that impact factor accuracy is questionable and that

citation studies should be adjusted to include variables such as peculiarity of the

knowledge produced, citation density – average number of references cited by origin

article – as well as the criterion of half-life (also from Thomson Reuters) or the number

of years required to find 50% of the references cited.

In the future, it will be interesting to note: if the calculation of impact factors evolves

into a more inclusive and parity model that values issues such as the scientific

productivity of researchers and scientific journals, based on criteria to define scientific

quality; or if the legitimacy of the knowledge produced will continue to be an extension

of the tyranny of the metrics and the assumptions that are intrinsic to it. What is

important to discuss – and what constitutes the aim of this article – is whether or not

scientific openness can thrive in a world of publishing relations in which impact factors

tend to perpetuate monopolies of scientific dissemination, resulting from the infinite

replication of the status inherited by the previous publication of others and perpetuated

by the repetition of practices that legitimised a given position in the ranking of impact

factors. Therefore, should science produce aristocratic scientists and knowledge, based

on the inheritance of those who already had a status before by publishing in a given

periodical; or should it be based on merit and virtue, regardless of where it is published,

and on the peer-review of the scientific knowledge produced? Science develops itself in

both democratic and autocratic societies, but it also has a contribution for the

strengthening of democracy. It is up to us, scientists and researchers, to decide what

contribution we are ready to give, beyond knowledge produced, to the society that gives

us context and conditions to investigate and publish.

References

Adler, R., Ewing, J. & Taylor, P. (2009). Citation statistics: a report from the International

Mathematical Union (IMU) in Cooperation with the International Council of Industrial and

Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS).

Statistical Science, 24(1), 1-14. Retirado de

https://www.jstor.org/stable/20697661?seq=1#page_scan_tab_contents

Archambault, E. & Larivière, V. (2009). History of the jornal impact factor: Contingencies

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

and consequences. Scientometrics, 79 (3), 635–649. http://dx.doi.org/10.1007/s11192-

007-2036-x

Bare, C. (2014). The guide to Open Science. [Post em blogue]. Retirado de

http://digitheadslabnotebook.blogspot.com/2014/01/guide-to-open-science.html

Barthes, R. (1972). S/Z. Rio de Janeiro: Nova Fronteira.

Bauer, K. & Bakkalbasi, N. (2005). An Examination of Citation Counts in a New Scholarly

Communication Environment. D-Lib Magazine, 11(9). Retirado de

http://www.dlib.org/dlib/september05/bauer/09bauer.html

Berghmans, S. & Kisjes, I. (2015). The evolution of Open Science – how digitization is

transforming research. Elsevier Connect. Retirado de

https://www.elsevier.com/connect/the-evolution-of-open-science-how-digitization-is-

transforming-research

Bohannon, J. (2016). Hate journal impact factors? New study gives you one more reason.

Science. Edição online, 6 de Julho de 2016. Retirado de

http://www.sciencemag.org/news/2016/07/hate-journal-impact-factors-new-study-

gives-you-one-more-reason

Borges, M. M. (2006). A Esfera. Dissertação de Doutoramento em Letras, área de

Ciências Documentais, Especialidade de Tecnologias da Informação. Faculdade de Letras

da Universidade de Coimbra. Retirado de

https://estudogeral.sib.uc.pt/jspui/handle/10316/8557

Bourdieu, P. (1994). O Poder Simbólico. Algés: Difel.

Cardoso, G., et al (2009). As políticas de Open Access. Res publica científica ou auto-

gestão? Sociologia, Problemas e Práticas, 60, 53-67. Retirado de

http://www.scielo.mec.pt/pdf/spp/n60/n60a04.pdf

Falagas M. et al (2008). Comparison of SCImago jornal rank indicator with jornal impact

factor. FASEB journal, 22 (8), 2623–2628. https://www.fasebj.org/doi/10.1096/fj.08-

107938

Garfield, E. (1955). Citation Indexes for Science: A New Dimension in Documentation

through Association of Ideas. Science, 122 (3159), 108-111. DOI:

10.1126/science.122.3159.108

Garfield, E. (2006). The History and Meaning of the Journal Impact Factor. JAMA, 295

(1), 90-93. DOI.1001/jama.295.1.90.

Greenwood, D. C. (2007). Reliability of jornal impact factor rankings. BMC Medical

Research Methodology, 7(48). https://doi.org/10.1186/1471-2288-7-48.

Howard, J. (2009). Humanities Journals Confront Identity Crisis. The Chronicle of Higher

Education, 55(29), 1. Retirado de https://eric.ed.gov/?id=EJ838618

Ironside, P. M. (2007). Advancing the Science of Nursing Education: Rethinking the

Meaning and Significance of Impact Factors. The Journal of Continuing Education in

Nursing, 38 (3), 99-100. DOI10.3928/00220124-20070501-09

Johnstone, M.J. (2007). Journal impact factors: implications for the nursing profession.

International Nursing Review, 54 (1), 35-40. DOI: 10.1111/j.1466-7657.2007.00527.x

JANUS.NET, e-journal of International Relations

e-ISSN: 1647-7251

Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44

The impact factor as a legitimator of the scientific knowledge produced: a review of the literature

Tiago Lima Quintanilha, Gustavo Cardoso

Kampourakis, K. (2018). What is the impact of the impact factor? Science & Education,

27(5-6), 405-406.

Kupiec-Weglinski, J.W. (2015). Journal Impact Factor (JIF): The good, the bad, and the

ugly. Nowotwory. Journal of Oncology, 65(6), 481-482. DOI: 10.5603/NJO.2015.0094

Moustafa, K. (2015). The Disaster of the Impact Factor. Science and Engineering Ethics,

21(1), 139-142

Muller, J. Z. (2018). The Tiranny of Metrics. The Chronicle of Higher Education. Edição

online de 21 de Janeiro de 2018. Retirado de https://www.chronicle.com/article/The-

Tyranny-of-Metrics/242269?cid=wsinglestory_hp_1

Oppenheim, C. (2008). American Scientist Open Access Forum. [Post em blogue].

Retirado de http://openaccess.cmb.med.rug.nl/?p=146

Paulus, F.M., Cruz, N. & Krach, S. (2018). The Impact Factor Fallacy. Frontiers in

Psychology, 9(1487), 1-7.

Pirmez, C., Brandão, A.A., Momen, H. (2016). Scientific journal publishing is too complex

to be measured by a single metric: Time to review the role of the impact factor! Memórias

do Instituto Oswaldo Cruz, 111(9), 543-544.

Priem, J. & Hemminger, B. (2012). Decoupling the scholarly journal. Frontiers

Computational Neuroscience, 6(19), 1-13. https://doi.org/10.3389/fncom.2012.00019

Quintanilha, T. L. (2015). Considerações sobre os desafios múltiplos da Ciência Aberta.

Estudos em Comunicação, 21, 13-34. DOI: 10.20287/ec.n21.a02.

Saarela, M. et al (2016). Expert-based versus citation-based ranking of scholarly and

scientific publication channels. Journal of Infometrics, 10, 693-718.

https://doi.org/10.1016/j.joi.2016.03.004

Satyanarayama, K. & Sharma, A. (2008). Impact Factor. Time to move on. Indian Journal

of Medical Research, Editorial, 127(1), 4-6. Retirado de

http://www.ijmr.org.in/article.asp?issn=0971-

5916;year=2008;volume=127;issue=1;spage=4;epage=6;aulast=Satyanarayana;type

Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating

research. British Medical Journal, 314(7079), 498-502.

Sygocki, W. & Korzeniewska, E. (2018). Impact factor (IF) True or False? Przeglad

Elektrotechniczny, 1(1), 107-110.

Towpik, E. (2015). IF-mania: Journal Impact Factor is not a proper mean to assess the

quality of research, individual researchers, nor scientific institutions. Nowotwory. Journal

of Oncology, 65(6), 465-475. DOI: 10.5603/NJO.2015.0092

Vanclay, J. K., & Bornmann, L. (2012). Metrics to evaluate research performance in

academic institutions: a critique of ERA 2010 as applied in forestry and the indirect

H2index as a possible alternative. Scientometrics, 91 (3), 751–771.

http://dx.doi.org/10.1007/s11192-012-0618-8