22nd Nordic Workshop on Bibliometrics and Research Policy (NWB'2017) at House of Science and Letters, Helsinki, November 9-10, and the pre-workshop, November 8, 2017, 2017
The effects of performance-based research funding systems (PRFS) are much debated (Aagaard & Schn... more The effects of performance-based research funding systems (PRFS) are much debated (Aagaard & Schneider 2017). It is often assumed that PRFS create an increasing pressure to publish among academics (Fanelli & Larivière 2016). In Finland publications have been a criterion in the government’s block grant funding for universities since 2007, and the share of funding allocated on the basis of publications has multiplied after that.
To study if there is a pressure to publish and resulting inflation of scientific publications in the Finnish research community, we analysed personnel, publication and citation data on Finnish universities, and found that the average publication productivity among Finnish university staff has not increased from 1998 to 2016 (Pölönen & Auranen 2017). We also looked at the Web of Science publications in 1998-2014, and found that publication productivity of academics is not exceptionally high in Finland, compared to Sweden, Denmark and Norway. Citation impact has not remarkably decreased in Finland compared to the other Nordic countries since the introduction of the publication indicator in 2007.
PFRS may increase the pressure to publish (or experience of it) by raising the standards with regard to peer-review and publication channels. But also other factors contribute to pressure to demonstrate research prowess. Researchers face harsher competition for positions partaken by an increasing number of domestic and foreign researchers. A large share of universities’ research funding in Finland is competitive project funding, and often funding proposals are evaluated by international experts who appreciate publishing in leading international outlets.
Uploads
Papers by Otto Auranen
Data include articles from the Finnish universities in 2004-2008 indexed in the Thomson Reuters Web of Science (WoS) database. Total fractionalized number of articles is 32,242. All the articles which have at least one author with an address belonging to a Finnish university are included in the data. Publication Forum based points received by publications are compared to field-normalized citation impact scores of the same publications. Field-normalized citation impact is used to proportion citation numbers of publications according to average citation numbers of different fields of science.
Differences between Publication Forum points and citation impact scores are compared according to disciplines, universities, and more specifically within one discipline
(mathematics and statistics).
Results show that the greater the number of publications, the higher is the convergence between Publication Forum points and citation impact scores. As a rule, Publication Forum Rating is able to indicate research quality relatively well in the case of
large publication numbers. Publications in higher ranked journals receive a higher average number of citations than publications in lower ranked journals. Furthermore,
higher proportion of the publications in higher ranked journals is among the group of most cited tenth of world’s publications (Top10 publications) than of the publications in lower ranked journals. Higher ranked journals include fewer publications with no citations than the lower ranked journals.
The smaller the publication numbers evaluated by the Publication Forum Rating, the more arbitrary is the relation to citation impact scores. In the case of individual publications, the difference between results from Publication Forum Rating and citation impact is particularly evident. The conclusion is that Publication Forum Rating is not usable in evaluating the quality of individual publications. Even when evaluating large
publication numbers, the Rating is better suited to be used within disciplines than among them.
Data include articles from the Finnish universities in 2004-2008 indexed in the Thomson Reuters Web of Science (WoS) database. Total fractionalized number of articles is 32,242. All the articles which have at least one author with an address belonging to a Finnish university are included in the data. Publication Forum based points received by publications are compared to field-normalized citation impact scores of the same publications. Field-normalized citation impact is used to proportion citation numbers of publications according to average citation numbers of different fields of science.
Differences between Publication Forum points and citation impact scores are compared according to disciplines, universities, and more specifically within one discipline
(mathematics and statistics).
Results show that the greater the number of publications, the higher is the convergence between Publication Forum points and citation impact scores. As a rule, Publication Forum Rating is able to indicate research quality relatively well in the case of
large publication numbers. Publications in higher ranked journals receive a higher average number of citations than publications in lower ranked journals. Furthermore,
higher proportion of the publications in higher ranked journals is among the group of most cited tenth of world’s publications (Top10 publications) than of the publications in lower ranked journals. Higher ranked journals include fewer publications with no citations than the lower ranked journals.
The smaller the publication numbers evaluated by the Publication Forum Rating, the more arbitrary is the relation to citation impact scores. In the case of individual publications, the difference between results from Publication Forum Rating and citation impact is particularly evident. The conclusion is that Publication Forum Rating is not usable in evaluating the quality of individual publications. Even when evaluating large
publication numbers, the Rating is better suited to be used within disciplines than among them.
To study if there is a pressure to publish and resulting inflation of scientific publications in the Finnish research community, we analysed personnel, publication and citation data on Finnish universities, and found that the average publication productivity among Finnish university staff has not increased from 1998 to 2016 (Pölönen & Auranen 2017). We also looked at the Web of Science publications in 1998-2014, and found that publication productivity of academics is not exceptionally high in Finland, compared to Sweden, Denmark and Norway. Citation impact has not remarkably decreased in Finland compared to the other Nordic countries since the introduction of the publication indicator in 2007.
PFRS may increase the pressure to publish (or experience of it) by raising the standards with regard to peer-review and publication channels. But also other factors contribute to pressure to demonstrate research prowess. Researchers face harsher competition for positions partaken by an increasing number of domestic and foreign researchers. A large share of universities’ research funding in Finland is competitive project funding, and often funding proposals are evaluated by international experts who appreciate publishing in leading international outlets.