Skip to main content

Hyphens in paper titles harm citation counts and journal impact factors

According to the latest research results, the presence of simple hyphens in the titles of academic papers adversely affects the citation statistics, regardless of the quality of the articles. The phenomenon applies to all major subject areas. Thus, citation counts and journal impact factors, commonly used for professorial evaluations in universities worldwide, are unreliable.
Scientific papers
This breakthrough finding poses a fundamental challenge to the rule of the game in determining the contributions of papers, journals, and professors. It is unveiled in a paper titled "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors" by Zhi Quan Zhou, T.H. Tse, and Matt Witheridge, recently published in IEEE Transactions on Software Engineering, the top journal in the field.
T.H. Tse is an honorary professor in computer science at the University of Hong Kong (HKU). Zhi Quan Zhou received the PhD degree from HKU and is currently an associate professor in software engineering at the University of Wollongong, Australia. Matt Witheridge is a PhD student at the University of Wollongong.
Scopus and Web of Science are the two leading citation indexing systems. Scopus provides the citation statistics to support the Times Higher Education World University Rankings and the QS World University Rankings. Web of Science provides the journal impact factor that supports the ranking of major journals. Because of the importance of these two indexing systems, it is essential to assure their quality. In particular, robustness testing refers to the verification of the systems' ability to deal with erroneous inputs or unexpected situations. For example, can the indexing system handle a citation properly if there is a minor typo when quoting the paper title?
Professor Tse and team members proposed an innovative method named "metamorphic robustness testing" to verify Scopus and Web of Science. The in-depth study uncovered robustness defects in both systems that might produce erroneous citation counts for papers with hyphens in the titles, so that the journal impact factors subsequently computed are problematic.
Back in 2015, Letchford and colleagues conducted a large-scale study on Scopus, and found that papers with shorter titles tended to be cited more than those with longer titles. Their results were widely reported in international media including Science and Nature.
On the contrary, Professor Tse and the present team find that it is actually the number of hyphens in the title that serves as the more dominating factor for citation counts. Usually, the number of hyphens is correlated to a paper's title length, thus giving the misinterpretation that citation counts depend on title length.
Citation practices vary across subject areas. Publications in certain fields may have systematically higher citation counts than in other fields. For example, one may argue that papers in chemistry (where paper titles often carry hyphens as part of the chemical nomenclature) only receive relatively limited numbers of citations, giving rise to a spurious negative correlation between hyphens and citations. Hence, the team carried out focused studies on journals in specific subject areas. The results indicated that hyphens adversely affect the citation counts of papers even if the study is only limited to some particular discipline.
To build on the findings at the article and discipline levels, the team investigated the impact of hyphens in paper titles at the journal level. Journal impact factor (JIF) is a common metric for determining the citation frequency of an academic journal. It is frequently used to represent the relative importance of a journal within its field. A software engineering field-wide study reveals that the higher JIF-ranked journals are publishing a lower percentage of papers with hyphenated titles.
The team further conducted an analysis of the validity of the research to avoid falling into the trap of equating correlation with causation.
"Our results question the common belief by the academia, governments, and funding bodies that citation counts are a reliable measure of the contributions and significance of papers. In fact, they can be distorted simply by the presence of hyphens in article titles, which has no bearing on the quality of research. Similarly, our results also challenge the validity of journal impact factors," said Professor Tse.
"These surprising results are of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. They are applicable to all faculties in any university," he added.
Story Source:
Materials provided by The University of Hong Kong. 
Note: Content may be edited.

Comments

Post a Comment

Popular posts from this blog

Dark matter may be older than the Big Bang

Dark matter, which researchers believe make up about 80% of the universe's mass, is one of the most elusive mysteries in modern physics. What exactly it is and how it came to be is a mystery, but a new Johns Hopkins University study now suggests that dark matter may have existed before the Big Bang. The study, published August 7 in  Physical Review Letters , presents a new idea of how dark matter was born and how to identify it with astronomical observations. "The study revealed a new connection between particle physics and astronomy. If dark matter consists of new particles that were born before the Big Bang, they affect the way galaxies are distributed in the sky in a unique way. This connection may be used to reveal their identity and make conclusions about the times before the Big Bang too," says Tommi Tenkanen, a postdoctoral fellow in Physics and Astronomy at the Johns Hopkins University and the study's author. While not much is known about its origins,...
Post a Comment
Read more

Home births as safe as hospital births: International study suggests

A large international study led by McMaster University shows that low risk pregnant women who intend to give birth at home have no increased chance of the baby's perinatal or neonatal death compared to other low risk women who intend to give birth in a hospital. The results have been published by  The Lancet 's  EClinicalMedicine  journal. "More women in well-resourced countries are choosing birth at home, but concerns have persisted about their safety," said Eileen Hutton, professor emeritus of obstetrics and gynecology at McMaster, founding director of the McMaster Midwifery Research Centre and first author of the paper. "This research clearly demonstrates the risk is no different when the birth is intended to be at home or in hospital." The study examined the safety of place of birth by reporting on the risk of death at the time of birth or within the first four weeks, and found no clinically important or statistically different risk between home...
Post a Comment
Read more

GSAT-11 satellite to be launched from French Guiana on Dec 5th

GSAT-11 satellite to be launched from French Guiana on Dec 5th GSAT-11 would be located at 74 East and is the fore-runner in a series of advanced communications satellite with multi-spot beam antenna coverage over Indian mainland and Islands, ISRO said. GSAT-11 is the next generation “high throughput” communication satellite configured around ISRO’s I-6K Bus. (PTI/Representational). Indian space agency ISRO is scheduled to launch GSAT-11, the “heaviest” satellite built by it, on-board Ariane-5 rocket of Arianespace from French Guiana on December 5. Weighing about 5,854 kg, GSAT-11 would play a vital role in providing broadband services across the country, and also provide a platform to demonstrate new generation applications, the Indian Space Research Organisation (ISRO) said. It is the “heaviest” satellite built by ISRO, the space agency said. GSAT-11 is the next generation “high throughput” communication satellite configured around ISRO’s  I-6K Bus, and it...
Post a Comment
Read more