Measuring Research Impact
What are Bibliometrics?
Bibliometrics is the quantitative analysis of research literature, based upon citations, and can be used to evaluate the impact on the academic community of a research paper, an individual researcher, a research group or institution, or a journal.
The types of bibliometric measures include:
- Citation counts: the number of times a research output appears in the reference lists of other articles and books. Found in: Google Scholar, Scopus and Web of Science.
- H-index: designed to measure an author's productivity and impact. It is the number of an author’s publications (h) that have h or more citations to them. Found in: Google Scholar, Scopus and Web of Science.
- Field-weighted citation impact: the ratio of citations received relative to the expected world average for the subject field, publication type and publication year. It can apply to a research output or group of research outputs. Found in SciVal.
- Outputs in top percentiles: the number or percentage of research outputs in the top most-cited publications in the world, UK, or a specific country. Found in SciVal.
- Journal Impact Factor: based on the average number of citations received per paper published in that journal in the preceding two years. Found in Journal Citation Reports.
- CiteScore: the average number of citations received in a calendar year by all items published in that journal in the proceeding three years.
- SCImago Journal Rank: places a higher value on citations from more prestigious journals.
- Scopus SNIP: a ratio of a journal's citation count per paper and the citation potential in its subject field. The Scopus SNIP normalises citation rate subject differences. Found in Scopus.
(Adapted from the Metrics Toolkit licensed under a CC-BY 4.0 licence.)
For further information about the different metrics available please visit the Metrics Toolkit.
Where does it fit in the Vitae Research Development Framework?
Research Skills (RDF A1).
Why should you do it?
Bibliometrics can be used to identify top performing journals in a subject area, in order to: decide where to publish: learn more about a subject area or identify emerging areas of research.
However, bibliometrics and impact factors are not necessarily a measure of quality - an article may be cited frequently because other authors are refuting its findings; nor are bibliometric scores are not applicable for all subject areas because some research fields cite papers more than others, for example, in medicine and health there is a strong culture of citing and using other studies to validate findings.
Bibliometrics must be considered carefully and in context. A golden rule is that bibliometic scores should only only be considered on a like-for-like basis; in other words a bibliometric score for a particular psychology journal should only be considered againsed other psychology journals.
How do you do it?
At the University of Cumbria, the main citation tool is the Web of Science. Web of Science can provide article level citation data. The Journal Citation Reports (JCR), contained in Web of Science will provide you journal level metrics. Google Scholar also offers some citation searching and journal level metrics.
What are Altmetrics?
Altmetrics are based on the number of times an article is shared, downloaded or mentioned on social media, blogs or in newspapers.
Altmetrics can be considered alongside traditional bibliometric measures such as citation counts, h-index or journal impact factors. This will give a wider picture of how a piece of research is being read and discussed. Altmetrics may also give a more immediate indication of how an article is received than citations in publications.
Like any other metrics, altmetrics have their limitations.
A high number of shares or social media mentions does not necessarily mean that an article is of high quality. An article may be mentioned on social media because it contains something amusing or unusual.
Social media can also be easily manipulated and "likes" or mentions can be paid for or generated, so the numbers might not reflect the actual level of public interest in a piece of work.
More info can be found on the Altmetric for Researchers site.
Who can help?
Library Resources and Research Manager | Library Resources and Skills Team | Information Services
T: 01524 384455. Ext: 4455 | E: email@example.com
Useful guides and resources:
- An Introduction to Bibliometrics (Presentation)
In this short video, Prof John Walsh from the School of Geological Sciences at University College Dublin discusses some the limitations that they have encountered in using Bibliometrics in their field