Measuring Academic Impact

The Library can support you during the REF, and help you to understand the ways that the academic impact or influence of your research can be measured using publication and citation data, and altmetrics.

Research Excellence Framework (REF)

The REF is the system used by HEFCE to assess the quality of research conducted at higher education institutions in the UK. There is a strong emphasis on open access in the post-2014 REF.

Academic impact

The impact of research can be illustrated and assessed in many ways, both qualitative and quantitative.

Bibliometrics is the quantitative analysis of research publications and citations, and is one way in that the academic impact of a publication, research group or individual can be measured. This data is currently very influential in University rankings.

These measures are used alongside qualitative measures such as peer review. These metrics are not intended to be used to assess societal or economic impact.

Measuring your Research Impact is a comprehensive and interactive tutorial developed by four Irish academic libraries.


Sources of citation and publication data

There are several sources of publication and citation data that can be used to calculate citation metrics at Lancaster University. The data will vary depending on the coverage of each product or database. 

Scopus

Scopus is a very large abstracting and indexing database produced by academic publisher Elsevier. It has a broad coverage of around 22,000 journal titles, academic books, conference papers and patents.

It provides comprehensive citation data back to 1996, and has expanded to over 5 million pre-1996 records in the last 2 years.

Scopus allows you to analyse citation data at the journal, article and author level, and provides access to a number of unique features, such as source normalised citation metrics which enable comparison of articles and journals across disciplines.

Access Scopus

Scopus data can be further analysed using SciVal.

Journal Metrics (Scopus)

The user-friendly Journal Metrics rank is based on data from Scopus. This product allows you to analyse, compare and rank journal titles based on their publication and citation data. Arrange titles in subject categories or search for individual titles. Use the SNIP or SJR if you are trying to rank journal titles in different subject categories.

Access Journal Metrics

SciVal

SciVal is a research intelligence solution providing users with citations and publications data from over 7,500 research institutions in 220 countries worldwide.  SciVal is based on output and usage data from Scopus. 

It allows users to search for research output data for their own publications, for the university as a whole, or for other researchers and institutions. 

It can be used in a variety of ways including:

  • to provide output and impact data for a specific area of research or a specific institution
  • to identify partners for future research collaborations
  • to highlight strengths and weaknesses in research output in order to optimise departmental or institutional strategy.

Access SciVal and create an account using your Lancaster University email address.

Download the SciVal introductory guide (PDF) for Lancaster University users.

A quick guide and a more comprehensive online manual are available through the Help section of the SciVal website, but for any other queries, please contact the Academic Services Team at academicliaison@lancaster.ac.uk

Web of Science

Web of Science has very deep coverage of over 12,500 influential journals and conferences, now provided by Clarivate Analytics (formerly by Thomson Reuters). It comprises both the earliest citation indexes from the mid-20th century, to emerging sources.

Web of Science allows you to analyse citation data at the article and author level using the ‘Citation Report’ function. Journal titles are analysed using the Journal Citation Reports (JCR), where you can find metrics such as the Journal Impact Factor, and compare journals in different subject categories.

Access Web of Science

Useful video: Web of Science: Citation Reports

Journal Citation Reports (JCR)

Journal Citation Reports are produced annually based on publication and citation data from Web of Science. This product allows you to analyse, compare and rank journal titles based on their publication and citation data, and are arranged in subject categories. This is the authoritative source of the Journal Impact Factor, and other indicators. JCR are usually released in June for the previous calendar year.

Access Journal Citation Reports

Useful video: Web of Science: Journal Citation Reports

Google Scholar

Typically Google Scholar has a broad coverage including grey literature, so citation counts are likely to be higher than in the other products. However, be aware that there may be false citations that inflate the citation counts in Google Scholar. Individuals can register for their own Google Scholar profile and create and track a list of their publications.

Google Scholar also calculates citation metrics for individual authors with a Google Scholar Profiles, and ranks publications by language and in research areas.

Access Google Scholar

Publish or Perish can be used for further analysis of Google Scholar citation data.

Publish or Perish

Publish or Perish, created by Anne-Wil Harzing, is free software which calculates a wide range of metrics based on citation data from Google Scholar. Prof. Harzing critiques and develops novel indicators based on this data.

Access Publish or Perish.

There are detailed help materials within the software, in addition to the Publish or Perish tutorial and accompanying book.


Citation analysis

Journal metrics

  • Journal Impact Factors in the Journal Citation Reports database ranks journals using citation data from ISI Citation Indexes in the Web of Knowledge based on the previous 2 years
  • 5 year Impact Factor extends the citation data over 5 years
  • SCImago Journal Rank expresses the number of weighted citations in the selected year by documents published in the three previous years.
  • SNIP based on Scopus data aims to allow direct comparison between journals in different fields with different citation patterns. It provides a ratio of a journal's citation count per paper and the citation potential in its subject field. Journal Metrics (Scopus) is a user-friendly rank which includes the SNIP and SJR. 

Author metrics

  • Find citations per year and average citations per article for authors in Web of Science and Scopus
  • h index is commonly used as an indicator of the influence and productivity of individuals, institutions or research groups using citation data from Web of Science, Scopus and Google Scholar. Note that h-index will vary depending on which source you use. Although it is commonly used it is not useful for comparing author performance across disciplines or at different stages in their careers.
  • Google Scholar profiles show citations per year for individual authors
  • Scopus has a user-friendly 'Analyze Author Output' tools, which can be used to identify trends in an author's publishing history
  • Publish or Perish calculates a wide range of metrics, including improvements on the h-index, based on citation data from Google Scholar
  • SciVal enables you to analyse citations and publication data for individuals, teams, and institutions. It is possible to define your own research groups to analyse research performance and benchmark against other teams or institutions.

Article level metrics

  • Find citation counts for individual articles from Web of Science, ScopusGoogle Scholar and publishers' websites.
  • Scopus calculates the Field Weighted Citation Impact (FWCI) and benchmarking percentiles for articles
  • Many repositories and digital libraries provide usage statistics for downloads and views, which may indicate future citations.

Alternative metrics

Altmetrics is an emerging field of research impact analysis which goes beyond using citation data from a selection of peer-reviewed journals to include the data that is generated from the social web when people share links, add bookmarks, download articles or blog about research. Altmetrics also track the attentions received by other 'research outputs', such as data, posters, presentations, websites and so on.

These metrics can complement bibliometrics in understanding the attention that research is attracting, and may indicate future citations or societal impact.

The University doesn't currently subscribe to any specific altmetric products. Many publishers, such as Nature and PLoS, now present them alongside other article-level metrics. Scopus provides altmetrics such as social media activity alongside other citation metrics.


Help with citation analysis

All of the tools mentioned come with help and tutorials, with thorough explanations of how the metrics are calculated.

The Library can provide help to:

  • effectively use Web of Science, Journal Citation Reports, Scopus, SciVal or Google Scholar for citation analysis
  • decide which sources and metrics are right for your purpose
  • understand the limits of each source
  • interpret the citation data

Contact the Academic Services Team at academicliaison@lancaster.ac.uk