# Artificial intelligence technologies to support research assessment: A review

**Authors:** Kayvan Kousha, Mike Thelwall  
**arXiv:** [2212.06574](https://arxiv.org/abs/2212.06574)  
**Date:** 2022-12-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2212.06574v1/

## Key points

- Authors Kayvan Kousha and Mike Thelwall from the University of Wolverhampton delve into predicting journal article citation counts or quality using article text and metadata
- Indicators associated with higher impact research include article titles, abstracts, cited references, readability, number of authors, international collaborations, journal impact factors, and authors' h-index
- Machine learning techniques are used to predict citation counts or quality scores for journal articles and conference papers
- Evidence shows the association between bibliometric indicators and quality score rankings in UK Research Assessment Exercises (RAEs) and REFs as well as in other countries like Australia and Italy
- Public datasets from platforms like Dimensions, OpenCitations, Altmetric.com, and Publons are used to predict scholarly impact of articles
- Technology is utilized to automate editorial processes for quality control and reviewer suggestions
- Tools are designed to efficiently match reviewers with articles and categorize journal articles into specific fields
- The report addresses issues related to bias and transparency in technology-assisted assessment processes.

## Summary

In this supplementary report on the responsible use of technology-assisted research assessment, authors Kayvan Kousha and Mike Thelwall from the Statistical Cybermetrics and Research Evaluation Group at the University of Wolverhampton in the UK delve into various aspects of predicting journal article citation counts or quality from both article text and metadata. The report covers a range of indicators that are associated with higher impact or quality research, including factors such as article titles, abstracts, lengths, cited references, readability, number of authors, international collaborations, journal impact factors, and authors' h-index. The report highlights studies that have utilized machine learning techniques to predict citation counts or quality scores for journal articles and conference papers. It also explores evidence regarding the strength of association between bibliometric indicators and quality score rankings from previous UK Research Assessment Exercises (RAEs) and REFs in different subjects and years, as well as similar evidence from other countries like Australia and Italy. Furthermore, the report surveys studies that have used public datasets of citations, social media indicators, or open review texts from platforms such as Dimensions, OpenCitations, Altmetric.com,
and Publons to help predict the scholarly impact of articles. These findings were used to inform experiments utilizing machine learning to predict REF journal article quality scores. Additionally,the report delves into technology aimed at automating editorial processes for providing quality control for papers and reviewers' suggestions. It also discusses tools designed to match reviewers with articles efficiently and automatically categorize journal articles into specific fields. The report addresses issues related to bias and transparency in technology-assisted assessment processes. Overall,this detailed summary provides a comprehensive overview of how technology is being leveraged to support research assessment practices while emphasizing responsible usage considerations in evaluating scholarly work.