Purpose: Business and management journal rankings are controversial but influential for scholars seeking publishing venues and for appointment, tenure and promotion committees needing to evaluate applicants’ work. Whilst some prominent rankings are citation-based, others are constructed by field experts. This article assesses whether Large Language Models (LLMs) can provide credible new business and management journal rankings.
Design/methodology/approach: Based on mean ChatGPT 4o-mini scores for business and management articles published between 2014 and 2020 and submitted to the UK Research Excellence Framework (REF) 2021, ChatGPT-based rankings were compared with expert rankings from the Australian Business Deans Council (ABDC) and the Chartered Association of Business Schools (CABS), weighted normalised citation-based rankings, mean REF citation scores, and mean REF departmental quality scores.
Findings: For the 43 journals with at least 50 articles and data from all six sources, the ChatGPT scores correlated more strongly with expert rankings (CABS: 0.438, ABCD: 0.510) than any of the citation rankings except Scimago Journal Rank (SJR) for one of the two (CABS: 0.664, ABCD: 0.360). Journal scores calculated from REF departmental quality score rankings had the highest Spearman correlations with the established rankings, however (CABS: 0.717, ABCD: 0.583). If rankings based on REF departmental quality scores are taken as optimal, then ChatGPT scores have the highest correlation with this (0.830), greater even than with the two expert rankings.
Originality/value. ChatGPT-based journal quality scores are plausible new ranking mechanism for business and management journals and may be superior to citation-based rankings in some cases, potentially providing more current, finer grained and cheaper results.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)