Ranking Journals: Academic Journal Guide 2021 (“ABS List”)
The influential Chartered Association of Business Scholars has just published its Academic Journal Guide 2021 (“ABS list”). What does this mean for the operations and supply chain management (OSCM) research community? I have looked at the ranks for 15 major OSCM journals.
Once again, only the following OSCM journals were classified in category 4*: Journal of Operations Management, Management Science and Operations Research. To be honest, I wonder if the asterisk is really still appropriate for Operations Research.
The following OSCM journals were given a 4 in the 2021 ABS list: European Journal of Operational Research, International Journal of Operations & Production Management, Journal of Supply Chain Management and Production & Operations Management. This is good news for our discipline, because it means that the Journal of Supply Chain Management has moved up into this important category. However, it would have been time to give this journal not just a 4, but a 4*.
The following OSCM journals were given a grade of 3: Decision Sciences, International Journal of Production Economics, Journal of Business Logistics, Journal of Purchasing & Supply Management, Manufacturing & Service Operations Management and Supply Chain Management: An International Journal. Here, there are even two new entries: Journal of Business Logistics and Journal of Purchasing & Supply Management. While this is good news for both logistics and procurement scholars, I would have expected Journal of Business Logistics and Manufacturing & Service Operations Management to rank even higher.
The International Journal of Physical Distribution & Logistics Management lands at a 2 and the International Journal of Logistics Management lands at a 1 again. The low scores for these two journals in the 2021 ABS list are particularly strange given their quality.
My conclusion: The team behind the 2021 Academic Journal Guide appears to have listened – at least partly – to the harsh criticism from OSCM scholars. Although our discipline is certainly still underrated, compared to many other disciplines, there are finally some bright spots that give OSCM researchers a little more air to breathe. Empirically-focused OSCM journals were particularly disadvantaged by the ABS list in the past and three of them have now been upgraded. This step was overdue.
This slightly positive development for our discipline should not hide the harmful effects of rankings in general. Academia is increasingly about metrics rather than content. Assessment committees, bonus decisions and tenure-track regulations are increasingly about counting the names of certain journals per year – instead of reading them. The REF system in the UK has turned academic debate (i.e., quality) into a “race for points” (i.e., quantity) in the home country of the ABS list. This is a very negative development.
Therefore, it is gratifying that more and more universities are signing the San Francisco Declaration on Research Assessment (DORA), which asks “not [to] use journal-based metrics […] as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions”.
Research Quality vs. Research Quantity
I recently read on LinkedIn how a department head bragged about how many papers his team published in highly ranked journals over the past year. This mentality has to stop because it does not lead to more, but to less quality. In fact, quantity competes with quality. Often “motivated” by the disincentive systems of their universities, many academics waste their time writing lots of papers that no one ever will cite. They should rather invest this time in writing one great paper. This can take years, but is worth doing. For example, Mark Granovetter is a highly acclaimed academic who some consider worthy of the Nobel Prize in Economics. Google Scholar counts almost 150,000 citations of his work. Yet, 70% and 84% of these citations refer to just 2 and 5 of his great papers, respectively. So if you ever sit on an assessment committee or make bonus decisions, do not just count the number of publications in certain journals per year. You might then overlook academic leaders like Granovetter.
2018 Impact Factors of SCM Journals
Claviate Analytics have recently published their newest InCites Journal Citation Reports. It is great to see that the 2018 impact factors of all but one journals related to supply chain management have increased again, which highlights the rapidly growing relevance of our discipline. Two journals have an impact factor larger than 7: Journal of Operations Management (7.776; +2.9) and Journal of Supply Chain Management (7.125; +1.0). With an impact factor larger than 5, International Journal of Physical Distribution & Logistics Management (5.212; +1.0) has now arrived in the first league of management journals. Other SCM-related journals with high impact factors are: Supply Chain Management: An International Journal (4.296; +0.5), Management Science (4.219; +0.7), International Journal of Operations & Production Management (4.111; +1.2), Journal of Business Logistics (3.171; +0.3) and Journal of Purchasing & Supply Management (3.089; −0.6). But there are even more SCM journals with an impact factor around 2: Manufacturing & Service Operations Management (2.667; +0.9), Operations Research (2.604; +0.3), International Journal of Logistics Management (2.226; +0.5), Production and Operations Management (2.171; +0.4) and Decision Sciences (1.960; +0.3). Although the impact factor is certainly an imperfect proxy of a journal’s quality, I can only hope that rather conservative qualitative rankings, such as the ABS-AJG list, the UT Dallas list or the FT50 list, will finally be adapted to this new reality. This step is urgently needed!
Impact Factors of Supply Chain Management Journals
Note: The following text refers to a previous version of the JCR impact factors. A more recent version are the 2018 JCR impact factors (see there).
More than many other management disciplines, SCM has been very successful to professionalize and reinvent itself. A good indication for this development are the 2017 JCR journal impact factors, which have just been released. Many, although not all, impact factors of SCM journals have improved. The journal with the highest impact factor among SCM journals and the seventh-highest one among more than 200 management journals is Journal of Supply Chain Management (6.105). Three SCM journals have impact factors between 4 and 5: Journal of Operations Management (4.899), International Journal of Production Economics (4.407) and International Journal of Physical Distribution & Logistics Management (4.215). IJPDLM is now even the Emerald journal with the highest impact factor, which is a great achievement! Two other SCM journals range between 3 and 4: Supply Chain Management: An International Journal (3.833) and Journal of Purchasing & Supply Management (3.667), but International Journal of Operations & Production Management (2.955) and Journal of Business Logistics (2.891) also come close to 3. Both International Journal of Logistics Management (1.776) and Decision Sciences (1.641) were able to improve their impact factors and get closer to 2. It might take some time until our journals will finally be acknowledged by qualitative rankings such as CABS’s AJG and VHB-JOURQUAL, as such rankings tend to be quite conservative. However, with such high impact factors there should be no doubt anymore that SCM plays in the same league as accounting, marketing and finance.
Ranking Journals: Academic Journal Guide 2018 (“ABS List”)
Note: The following text refers to the 2018 version of the Academic Journal Guide. There is a more recent version: Academic Journal Guide 2021 (see there).
Three years ago, the Chartered Association of Business Schools (ABS) has released its last ranking of business journals: the Academic Journal Guide (AJG), also known as the “ABS list”. This ABS ranking has become quite influential as the guiding journal ranking across management disciplines in the UK. Although the ranking has been heavily criticized (see my previous post) and more democratically-developed rankings exist (e.g. VHB-JOURQUAL), the ABS ranking has since been adopted by many business schools also in other countries.
Shortly after the publication of the last ABS list, Nature has published ten principles to guide research evaluation, which have since become known as the Leiden Manifesto. One of these principles argues that quantitative evaluation should support qualitative, expert assessment, not replace it. Similarly, the San Francisco Declaration on Research Assessment (DORA), which has been signed by thousands of researchers worldwide, asks “not [to] use journal-based metrics […] as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions”. The Community for Responsible Research in Business Management (cRRBM) questions whether “even the academy is being served when faculty members are valued for the quantity and placement of their articles, not for the benefit their research can have for the world”. Indeed, the negative consequences of rankings are well-documented (see, for example, Espeland & Sauder, 2007 and Grant & Kovács, 2018).
Unfortunately, the ABS list has often been used in exactly the way the Leiden Manifesto and DORA wanted to prevent us from doing – with very negative consequences for our discipline. Many SCM researchers feel that the ABS list undervalues the journals of our discipline. For example, Journal of Supply Chain Management, a journal with one of the highest impact factors in management (5.789), is ranked by ABS 2015 as a “3” only and Journal of Business Logistics as a “2”. Comparing our journals to other disciplines, like accounting or marketing, it becomes apparent to me that these journals deserve a “4*” and “4”. Worse even, many SCM journals have suffered from their low ABS rankings, as SCM researchers who took the ABS ranking too seriously, felt they should publish in the higher-ranked journals of other disciplines. AJG 2015 has thus become a self-fulfilling prophecy.
Today, the Academic Journal Guide 2018 has been released. It contains more than 1,500 entries. Unfortunately, AJG 2018 did not adapt the ranks of leading SCM journals. When it comes to empirical journals, only Journal of Operations Management gets a “4*”, while Decision Sciences, Journal of Supply Chain Management and Supply Chain Management: An International Journal get a “3” again. International Journal of Physical Distribution and Logistics Management, Journal of Business Logistics and Journal of Purchasing and Supply Management all still get a “2” only. The fact that the rank of none of these journals has changed indicates that the new ranking does not seem to cover all the changes and improvements our discipline has made in the last couple of years. Only four out of eight journals (four empirically- and analytically-focused journals each) that have been identified by leading SCM schools as the top journals of our discipline (sometimes informally referred to as “the SCM basket of eight”; see The SCM Journal List), got an AJG rank higher than 3. My suggestion would be to ignore AJG 2018, as it does not seem to represent the journals of our discipline properly.
Interestingly, even the Chartered ABS itself just tweeted that the REF2021 sub-panel will ignore their own list, so they might well be aware of its shortcomings:
#REF2021 sub-panel chair: “We will not be using metrics. Our role is to assess originality, significance and rigour. We will not use #AJG2018 to do so.” Robert Blackburn #ARC2018
But there is some hope: This latest iteration of AJG was an interim review, the main purpose being to include new journals. The next iteration, planned to be published in 2021, will then be a major review. This is certainly very late, but better late than never: Careers can depend on such lists. Therefore, our discipline urgently deserves a better quality of the AJG. I hope the team behind the list is aware of their responsibility.
Getting Starry-eyed about Journals (Guest Post by Alan McKinnon)
My guest post today comes from Alan McKinnon who for several years has been raising concerns about the academic obsession with journal rankings and low rating of logistics/SCM journals. He has just published a new paper updating his earlier arguments.
In a paper that I wrote five years ago I argued that the development of logistics/supply chain management (SCM) as a discipline was being impaired by the relatively low ranking of specialist journals in this field. I was surprised and heartened by the favourable response I received both from logistics/SCM researchers and academics in other disciplines experiencing a similar problem. I have now returned to the journal ranking debate with a sequel to my original article which reviews recent literature on the subject, analyses new data on the validity of the journal ranking as an indicator of research quality and discusses the recalibration of logistics/SCM journals since 2010/11. The literature challenging the principle, practice and application of journal ranking has been steadily expanding and becoming more critical. Regrettably this is not deterring university managers from basing many recruitment, promotional and resource allocation decisions on the rating of journals. Data generated by the UK government’s assessment of university research (REF) has confirmed that, in the field of business and management, the journal ranking is an unreliable predictor of the quality and impact of an individual journal paper. In this analysis, papers published in lower ranked journals tended to be under-valued, a finding of particular relevance to logistics/SCM journals as they tend to be on the 2nd or 3rd tiers of the major journal lists. Since 2010/11, there has been some overall improvement in the relative standing of these journals, though a couple have been downgraded in the widely-used ABS list. Fortunately the backlash against journal rank “fetishism” has begun with bottom-up campaigns such as DORA and top-down, government-led initiatives in countries such as the UK and Australia aiming to make research assessment fairer, more transparent and more rigorous.
Alan McKinnon is Professor of Logistics in Kühne Logistics University, Hamburg and Professor Emeritus at Heriot-Watt University, Edinburgh. You can find out more about his research and publications at www.alanmckinnon.co.uk and follow him on Twitter @alancmckinnon.
McKinnon, A.C. (2017). Starry-eyed II: The Logistics Journal Ranking Debate Revisited. International Journal of Physical Distribution & Logistics Management, 47 (6). DOI: 10.1108/IJPDLM-02-2017-0097
Impact Factors of Supply Chain Management Journals
Note: The following text refers to a previous version of the JCR impact factors. A more recent version are the 2018 JCR impact factors (see there).
Few days ago, Thomson Reuters published the 2015 impact factors of well-known management journals as part of their Journal Citation Reports. Two SCM-related journals have an impact factor of 4 or larger: Journal of Supply Chain Management and Journal of Operations Management. Two other journals have an impact factor between 2.5 and 3: Supply Chain Management: An International Journal and Journal of Purchasing & Supply Management. Journals with an impact factor between 2 and 2.5 are: Journal of Business Logistics, Transportation Research Part E, International Journal of Operations & Production Management, International Journal of Physical Distribution & Logistics Management. Journals with an impact factor between 1.5 and 2 are: Manufacturing & Service Operations Management and Production and Operations Management. Among the journals with an impact factor between 1 and 1.5 is: Decision Sciences. Journals with an impact factor below 1 are: International Journal of Logistics: Research & Applications, International Journal of Logistics Management and Interfaces. However, keep in mind that journal rankings have a downside and should not be the dominating criteria for judging the value of our research. Financial Times has recently decided to include M&SOM rather than JSCM on the FT journal list which indicates that their list is not reliable at all to make SCM faculty decisions. Qualitative rankings such as VHB-JOURQUAL can be a good supplement to quantitative impact factors.
2016 SCM World University 100
This week, SCM World have published their “SCM World University 100” ranking, which aims to list the best business schools for supply chain talent worldwide. According to this ranking, the top 5 universities worldwide are: (1) Michigan State University, (2) Penn State University, (3) University of Tennessee, (4) Massachusetts Institute of Technology, and (5) Arizona State University. I am very pleased that Copenhagen Business School’s SCM program was selected as one of the top 10 programs in the EMEA region. The ranking is based on survey data collected from more than 2,000 supply chain professionals during the last couple of months. In the survey the participants were asked: “As a marker of supply chain talent, please select your top three universities.” Hereby, the respondents could select from a list of 192 universities that are known to offer supply chain management within their business programs. But be careful: Always keep in mind that there are both intended and unintended consequences of such rankings.
Ranking SCM Journals: VHB-JOURQUAL 3
The German Academic Association for Business Research (VHB) has published JOURQUAL 3, a journal ranking based on the ratings of more than 1,100 VHB members. A-graded SCM journals are: Production and Operations Management and Journal of Operations Management. Several SCM journals received the grade B, including: International Journal of Production Economics, International Journal of Physical Distribution & Logistics Management, Journal of Supply Chain Management, Journal of Business Logistics, International Journal of Operations & Production Management, Journal of Purchasing & Supply Management and Supply Chain Management: An International Journal. Finally, International Journal of Logistics: Research and Applications, International Journal of Logistics Management and Logistics Research received the grade C. No SCM journal received the best (A+) or the worst grade (D). Qualitative rankings such as this list or the ABS ranking can be a good supplement to quantitative rankings based on impact factors. Always keep in mind that journal rankings have a downside and should not be the dominating criteria for judging the value of our research.
Ranking SCM Journals: ABS Academic Journal Guide 2015
Note: The following blog post is about an older version of the Academic Journal Guide (“ABS list”). The 2018 AJG is discussed in another blog post (follow this link).
The UK-based Association of Business Schools (ABS) has published its Academic Journal Guide. It is the successor of the often criticized Academic Journal Quality Guide. And this is how the new Guide ranks supply chain management journals: The only grade 4* (“excellent”) journal is: Journal of Operations Management. Other “top journals” (grade 4) are: International Journal of Operations & Production Management and Production and Operations Management. Examples of “highly regarded” journals (grade 3) in the list are: Journal of Supply Chain Management and Supply Chain Management: An International Journal. Some other “well regarded” journals (grade 2) are: International Journal of Physical Distribution & Logistics Management, Journal of Business Logistics and Journal of Purchasing & Supply Management.
Given the low grades of some journals with high impact factors and considering their reputation in our field, I am not convinced of the quality of this new ABS list. For example, in spite of its reputation as a leading SCM journal and its higher impact factor, ABS ranks JBL two (!) grades lower than IJOPM. Another ranking, VHB-JOURQUAL, seems to reflect the theoretical and methodological breadth of our discipline much better – maybe because it is based on the opinions of several hundred business researchers rather than an expert panel like in the case of ABS. However, qualitative rankings like ABS and JOURQUAL can be a good supplement to quantitative rankings based on impact factors.
But always keep in mind that journal rankings have a downside and should not be used as criteria for judging a researcher (they can only be used for judging a journal, in fact). I fear that the new ABS ranking will serve as exactly such a criterion in many business schools now. Isn’t the quality of our own articles a much better criterion than the average quality of all articles published in a journal (including the very bad and very good ones)? But this would require the members of an appointment committee to read what the candidates have actually published – maybe too much of an effort? And, if paradigm shifts often start in low-ranked journals, should our incentive system really prevent us from publishing in journals with ABS ranks below 3?
