In her insightful Nature comment Rein in the Four Horsemen of Irreproducibility, Dorothy Bishop describes how threats to reproducibility, recognized but unaddressed for decades, might finally be brought under control, by avoiding what she refers to as “the four horsemen of the reproducibility apocalypse”: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known). In the video below she makes several important points. My perception is that the SCM research community does not take the reproducibility debate seriously enough.
Today, I would like to call attention to a highly-cited article by Baldwin & Lopez-Gonzalez, titled Supply-chain Trade: A Portrait of Global Patterns and Several Testable Hypotheses, which was published in The World Economy in 2015. The journal’s perspective – trade policy and other open economy issues – differs from the supply chain management perspective I normally talk about here, which gives this article an interesting complementary perspective. The authors use the term “supply-chain trade” to characterize “complex cross-border flows of goods, know-how, investment, services and people”. They compare two positions: “According to policymakers [supply-chain trade] is transformative; among economists, however, it is typically viewed as trade in goods that happens to be concentrated in parts and components”. Based on two rich datasets, they argue “that the facts are on the side of the policymakers”, as “[f]lourishing supply-chain trade has revolutionised global economic relations and the revolution is still in full swing”. Definitely a good read!
Baldwin, R. & Lopez-Gonzalez, J. (2015). Supply-chain Trade: A Portrait of Global Patterns and Several Testable Hypotheses. The World Economy, 29 (1), 65-83. https://doi.org/10.1111/twec.12189
Lambert & Cooper’s (2000) paper Issues in Supply Chain Management has certainly been one of the most influential articles of our discipline. Herein, they presented a framework for SCM as well as questions for how it could be implemented. The framework contained a set of cross-functional, cross-organizational business processes that could be used as a way to manage relationships with customers and suppliers. The article continues to be an important cornerstone in research on the topic of integration. Now, more than fifteen years later, Lambert & Enz (2016) present an updated version, Issues in Supply Chain Management: Progress and Potential. Herein, the authors “review the progress that has been made in the development and implementation of the proposed SCM framework since 2000 and identify opportunities for further research”. Interestingly, they have changed their minds about some statements made in the 2000 article, for example that competition is no longer between companies, but between supply chains, which they now argue is not technically correct. The authors also present a revised version of the framework from 2000.
Lambert, D.M. & Cooper, M.C. (2000). Issues in Supply Chain Management. Industrial Marketing Management, 29 (1), 65-83. https://doi.org/10.1016/S0019-8501(99)00113-3
Lambert, D.M. & Enz, M.G. (2016). Issues in Supply Chain Management: Progress and Potential. Industrial Marketing Management, 62, 1-16. https://doi.org/10.1016/j.indmarman.2016.12.002
Claviate Analytics have recently published their newest InCites Journal Citation Reports. It is great to see that the 2018 impact factors of all but one journals related to supply chain management have increased again, which highlights the rapidly growing relevance of our discipline. Two journals have an impact factor larger than 7: Journal of Operations Management (7.776; +2.9) and Journal of Supply Chain Management (7.125; +1.0). With an impact factor larger than 5, International Journal of Physical Distribution & Logistics Management (5.212; +1.0) has now arrived in the first league of management journals. Other SCM-related journals with high impact factors are: Supply Chain Management: An International Journal (4.296; +0.5), Management Science (4.219; +0.7), International Journal of Operations & Production Management (4.111; +1.2), Journal of Business Logistics (3.171; +0.3) and Journal of Purchasing & Supply Management (3.089; −0.6). But there are even more SCM journals with an impact factor around 2: Manufacturing & Service Operations Management (2.667; +0.9), Operations Research (2.604; +0.3), International Journal of Logistics Management (2.226; +0.5), Production and Operations Management (2.171; +0.4) and Decision Sciences (1.960; +0.3). Although the impact factor is certainly an imperfect proxy of a journal’s quality, I can only hope that rather conservative qualitative rankings, such as the ABS-AJG list, the UT Dallas list or the FT50 list, will finally be adapted to this new reality. This step is urgently needed!
Three years ago, the Chartered Association of Business Schools (ABS) has released its last ranking of business journals: the Academic Journal Guide (AJG), also known as the “ABS list”. This ABS ranking has become quite influential as the guiding journal ranking across management disciplines in the UK. Although the ranking has been heavily criticized (see my previous post) and more democratically-developed rankings exist (e.g. VHB-JOURQUAL), the ABS ranking has since been adopted by many business schools also in other countries.
Shortly after the publication of the last ABS list, Nature has published ten principles to guide research evaluation, which have since become known as the Leiden Manifesto. One of these principles argues that quantitative evaluation should support qualitative, expert assessment, not replace it. Similarly, the San Francisco Declaration on Research Assessment (DORA), which has been signed by thousands of researchers worldwide, asks “not [to] use journal-based metrics […] as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions”. The Community for Responsible Research in Business Management (cRRBM) questions whether “even the academy is being served when faculty members are valued for the quantity and placement of their articles, not for the benefit their research can have for the world”. Indeed, the negative consequences of rankings are well-documented (see, for example, Espeland & Sauder, 2007 and Grant & Kovács, 2018).
Unfortunately, the ABS list has often been used in exactly the way the Leiden Manifesto and DORA wanted to prevent us from doing – with very negative consequences for our discipline. Many SCM researchers feel that the ABS list undervalues the journals of our discipline. For example, Journal of Supply Chain Management, a journal with one of the highest impact factors in management (5.789), is ranked by ABS 2015 as a “3” only and Journal of Business Logistics as a “2”. Comparing our journals to other disciplines, like accounting or marketing, it becomes apparent to me that these journals deserve a “4*” and “4”. Worse even, many SCM journals have suffered from their low ABS rankings, as SCM researchers who took the ABS ranking too seriously, felt they should publish in the higher-ranked journals of other disciplines. AJG 2015 has thus become a self-fulfilling prophecy.
Today, the Academic Journal Guide 2018 has been released. It contains more than 1,500 entries. Unfortunately, AJG 2018 did not adapt the ranks of leading SCM journals. When it comes to empirical journals, only Journal of Operations Management gets a “4*”, while Decision Sciences, Journal of Supply Chain Management and Supply Chain Management: An International Journal get a “3” again. International Journal of Physical Distribution and Logistics Management, Journal of Business Logistics and Journal of Purchasing and Supply Management all still get a “2” only. The fact that the rank of none of these journals has changed indicates that the new ranking does not seem to cover all the changes and improvements our discipline has made in the last couple of years. Only four out of eight journals (four empirically- and analytically-focused journals each) that have been identified by leading SCM schools as the top journals of our discipline (sometimes informally referred to as “the SCM basket of eight”; see The SCM Journal List), got an AJG rank higher than 3. My suggestion would be to ignore AJG 2018, as it does not seem to represent the journals of our discipline properly.
Interestingly, even the Chartered ABS itself just tweeted that the REF2021 sub-panel will ignore their own list, so they might well be aware of its shortcomings:
#REF2021 sub-panel chair: “We will not be using metrics. Our role is to assess originality, significance and rigour. We will not use #AJG2018 to do so.” Robert Blackburn #ARC2018
But there is some hope: This latest iteration of AJG was an interim review, the main purpose being to include new journals. The next iteration, planned to be published in 2021, will then be a major review. This is certainly very late, but better late than never: Careers can depend on such lists. Therefore, our discipline urgently deserves a better quality of the AJG. I hope the team behind the list is aware of their responsibility.
There has been a recent trend in several management disciplines, including supply chain management, to create knowledge by systematically reviewing available literature. So far, however, our discipline lacked a “gold standard” that guides researchers in this endeavor. The Journal of Supply Chain Management has now published our new article, Durach, Kembro & Wieland (2017): A New Paradigm for Systematic Literature Reviews in Supply Chain Management. Our systematic literature review process follows six steps: (1) develop an initial theoretical framework; (2) develop criteria for determining whether a publication can provide information regarding this framework; (3) identify literature through structured and rigorous searches; (4) conduct theoretically driven selection of literature and a relevance test; (5) develop two data extraction structures, integrate data to refine the theoretical framework, and develop narrative propositions; and (6) explain the refined framework and compare it to the initial assumptions. We believe that these best-practice guidelines, although developed for the SCM discipline, can be used as a blueprint also for adjacent management disciplines.
Durach, C.F., Kembro, J. & Wieland, A. (2017). A New Paradigm for Systematic Literature Reviews in Supply Chain Management. Journal of Supply Chain Management, 53 (4), 67-85. DOI: 10.1111/jscm.12145
In a previous post, it was demonstrated that researchers can play two different roles, as they can either build or test theories. An SMJ article by Miller and Tsang (2011), which is titled Testing management theories: Critical realist philosophy and research methods, focuses on the latter role we can play. The authors claim: “Not only do we have a plurality of theories within management research, there is also no consensus about the criteria for evaluating theories.” Taking a critical realist perspective, they advance practical guidance for evaluating management theories by proposing a four-step approach to theory testing. This approach includes (1) identifying the hypothesized mechanisms, (2) testing for the presence of the mechanisms in the empirical setting, (3) testing isolated causal relations, and (4) testing the theoretical system. The authors underline that “steps 2 and 3 have been neglected for the most part”. In sum, a lot can be learnt about theory testing from this brilliant article.
Miller, K., & Tsang, E. (2011). Testing management theories: Critical realist philosophy and research methods. Strategic Management Journal, 32 (2), 139-158 DOI: 10.1002/smj.868