Perceptions of journal quality and research paradigm: results of a web-based survey of British accounting academics

Abstract

This paper reports the results of a web-based perception study of the ranking of peer reviewed accounting journals by UK academics. The design of the survey instrument allows an interactive selection of journals to be scored. The web-based format is unique in that it also includes a step in which respondents classify the journals according to methodological perspective (paradigm). This is depicted graphically in the paper in a bubble diagram that shows the “positioning” of journals according to perceptions of both paradigm and quality.

Introduction

The survey that this paper reports uses a web-based, interactive questionnaire to elicit the views of academics in British accounting and finance departments on journal paradigm and quality. The use of a web-based delivery system for the survey allows the respondents to interact with the site to select journals from a prepared list or enter additional journals and classify them by paradigm and then score those journals for quality. This provides a focus on journals with which respondents are familiar (Brown & Huefner, 1994; Zeff, 1996) and incorporates the additional element of the classification by paradigm¹ which is seen as important in the face of growing differences in the style of research conducted and published by accounting academics (Bricker, 1989; Williams & Rodgers, 1995).

Research suggests that even the rankings of well-known journals vary by region (Ballas & Theoharakis, 2003; Brinn, Jones, & Pendlebury, 2001; Locke & Lowe, 2002; Zeff, 1996). In the UK the importance of this regional effect is heightened by Brinn et al.'s (2001) finding that accounting and finance academics believe that for the purposes of the Research Assessment Exercise (RAE), “[p]ublication in top UK research journals was perceived to be the greatest indicator of research quality” (p. 333). The authors express surprise that the UK journals are seen as more important (even if only marginally) than the `top' US journals. They contrast this to an earlier study (Brinn, Jones, & Pendlebury, 1996) which reported that British academics consistently ranked top US journals well ahead of the top UK journals in terms of research quality. There are possible explanations, such as: perceptions of journal quality having changed since their 1993 survey (published in 1996); or as they speculate, that academics believe that factors other than quality may be considered as influential in the RAE evaluations.

Otley (2002) reviews the 2001 RAE and makes a number of interesting comments regarding possible future approaches to the Exercise. While highlighting the strength of the current procedure's reliance on quality assessments by subject experts, he does raise the possibility that using more mechanistic approaches could reduce the cost and the burden on institutions and panel members (p. 401). The possible alternative approach he described as, “counting the number of research outputs and weighting these by quality ratings of journals or citation counts.” (p. 401). Even if this explicit approach to using journal quality as a weighting is not adopted in the future, the indication is that it has an implicit impact in the evaluation, as would be expected. Professor Otley also notes that;

In general the work examined tended to be concentrated on those items which had no external `imprimatur' of quality, such as books, research reports and working papers. Nevertheless, a considerable number of journal articles were also read in detail. (p. 391)

Implicit understandings of what is a quality journal would appear to have a role to play in these assessments.

Consensual evaluations of journal quality have the potential to impact on evaluations of research quality and as a result on the outcomes of the RAE process, individuals' promotion prospects and publishing strategies. Given that the last published survey was conducted in 1993, this suggests that it is timely to reassess perceptions of academic accounting journals in Britain. We compare our results to Brinn et al.'s (1996) findings and note some broad similarities but also some key differences for individual journals. In addition we identify some underlying factors driving the perceptions of the journals.

The paper is further divided into three broad sections. The next section reviews the literature related to journal ranking studies and highlights the key aspects of the method used in this research. The results of the survey and key findings in relation to the dimensions of paradigm and score are analysed, considering specifically a comparison to an earlier journal ranking study and specific demographic factors linked to the results. Finally some concluding comments are offered.