1. Cards on the table

To put my cards on the table straightaway, I must state that I myself have not been a genuine victim of the peer review process. I have after all managed to shepherd about 200 books and papers into print, and some of the time the review process has been enormously productive and insightful. My own debt to editors and reviewers is beyond calculation.

Mind you, the going has not been entirely peaceful. Many submissions of mine have received at least one scathing rejection where the reviewer went to great lengths every means to prove how abysmally unpublishable my work must be. Indeed, the negative reviews were most often by far the longest. Only occasionally were my submissions brushed off by reviewers giving brief standby rebukes such as ‘failed to review the relevant research literature’ or ‘contributes nothing new’. On two occasions, my submission was rejected unread on the grounds that I was a professor in a Department of English and as such (one reviewer said) could not have ‘a deeper understanding of the issues’ — including, as it happened in this case, issues which I myself had introduced into the literature.

On the one hand, all this may be just business as usual. Reviewers may feel a professional calling to be censorious or captious. Some editors have told me they always send a paper to one reviewer who will despise it, on the logic that every imaginable flaw will get hunted down. I confess this logic eludes me, since a more probable tendency would be to exaggerate flaws that are of no real consequence or even to fabricate ones. Worse yet, the logic frankly undercuts the vaunted notion of ‘objective reviewing’, to which I shall return in section 2.

I must also admit that my submissions are not exactly the bland, conventional disquisitions one expects in a ‘normal science’ — say, in literary studies on ‘pessimistic irony in Thomas Hardy’, and in linguistics on ‘subject deletion rules in passive relative clauses’. For reasons that only gradually became quite clear to me, I did not regard either literary studies or linguistics as ‘normal sciences’, but rather as denatured enterprises harbouring complacent and limiting preconceptions about what constitutes ‘research’. I could plainly see that neither of them was genuinely relevant to what I considered the vital issues of the times, such as literacy, language teaching, and user-friendly communication. So I wrote quite frankly about why these issues should concern us and why our conventional wisdom was standing so uncomfortably in the way.

Now, if the peer review process really served the defensive manoeuvres of a ‘closed shop’ or ‘old-boy network’, as is alleged around late-hour cash bars at professional conferences, then I could hardly have gotten into print at all. I need no reviewers come from the gravely dignified halls of academe to tell me how painful or disturbing my work might be to some of my colleagues. I am sorely troubled by the prospect of implying that their entire careers and research have been based on serious misconceptions of ‘language’ or ‘literature’. But if the conclusions I draw from sustained rational examination of the evidence do foster such implications, then theses need to be faced; and, if I have misjudged, I ought to be told how and why in the same terms.

So if I do get printed and read, then surely because many colleagues are also uncomfortable about the self-imposed limitations in their own fields. They face much the same practical problems as I do in their daily work, and they do not relish being told by ‘distinguished experts’ that their problems ‘are of no theoretical interest’ (to quote a favourite dismissal of Mr Chomsky’s). In the years since 1975, when my work began in real earnest, the fields of language research have undeniably undergone a momentous liberation, though I cannot tell how far my work has contributed.

Finally, I must state that I have acted as a reviewer for some nine editorial boards and, in an unofficial capacity, for as many more. Perhaps I have not been a very typical one, though; I see my function not so much as passing judgement on what does or does not deserve to be published, period, but as advising would-be authors who face an intensely personal and professional urgency of getting their work into print. Aside from a small number of submissions where I was stumped, I have either approved the submission with specified revisions or else given advice on how to make it approvable. I have also been quite active in helping colleagues revise their own work before they submit it in the first place.

I mention these factors to indicate why my reservations against peer review cannot be explained from feelings of personal resentment. My motivation is rather that peer review entails some ‘pre-modern’ conceptions of ‘science’, ‘research’, ‘significance’, and so on, as well as a ‘pre-scientific’ model of writing and reading. Based upon my own work in critical discourse analysis, I have hit upon a disquieting surmise that the process may be fundamentally flawed; it may actually hinder rather than promote the genuine goals of academic research (as I understand them), and may drive a wedge between the official theories of reviewing and its concrete practices recorded in the discourse of reviewers.

I shall first present my lines of argument; then summarise some seminal studies; and finally examine some authentic data my own submissions have attracted.

2. Gate-keeping in the ‘academic world’

The ‘academic world’ is exquisitely sensitive to the distinction between insiders and outsiders. This world must seem strangely sheltered and insulated from society at large if its main function is to produce knowledgeable citizens in that society. But the strangeness disappears if its main function is rather to set the knowledgeable citizens distinctly apart from the rest of society. If so, the main value of an academic ‘education’ would be to claim a status of higher prestige and larger rewards than the ‘uneducated masses’. This function could also explain the common usage of the term ‘academic’ to mean: purely concerned with the university ambience as opposed to everyday life.

At all events, the academic world is well-fenced both externally from the outsiders and internally within an elaborate systems of ranks and titles for the insiders. In consequence, a fair share of its activities are concerned with gate-keeping: exerting careful control upon who is or is not allowed to enter where. Academic students confront standardised tests, entrance requirements, group requirements, degree requirements, and eventually the defence of a thesis or dissertation. Academic staff in turn confront byzantine procedures for being hired, evaluated, assessed, and promoted.

As the universities have grown exponentially in the later 20th century, so too have the methods of gate-keeping. Academic staff can no longer feel competent and qualified merely by holding a ‘master’s or doctor’s degree’ and ‘performing our normal teaching duties’. We are also expected to ‘produce and publish research’, and, wherever remotely possible, to ‘generate outside funding’, which ironically seems to mean asking for extra money to do the jobs we are supposed to be doing anyway.

An academic field might sustain these pressures reasonably well if its research procedures are both clearly defined and materially demanding, as is true for particle physics, inorganic chemistry, or genetic biology. But matters are utterly different in the two main fields in our language programmes, namely ‘literary studies’ and ‘linguistics’. Although for different reasons, both fields have nurtured a strongly anti-empirical streak. So they have been turned back upon themselves, steadily piling more language upon language, more discourse upon discourse, interpreting interpretations, criticising criticisms, and so forth. This outpouring duly floods into the outlets for publication, such as journals, edited volumes, book series, and conference proceedings, who must respond with their own gate-keeping.

Their major strategy is known as ‘peer review’. Though it is a huge business today, its history is nearly as long as the practices of academic publishing themselves (cf. Bazerman 1988 for a historical account). Its origins might be appreciated in an ambience where a share of academic writing was produced by aspiring amateurs. Right up into the 19th century, philosophy and science were cultivated in part by ‘educated gentlemen’ who regarded their own products with uncritical admiration, as when Goethe reportedly esteemed his work on the ‘theory of colours’ above his literary efforts.

So gate-keeping by peer review began when the ‘academic world’ was still relatively open and fuzzy at the edges. The most logical gate-keepers would be colleagues whose distinguished credentials seem unimpeachable, and they have been tapped as ‘editors’ and the ‘editorial boards’. They remain our proud heritage today despite the extent to which the academic world has been thoroughly professionalised and amateurs are effectively excluded. Accordingly, the gate-keeping function has been retargeted for ‘upholding standards’ and ‘ensuring quality’ among the submissions of authors who are already certified to be academics by their education, advanced degrees, or staff positions. The world of insiders has been regrouped by exquisitely precise ranking.

Our heritage also includes at least three implicit problems. Our first problem is that the procedures for establishing ‘distinguished credentials’ are unsystematic. In general, you will need the hearty approval of established gate-keepers, and obtaining it encourages you in your early career to declare support for their work and reputations and to postpone your own quest for originality and creativity. In a kind  of Catch-22, the aspiration to become distinguished requires you to appear so before you are. Some gestures of premature self-advertisement may be needed to create a reputation of authority you hope to grow into later. Yet such gesturing is a delicate operation, since academic decorum stipulates that authority should be claimed only once it has been genuinely earned.

Our second problem lies in the precarious notion that the merits of a work are clearly determined by its inherent qualities. This logic is needed to ensure that a competent gate-keeper can pass an objective judgement upon those merits independently of his or her own commitments to particular theories and methods within the academic field. Today, such a logic implies a distinctly ‘pre-scientific’ model of the reception and comprehension of discourse, now that extensive research has demonstrated how deeply one’s own prior knowledge and attitudes are implicated in every production of meaning. Your ‘objectivity’ toward meanings which do conflict with your own commitments must be produced after the fact by a process of adjusting and rebalancing. But you are prone to interpret the conflict as a violation of academic standards that justifies you in rejecting the work.

As if to assist objectivity, some fields or journals stipulate a standard format for submissions, with sections like ‘methods’, ‘results’, ‘discussion’, and ‘conclusions’. Reviewing can then focus on routine concerns, e.g., by checking whether the ‘methods’ were ‘sound’ or the ‘results’ were calculated with the ‘proper statistics’. In return, such formats tend to freeze out or conceal the subjectivity of the authors as implicated in the choice of topic or the interpretation of human significance — what Neisser has dubbed ‘ecological validity’ — which is often worlds apart from ‘statistical significance’. I have seen much experimental research on topics chosen just to accommodate standardised methods, e.g., on the recognition and recall of word lists whose relevance for real-life language activities was nowhere established.

Our third problem lies in the equally precarious notion that an established authority in a field is also an expert reader of works about the field. Undeniably, much of our academic activity is in fact centred upon the production and reception of discourse. But this fact is rarely acknowledged among academics who prefer to see themselves ‘performing experiments’, ‘making discoveries’, ‘exploring nature’, and so forth. This preference might explain why the production and reception of discourse are not accredited as subject-matters in their own right during the education of future academics in the sciences. Instead, the trainees are expected simply to absorb these matters as a by-product or sidelight of their ‘real’ initiation into the content of a the field. By the same logic, the completion of their education is widely accredited to qualify them as teachers in the field without any serious study of pedagogical method or technical communication.

Again as if to assist objectivity, reviewers are officially asked to address a set of standardised questions, notably whether the submission ‘makes a significant contribution to the field’ and whether ‘the style is appropriate to the subject-matter’. The widespread imposition of these questions seems to prevent us from noticing how problematic they are. The ‘field’ is apparently regarded as a monolithic whole with specifically marked areas of measurable ‘significance’. And the ‘appropriateness’ of a ‘style’ is treated as a matter of general consensus, even though it has not been successfully defined even in the field of stylistics. So being chosen an editor or a board member must automatically confirm you as an expert on ‘significance’ and ‘style’.

All these problems may actually be compounded by the anonymity of the reviewers. A displaced mode of communication ensues between the discourse of the submitting author and the ‘meta-discourse’ of concealed reviewers. Despite the standardised questions, reviewers retain substantial latitude in passing comments without having to take personal responsibility for them; I have received plenty of offhand derogatory comments the same people would not have made to my face. These cast discredit not so much upon me as upon the process of peer review, especially when editors straightfacedly accept them as grounds for a rejection.

The official logic must be that anonymity encourages frankness and security. By that same logic, authors are expected to react to negative comments with vengeful machinations. I believe we authors are more professional and mature than all that, and would be much happier conducting open dialogues with our reviewers than being shot at out of the darkness.

Our problems may be further compounded by the practice of blind reviewing, which imposes anonymity on the author’s side. Editors will affirm that this process ensures a selection of works standing on their own merits and not just on the reputations of well-known authors. Editors will not affirm the corollary of this logic, namely that well-known authors can be mistrusted of abusing their reputations by palming off substandard works. Still less will editors affirm the next corollary, namely that some authors can get to be well-known by temporarily suppressing their proclivity for writing substandard works, and then relapsing.

The implications of mistrust apply to reviewers as well. By a parallel logic, they should be prone to discriminate heavily according to the authors’ reputations. So they need to be partly blindfolded, as if they would not be objective if they came into possession of sensitive information. Trust is fully accorded only the editor to remain unaffected by knowing the authors’ identities.

To some degree, blind reviewing may be a fiction anyhow. I have seldom ‘blind-reviewed’ a work where I could not guess the identity of the author. Well-known authors can be guessed style and content; little-known authors from their assiduous citation of their own works in recent and obscure sources, such as conference presentations or unpublished theses or dissertations.

And even if authors do remain anonymous, ideas and theories do not, especially in a clannish field like linguistics, where the reverent citation of authority figures like Saussure, Bloomfield, or Chomsky is a standard routine. Authors are expected to position themselves within the canon of influential works and thus to assign themselves an ancestry and loyalty that deconstruct any personal anonymity they might sustain.

Moreover, an obvious and fundamental flaw in the logic of blind reviewing lies in implying that authorship is somehow irrelevant to academic communication at large. The name of an author is undoubtedly a leading factor when readers decide what is worth their attention. Yet the gate-keepers are asked to ignore that this same factor in their own decisions, as if readers will be compelled to read a work not by its author but by an inherent merit which they can somehow register in advance of the reading itself.

Carried to extremes, blind reviewing could unravel the interpersonal dynamics that make an academic field into an communal enterprise. It would project a world of disembodied voices and dehumanised ideas competing in an ‘academic marketplace’ of nameless agents. Indeed, a logical corollary might be to publish the contributions anonymously as well, so that everyone’s response would be as ‘objective’ as that of the ‘blind reviewers’. I feel reminded here of the occasional collectives whose work bore just the name of a group, such as ‘Bourbaki’ in mathematics; but the group’s name would impede ‘blindness’ and would have to be deleted too.

Jeopardising the interpersonal dynamics is also likely to undermine the coherence and continuity of the individual contributor. Each of my new works is naturally conceived within the discursive space of my previous writings. Since drastic limitations upon journal space positively preclude my reconstructing any cumulative argumentation, I must be content to invoke at least some prior foundations. I have for instance written at length to deconstruct the dichotomy whereby a ‘linguistic theory’ is established through a radical idealisation and abstraction of ‘language’ (or ‘langue’ or ‘competence’) away from ‘text’ (or ‘parole’ or ‘performance’). So I now take this deconstructive work as a point of reference in my further steps to show how the ‘text’ is not just an item within ‘parole’ or ‘performance’ (as is widely assumed), but rather a dialectical event that mediates between langue and parole or between competence and performance (Beaugrande 2000).

Or again, I have written extensively to deconstruct the dichotomy whereby ‘theoretical linguistics’ and ‘applied linguistics’ constitute two institutionally segregated enterprises. So I adopt this point of reference for working out a dialectic wherein theories are expressly designed for applications, whilst the outcome of applications is fed back into the requirements of the next generation of theories.

But when I submit a paper, the ‘anonymous blind reviewer’ may be oblivious of my points of reference and so reject my further steps for being ‘unsubstantiated’ or ‘unproven’ (cf. section 4). Many reviewers have apparently never thought of questioning the basic dichotomies within linguistics, and are by no means eager to publish writings which do so unless an extensive justification is given — which is just what I am denied, due to ‘limitations on space’.

Coherence is further impeded by the decorum that discourages reviewers from honestly admitting that their rejection of an unconventional work stems from their personal commitment to protect conventional notions and vested interests against public challenge. Any such admission would deconstruct the stance of an ‘objectivity’ that eliminates bias and welcomes ‘significant contributions’ to scientific progress.

Yet I have seen much solid evidence that, in my case anyway, such are precisely the real grounds for numerous rejections. I have repeatedly encountered reviewers who display no intention of allowing my challenges to be placed before the field for an open and rational discussion of pro and contra. They prefer to stifle dissent at the source, totally against the spirit of genuine research and exploration.

I would not suggest that these just happen to be the ‘less qualified’ reviewers among an otherwise objective community. No doubt they see themselves as high-minded defenders of ‘standards’. Instead, I would call in question the basic conception that ‘standards’ can be independent of personal and institutional commitments. A field is most precarious when no rational procedures of validation are practiced and acceptance or rejection is decided on the basis of assertion and argumentation. And I fear this situation holds in the more anti-empirical schools of literary studies and linguistics. Indeed, the latter field has excelled in devising complex arguments to demonstrate that a ‘theory’ or ‘hypothesis’ need not or even cannot be verified from authentic language data (cf. Beaugrande 1991, 1998 for actual sources). Prior commitments assume such a disproportionate role that some journals constitute sheltered reserves for insider groups, such as those who hold allegiance to a ‘generative’ trend like ‘government and binding’.

3. Empirical studies of the review process

Understandably enough, the review process has not been a frequent object of empirical studies. In the past, no academic field would have offered a congenial home. Nor would a field want to interrogate a process so intimately connected to its own methods of self-justification.

Yet eventually, reading research hit upon professional reading as a topic of potential concern. One logical data source would be the journal Behavior and Brain Science, which publishes each article along with commentaries by distinguished experts plus the author’s responses to these. (I was once invited, but at the author’s emphatic objection, quickly ‘ex-vited’ again.)

This journal was closely studied by Scardamalia and Bereiter (1991) in the course of their research on ‘expert readers’. They found the authors of an original paper frequently accusing the expert respondents of having misread the article. The ironic implication would be that the journal’s editors read the paper correctly and approvingly, whereas the respondents were unable to do the same, despite being accredited authorities in the field. By this irony, your own authority as an academic expert would be measured by whether you read the contribution approvingly, rather than the quality of the contribution being measured by whether you approve it.

If authors deny that the reservations of the respondents were motivated by real problems or defects in the original paper, then disapproval could only be a result of misreading. Apparently, these authors did not hold a high opinion of the process of critical commentary, which should logically apply to peer reviewing as well.

In a more subversive study, Peters and Ceci (1982) made cosmetic changes to 12 published research articles in psychology, replaced the names of prestigious authors and home institutions with fictional, obscure-sounding ones (e.g. ‘Dr. Wade Johnson of the Tri-Valley Center for Human Potential’), and resubmitted them to the identical high-profile professional journals that had published them some 18 to 32 months earlier. Only 3 articles were recognized, while 8 out of the other 9 were rejected almost unanimously on the grounds of ‘methodological flaws’.

To explain such startling results, several hypothesis would be logically possible. Perhaps the papers did not deserve to be published the first time around. Either they simply slipped through a superficial review process, or the reviewers were discriminating in favour of renowned authors or home institutions. The second review was more thorough and detected the real defects that were there all along.

Then again, perhaps the papers did deserve to be published but, the second time around, encountered a superficial review process, or the reviewers were discriminating against obscure authors or home institutions. Either way, the papers got faulted for imaginary defects.

Or again, perhaps the reviewers on the second round were looking much harder for ‘flaws’ because their journals had accumulated a heavy backlog of accepted items awaiting publication. They could have tightened up their criteria and cracked down on ‘flaws’ that would not have seemed to justify rejection under other less congested conditions.

Or yet again, perhaps the editorial boards had undergone some major shift in perspective in the intervening period, due to a change among ‘scientific paradigms’. The papers themselves were not to blame but fell victims to ‘scientific progress’. This hypothesis is by far the most reassuring one. But it stretches our credibility to assume that all the journals could have changed in the same way and in so short a time period.

Finally, the null hypothesis would be that acceptance or rejection fluctuates massively and independently from either the quality of submissions or the paradigms of a field. If so, the same operation might just as well be repeated with a high acceptance rate next time, rather like a lucky throw of dice.

Fittingly enough, the findings of Peters and Ceci appeared in Behavioral and Brain Sciences, along with replies from no less than 56 respondents, mostly journal editors or reviewers or else investigators of review bias. Consensus was certainly not prominent. Some respondents castigated Peters and Ceci for shamefully unethical conduct and vowed that such studies must not be performed. Some admitted a concern over carelessness, institutional bias, or scientific incompetence. And some proffered harmless or mechanical explanations, e.g.: high rates of rejection and disagreement among reviewers are normal factors in peer review; the statistical probability of a given reviewer having read the same article before was not very high; the articles were actually recognized and tacitly rejected on those grounds because suggesting plagiarism seemed too drastic; citing ‘methodological flaws’ is the routine catch-all grounds for rejection in experimental psychology.

The respondents were not eager to draw the conclusion that the prominent academics chosen for editorial boards on the basis of their own research and publication are neither expert readers nor expert judges of technical prose. Their lack of expertise could explain the wide margin of inconsistency among their judgements and perhaps even their inability to recognise a disguised resubmission.

Nor were the respondents at all eager to jettison the notion that the merits of a submitted work are clearly determined by its inherent qualities. Since (as I pointed out in section 2) this notion is absolutely crucial for the ‘objectivity’ of the review process, it can persist despite mountains of evidence to the contrary. Ironically, some of the foremost experts in science still cling to a pre-scientific and, by today’s standards, unscientific conception of the text and the reader.

I would even raise the prospect that the achievement of academic prominence might foster a tendency to level off with convenient and complacent reading habits in the confidence of having thoroughly understood everything your field has to offer. You would readily take your own theories and methods to be the absolute measure of the ‘significance’ and ‘validity’ of any research placed before you. If that research conflicts with yours, then it must be rife with ‘methodological flaws’.

4. A case study

Here are excerpts from two responses, received around 1980, to my own submitted work from ‘blind reviewers’ commenting upon my ‘writing style’:

[1] Appropriate to the purpose, objective, lucid. The subject-matter is necessarily complex, including multiple systemic interrelationships. The writing style clarifies and exemplifies the relationships as simply and directly as necessary

[2] If I didn’t haven't review this article I would have stopped reading it shortly after I began. His/her main points are buried in a writing style that surely tested my patience, to be utterly frank. Diffuse, tiring, not to the point

I can hardly imagine two more conflicting assessments. They project totally contradictory views not merely of the same author but of the same submission. Yet the contradiction falls into place if we know the institutional ambiences. The reviewer in [1] was a professor of English Education, who also remarked that my ‘untempered rejection of sentence grammars (context-free) as avenues of understanding reading is important and, I believe correct, and in need of restatement’. The reviewer in [2] was a professor of Linguistics, who called my piece ‘polemical about linguistics, unnecessarily’. So the opposite responses were a direct product of each reviewer’s outlook on specific approaches in the field of linguistics, in this case some then fashionable sentence-based models of reading. By itself, this effect is hardly surprising; what struck me was how they were couched as evaluations of my style.

My other data for this discussion set arrived recently, in 1998. My submission was a report on a pilot course for teaching ‘business English’ at an African university. At the time, I had recently read Martin and Schumann’s (1996) compendiously researched best-seller Die Globalisierungsfalle: Der Angriff auf Demokratie und Wohlstand. The two authors, senior reporters for Germany’s top-ranked newsmagazine Der Spiegel, provide staggering overwhelming evidence of a world-wide movement by multi-national corporations and financial speculators to force down wages, avoid taxes, and override regulations for worker safety and environmental protection. Their evidence was amply confirmed by what I saw going on all around South Africa during its troubled transition to ‘democracy’ (Beaugrande 1999). So my interpretations of business English in the news media on topics like ‘privatisation’ made reference to this global context on the assumption that it would be common knowledge among teachers of ‘Business English’. They need only to pick up, say, the Los Angeles Times — hardly a subversive newspaper, you will agree — and they can read a headline like ‘Globalisation creating far more losers than winners’ (30 November 1999):

If human betterment were the object of globalisation, its instigators would have to admit that it has been a colossal failure. Market forces and unelected bureaucracies have been allowed to dictate the rules. […] Uncontrolled financial speculation in so-called emerging markets has led to disaster for the majority of the population in the affected countries.[…] Following the Mexican crisis and devaluation of 1994-95, half the Mexican population dropped below the poverty line. A year or two ago the Asian tigers were singled out as paragons of economic virtue. Today starvation has returned to Indonesia. A sharp increase in suicides has taken hold in Korea and Thailand, where workers no longer see any hope for themselves and their families.

But two of my four my reviewers seemed to be unaware of all this. My references were thus construed as arbitrary fabrications on my own part, viz.:

[3] there is a great deal of under-supported personal interpretation, fear, and judgment

[4] The bulk of the article is obtuse, wordy justification for a political position and goal. It presents strong and aggressive statements as fact with very little reference to either academic or other support except for the author's own views

[5] Verbal conclusions are inflammatory judgments and opinions stated as fact.

[6] The message I get in reading this work is that learners of ESP Business must obfuscate, must ‘sell out’, in order to get a good job.

[7] The author demonstrates a commitment to a sociopolitical position [that] Business English [is] a secret code […] designed to oppress the poor, set up insider/outsider strata and deny access to truth

To my amazement, one reviewer denied the very existence of a ‘world-wide recession’ and asserted that ‘there is no recession in Latin America’! Another edified me with this revelation:

[8] English legal language was developed by Anglo Saxons and Normans so that written texts would be mutually intelligible and succinct.

Insofar as I was indeed expressing a personal position, it has evolved in the course of my teaching, where three disturbing factors have caught my attention. One factor was the use of legal English to obfuscate and intimate ordinary consumers, as in [9]. Another was the stodgy, unhelpful advice about grammar and usage in assigned textbooks on ‘Business English’, as in [10]. And yet another was the verbose and murky style of widely used textbooks in the field of business education, as in [11], which seem to be striving for an signally ‘academic discourse’.

[9] Failing return of the product within 7 days, the Supplier’s liability shall be limited on return to Supplier of the product or parts thereof, to the replacement or repair (in sole discretion of Supplier or of its duly authorised service dealer) of the product to eliminate any defect in material and workmanship found to be due exclusively to any acts or omission on the part of Supplier of which defects Supplier shall have been notified in writing by the Customer within the aforesaid warranty period. [warranty on a space heater]

[10] choose those words which express just what you mean; […] use adjectives and adverbs with their proper meanings and correctly; […] vague words are always useless (Gartside 1990 [1969]: 25, 20-21).

[11] accounting is most widely defined as the process of identifying, measuring, and communicating economic information to permit informed judgements and decisions by users of the information (Woods 1988: 253)

But my critiques of these factors were mistaken by one reviewer for signals of my being ‘mad’ with no reason: 

[12] One wonders why he seems to be so mad at lawyers; but later, one sees he’s also mad at English departments and business faculties. […] Academic discourse takes a particularly hard hit.

Apparently, criticism of discourse practices was primitively viewed as verbal combat driven by irrational anger.

With unintentional irony, two reviewers complained that my own discourse was not academic enough. One brandished a brittle preconception of the proper format for a submission, and vowed I ought to be giving

[13] a description of the research methodology and analysis of results, and concluding with specific recommendations of steps in applying the author's findings to instructional delivery.

Dim views were expressed on whether my ‘references were appropriate and extensive’ (journal sheet):

[14] the manuscript does not provide the more familiar — and to me, necessary — presentation of a theoretical framework with references to the specific linguistic, sociolinguistic and discourse analysis literature

[15] there are almost no references to literature in linguistics or in business English. I’d have hoped to see references to say, Halliday or Eggins on systemics and/or functional linguistics, Swales on genre, perhaps, and Ellis and Johnson and/or Hutchinson and Waters on teaching business English.

The requirement that a report on an pilot project in practical teaching be obligated to review the ‘literature’ and quote extensive ‘references ’is not intuitively obvious, to put it mildly. Perhaps you must constantly prove yet again that you know who wrote what. I wonder what these reviewers would make of my 1997 volume with 1618 references but none for Eggins or for Hutchinson and Waters.

Quite predictably, the negative reviewers faulted my ‘writing style’ [16], while positive ones praised it [17] (compare also samples [1-2]):

[16] the writing style is too wordy, and often pompous.

[17] it is written in a very interesting and user friendly style that is likely to motivate most readers to read on.

In the end, the most negative review, despite being four times as long as the rest in its zeal to nail every fault, failed to carry the day. Other reviewers approached my paper with substantially more openness. They recognised my knowledge of the field [18-19] despite the absence of disquisitions on research methodology or recitations of ‘research literature’; and they found the methods and language materials helpful [20-23].

[18] The author demonstrates a high level of familiarity with the subject matter of the field.

[19] The paper is extremely sophisticated and demonstrates a considerable awareness of critical discourse analysis and a sensitivity to intertextuality.

[20] This is an important article which explores how the author found ways to tie building English language skills with learning business terminology, pointing out the inadequacies and imprecision of business textbook authors at the same time.

[21] The article represents an original thinking brought to the traditional problems of finding interesting materials for business English students, and deciphering imprecise and/or poorly written textbook English.

[22] Especially strong are the rewrites of unclear textbook excerpts.

[23] Great demonstration of the imprecision of expression by some textbook authors!

Happy end? Hardly. These data again forced me to acknowledge how far peer reviews can differ for the same submission. Yet I think the game does seem to be changing now. The focus of dispute is no longer such academic concerns as the ‘explanatory power’ of ‘context-free sentence grammars’, but the social and economic situation which forms the context of academic and professional discourse at large. I can appreciate why some academics would prefer to exclude that situation from the scope of teaching and research, and to believe that there is ‘no recession’, that legal language is ‘intelligible and succinct’, and that ‘English departments and business faculties’ are doing a wonderful job. But my critical analysis of actual discourse data has led me to different conclusions, which I had to report. If reviewers disqualify them as ‘under-supported personal interpretations’ [3], then they should suggest alternative ‘interpretations’ of the same data. And if they misprize my ‘political position’, they should expound their own instead of pretending that academic discourse is free of politics.

5. On the threshold of cyberspace

Though my data samples have been quite small, I believe larger ones would raise similar uncertainties about whether the process of peer review can function as is officially assumed. Several of its key notions seem at best unproven and at worst disproven, notably that the merit of a submission is determined by inherent qualities which are open to objective assessment by distinguished academics. These notions reflect a ‘pre-scientific’ model of writing and reading, and also  a benignly reassuring view of ‘normal science’ in the well-known Kuhnian sense. Serious breakdowns can be predicted for times of the Kuhnian ‘scientific revolution’ wherever editorial boards are staffed with ‘normal scientists’.

At present, however, we are in the throes of a revolution seemingly unanticipated in the philosophy of science, Kuhnian or otherwise. We are embarked on a profound reassessment of the nature and responsibilities of public or professional discourse in a ‘post-modern society’, even if academic training and research have in some ways not even securely registered the ‘modern society’. Instead of merely disputing among alternative ‘theories’ or ‘paradigms’, we are interrogating the procedures and implications of ‘doing science’, ‘building theories’, and so on, in a wide scope.

For the field of linguistics, the prospect that peer review could remain unaffected seems implausible, due especially to three emergent forces. One force emerges from ‘critical linguistics’ or, under the more current name, ‘critical discourse analysis’. This force is steadily impinging upon cherished notion of an ‘academic world’ placed above and beyond the social and political issues of the day, as reflected in some reviewers’ comments cited in section 4. The long lethargy of non-social, non-political linguistics in the airless abstractions of ‘syntax’ and ‘semantics’ is about to be roused, and peer reviewers will be hard pressed to hold back the tides.

A second force emerges from corpus linguistics, which is breaking down old prejudices against real data for being mere ‘parole’ or ‘performance’ we can safely disregard. Corpus data are leading us toward quite divergent views of ‘language’, ‘grammar’, ‘lexicon’, and so on from those entrenched in theoretical linguistics at least since the 1950s. In addition, we are realizing that applied linguistics needs to be fundamentally redesigned to integrate previously unavailable information about authentic usage. Here too, peer review will face unprecedented challenges.

These two forces are the most likely ones I see in linguistics for drawing peer review out of its protected corner. So far, we have seen mainly tortuous and defensive arguments from establishment linguists insisting that both methods are illegitimate and must not be allowed to affect business as usual. In parallel, peer reviewing is currently much occupied in keeping them out of print by abruptly raising or narrowing the ‘standards’ or by giving arguments are flimsy as pointing out typos, as I can testify for some of my own submissions.

In the past, significant progress in linguistics and language study has been strongly promoted by establishment of ‘revolutionary’ journals, whose boards are not committing to counter-revolutionary inertia. From my personal experience, I would cite among others Poetics, Text, Discourse Processes, Cognitive Science, Language and Society, and Discourse Studies. But the costs of new journals has grown prohibitive, as have the subscription prices for academic institutions and libraries.

So we come to the third and strongest force of all: the emergence of electronic media on the Internet that totally bypass the heavy labours and expenses of seeing ‘manuscripts’ through into ‘hard copy’. Of course, these need not be revolutionary journals; in the long run we can expect to see some of the prime organs of normal science moving into cyberspace. But new revolutionary journals are most likely to be electronic, such as Hypermedia. Nor again need electronic media dispense with peer review; some of them have it now. Yet the opportunity for reviewers to definitively kill submissions they dislike will be much less secure.

At all events, the drastic limitations on journal space which probably motivated many rejections are no longer vital. The selective functions may be gradually transferred from editorial boards over to the readers themselves who can ‘surf’ these journals on their own initiative. They may prove more able to the task than would be predicted after the long-standing paternalism of editorial boards, with their implied mistrust of readers to choose for themselves.

Then too, we may not see journals at all, but interactive websites. These are unlikely to operate through peer review and are thus most congenial for ‘revolutionary’ topics, such as van Dijk’s CRITICS website for ‘critical discourse analysis’, and Fairclough’s website for ‘language in the new capitalism’. Van Dijk is now organising a vast electronic ‘Encyclopaedia’ for discourse studies, which may become a major barometer of our revolutions.

Interactive websites can also support journals on topics of special interest to modest readerships, such as practicing language teachers in a range of specific environments. In particular, websites can take into account the local cultures and languages where a so-called ‘international language’ like English is being taught. We would finally be freed of the diet of culturally bland — and often culturally inappropriate — teaching materials authored in the UK or the US in hopes of being used ‘all over the world’.

The academic writer in turn has the option of creating a personal website and bringing works directly to the public without the intercession of either editors or peer reviewers. The greatest change is assuredly the resounding renunciation of the usual anonymity and blindness, which, I have suggested, need not bring the benefits claimed for them. Now, the identity of the author will not determine not just which research gets read but also which research gets made accessible. Here too, the implied mistrust, this time of authors, may prove unfounded. An author who posts substandard work on a website risks being cut off from the discourse community, whereas one who gets a substandard work into a journal may still command a readership elsewhere. So the sense of responsibility to do your best work should in fact be intensified, the more so when we cannot count on editorial feedback to point out our own lapses.

A key question for the future would be whether and how long research published in hard copy can maintain an aura of greater authority and prestige than research on the Internet. The answer will depend on how strongly the academic establishment insists on hard-copy as the measure of ‘achievement’ and ‘distinction’. Predictions are difficult to venture just now, when universities and their staffs are being constantly lashed by bureaucracies to ‘demonstrate excellence’ in the intensifying competition for funds and students. In current practice, you are ordered to submit hard copies of published research or at least exhaustive lists which spell out how many pages of hard copy each publication occupies; and some universities demand to know the number of citations your work has received, which is hardly feasible to calculate except for mainline journals on large indexes such as Sociological Abstracts. In the future, we may count citations on the Internet much more easily, as well as the visitations of an author’s websites. Besides, we may hope that more relevant measures of ‘excellence’ will win out, such as the success of a university or programme in placing its graduates in non-academic jobs beyond the realms of peer review and publication.

I would point out in closing that the present discussion could probably not have been published through the mechanics of peer review, not even in a revolutionary journal whose editorial board includes me; I might put the latter in a quandary merely by submitting it. I would make a similar point for a number of others discussions on my website, especially those in the ‘Frank Talk’ section on ‘external examiners’, ‘staff assessment’, and so on. There are still sacred cows in the academic world, nurtured by institutional inertia and conventional decorum rather than by rational argumentation and validation. Many of my colleagues may regard them as necessarily evils; but if they are evils indeed, they are not necessary, least of all in an intellectual ambience that flourishes by rational discourse.