Scholarly and Peer-Reviewed Journals

Overview

Scholarly or academic journals are periodicals that serve as forums for the distribution and discussion of research in a particular academic discipline. The publication of an article in a scholarly journal is the basic form of scholarly publication, and because the lag in journal publication is much shorter than in book publication, the research in scholarly journals generally represents the most current state of the field. The sheer number of scholarly journals is such that even experts cannot be expected to keep current with everything published in their field, though since the middle of the twentieth century it has been increasingly the case that scholars and academics specialize in narrow subfields within their discipline, where it is more feasible to keep abreast of developments and discourse related to that narrower area of concern. Even so, this volume of scholarly matter has a number of implications.

Scholarly journals are generally not profit-making ventures, nor do they pay scholars for their work. The journals are usually sponsored by universities and professional organizations, with subscriptions sold to other university libraries covering the costs of printing and distribution. Such “institutional subscriptions,” as opposed to personal subscriptions, are typically prohibitively expensive for individuals, though journals published by professional associations for their members tend to be more affordable. Online access to hundreds of journals is provided by services like EBSCOhost. Research is successfully attracted even without pay because of the “publish or perish” dilemma faced by university professors: a successful academic career requires the regular publishing of new work relevant to the individual’s field, which is generally a prerequisite for tenure and impacts salary and other considerations. Researchers publishing their work in journals, then, receive a tangible benefit from that publication, just not one provided by the journal itself. (In some cases the author may even be asked to pay a publication fee, but this is not a widespread practice among reputable journals and is akin to vanity publishing.)

There are significant problems with the publish or perish paradigm. Many allege that it puts a higher priority on publication than on teaching ability, especially in technical and scientific fields where publishing professors are necessary to attract research grant funds to the institution, whereas teaching ability has less of a direct financial impact. The need to publish diverts attention away from teaching and student interactions, as well as contributing to a glut of research and providing motivation for transgressions of research ethics. The paradigm also exacerbates the effects of institutionalized biases; women both publish less frequently than men and are cited less frequently than men, an effect that trickles down to make them less valued professors. Studies have found that women are cited less frequently even when they are published in journals with higher impact factors (IFs) than the men to which they are being compared.

Impact factor is a measure of a journal’s impact on the discipline it covers, by considering the yearly average number of citations referring to work published in the journal. Some agencies also report a five-year impact factor. IFs are useful only when comparing two journals within the same field, and it is important to remember that they measure not the quality of the research published but, in a sense, the popularity of the research, which is not an exact proxy for quality. There is considerable dissent over the fact that IFs are calculated on the journal level rather than the author level, as well as the practice of considering only citations to recently published articles, which is not necessarily an adequate reflection of the real and lasting impact of work on a discipline.

Unlike the magazines they otherwise resemble, scholarly journals are not meant for casual reading, and indeed in many cases it is assumed that few readers will read an issue cover to cover. In addition to research reports and similar articles, such journals usually publish reviews of recent books in the field, and perhaps an overview of research developments or follow-ups on previously published work. Primarily they serve to announce and share research with the greater scholarly community. In scientific research in particular, research articles clearly lay out the procedures that were followed by researchers and the exact results, rather than the summary and conclusions that one would read paraphrased in the mainstream press (or a popular science magazine like Scientific American). This enables other scholars to replicate the research; replicability is a necessary part of the scientific method. When there are discrepancies discovered by replication, this leads the discussion into important areas about what conclusions to draw, what factors may not have been considered, and so on.

Some of the oldest scholarly journals are multidisciplinary. Nature, founded in the United Kingdom in 1869, and Science, founded in the United States in 1880, are preeminent in their fields and are among the oldest scholarly journals; both are multidisciplinary, covering the full range scientific research.

Further Insights

Work that is published in scholarly journals goes through a process called “peer review,” sometimes called “refereeing.” Many of the “junk” journals that have accumulated in order to exploit authors or readers ignore peer review, which is the easiest way to identify them; that said, the peer-review process has a number of problems which also need to be addressed. The basic form of peer review is simple: work submitted for publication in a journal is read over by experts in the applicable field, who decide whether the work is sound and should be published. Often, peer reviewers submit notes for revision rather than simply rejecting a paper outright at first pass.

There are challenges in establishing impartial peer review. Even when papers are anonymized—reviewed without authors’ names appearing—in a sufficiently small field it will be hard to find qualified reviewers who are not familiar with the researchers, and who may even already be aware of the research that has been conducted. As in any field, rivalries exist, as do favorites, and there are always disagreements about certain aspects of the field which contribute to biases. Still, even if the peer-review process is inherently imperfect, it provides a better filter than none at all (or than depending solely on the judgment of an editor, who cannot be fully informed about every subfield).

The peer-review system developed out of processes used by the Royal Societies of Edinburgh and London and other national academies in the eighteenth century, but the modern system, with external reviewers used as a matter of course, did not become the international standard in scientific research until the mid-twentieth century. Medical journals adopted it earlier, while Nature did not formalize the process until 1967. Albert Einstein, in fact, had been critical of the review process when he was subjected to it in the early twentieth century, threatening to take his work to a journal that did not employ it.

The peer-review process follows three standardized steps. The desk evaluation is the editor’s initial evaluation of the manuscript (generally not the editor-in-chief, but a designee filling the role that in publishing is called the slush reader). At many journals, the desk rejection rate is high; if they receive many more submissions than they can publish, it makes sense to winnow the pile of submissions down before going through the lengthy and resource-intensive process of peer review. The second step is the blind review: manuscripts passing desk evaluation are distributed to reviewers or referees selected for their expertise and objectivity. Typically, there are multiple reviewers, and submissions are blind (meaning identifying information about the author has been removed). Usually reviewers are also unaware of who else is reviewing the paper. Different journals have different procedures for what is asked of a reviewer. Usually, they are asked to make a recommendation on some spectrum from acceptance to rejection, including the possibility of recommending a revision prior to a final decision. Reviewers need to express their views of the paper in a way that is explicable to the editor, who has the final say. Step three is the revision stage; nearly every paper, especially at leading journals, undergoes some degree of revision. Increasingly, some journals are publishing the full history of articles online, where readers can look at revision notes, the notes back and forth between author and reviewer, and so on, in an effort to make research more transparent. This is also exceptionally useful to scholars who are studying the workings of the field itself, as a way to examine the process by which research arrives in public view.

Whether the reviewers themselves are anonymous varies by journal. Double-blind reviews, in which the author is anonymous to reviewers and reviewers remain anonymous even after the process, are the standard in social science and humanities journals, having been adopted by sociologists in the 1950s as the peer-review process was first becoming standardized. It is less common in the physical sciences, where frequently the nature of the research itself will heavily imply the identity of the researcher—there are only so many physicists with access to a specific particle accelerator, for instance.

Peer reviewers are not responsible for the decision to publish a paper or not; that rests with the editor-in-chief or an editorial board. Usually, though, reviewers have significant influence on the decision. One purpose of the peer-review process is to make published research useful to non-experts—not just the general public, the private sector, and public agencies that may want to make decisions guided by research results, but scholars who are not qualified experts in a specific subfield. Without peer review, every reader would have to judge the merit of research on their own, usually without an appropriate foundation in the relevant material. When scholars in closely related fields draw on other work to build on or inform their own work, this lack of total competence could lead to repeating erroneous conclusions, therefore doubling the effect of the original error. Peer review is, ideally, a sort of certification that the research is reliable. That does not mean it will not be contradicted by further work in the area; no single study is sufficient to establish the proof of something.

In 2006, Nature began experimenting with a hybrid peer-review process that incorporated the traditional peer review with an open peer-review process that had been put in use by some journals the previous decade. Open peer review is the opposite of anonymous peer review: identities of authors and reviewers are known and published. Nature found that although employing open review did not seem to impact the quality of the review process, it did slow it down, simply because fewer reviewers were willing to work openly, making the recruitment process slower. Another variant peer-review process is the result- or conclusion-blind review, in which reviewers are not told the results or conclusion of the research, which in theory prevents their review from being biased by their expectations about the research outcome.

The peer-review process can be seen as simply the first step in an ongoing review process, part of the “conversation” that constitutes the ongoing research in a given area. After publication, readers and fellow researchers not involved in the peer-review process continue to offer feedback, and some journals allow authors the opportunity to respond. This has become especially common in the internet age, in which articles are often published online first, allowing responses to be published in the print version.

Issues

In the twenty-first century, there has been a push across multiple areas to advocate for open-source or open-access alternatives of things previously kept proprietary: it began with the open-source software movement in the twentieth century and the founding of Creative Commons in 2001, and expanded in other areas, often in response to restrictive copyright laws or high costs attached to the access of information. Scholarly journals are in essence philosophically compatible with open access: their whole purpose is not to promote proprietary content, but to share research results and ideas as the lifeblood of the scholarly community. The open-access research model emphasizes this spirit of sharing while reducing or eliminating access fees. This can be accomplished through an open-access article repository, to which articles are uploaded by the author after peer review, or an open-access journal, which is a journal made freely accessible on its website. Funding for the journal may be provided through fees charged to authors (or the sponsors of research), a sponsoring academic institution or government body, or through traditional subscriptions, in which case there is usually an embargo period before articles become available for free.

In addition to well-established gender bias and the impact of “publish or perish,” there are a number of pitfalls associated with peer-reviewed journals. Thomas Kuhn and others have pointed out the hostility in the sciences to work that contradicts prevailing beliefs. Sociologists have further established that competition for research funds results in work that favors mainstream views over that which challenges them. In addition, journals usually lack any mechanism for dealing with “peer-review failures”—that is, when the peer-review process fails to detect serious errors in work that is consequently recommended for publication. This oversight has been exploited on a number of occasions, such as to publish hoax articles. There is also little ability to watch for plagiarism, self-plagiarism, or faked research results.

Bibliography

American Psychological Association. (2021). Peer review. www.apa.org/pubs/journals/resources/peer-review

Abubakari, A. et al. (2023, July 13). The future of academic publishing. Nature Human Behavior, 7, pp. 1021–1026. doi.org/10.1038/s41562-023-01637-2

Cook, B. G. (2016). Reforms in academic publishing: Should behavioral disorders and special education journals embrace them? Behavioral Disorders, 41(3), 161–172. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=118951099&site=ehost-live

Dali, K., & Jaeger, P. T. (2018). Beyond scholarly publishing: The human dimension of peer review in LIS. Library Quarterly, 88(2), 99–124. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=128435619&site=ehost-live

Hill, T. E., Martelli, P. F., & Kuo, J. H. (2018). A case for revisiting peer review: Implications for professional self-regulation and quality improvement. PLoS ONE, 13(6), 1–20. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=130380607&site=ehost-live

Malcom, D. (2018). It’s time we fix the peer review system. American Journal of Pharmaceutical Education, 82(5), 385–387. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=130616400&site=ehost-live

Podder, V., Price, A., Sivapuram, M. S., Ronghe, A., Katta, S., Gupta, A. K., & Biswas, R. (2018). Collective conversational peer review of journal submission: A tool to integrate medical education and practice. Annals of Neurosciences, 25(2), 112–120. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=130664538&site=ehost-live

Ross-Hellauer, T., Deppe, A., & Schmidt, B. (2017). Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PLoS ONE, 12(12), 1–28. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=126748385&site=ehost-live

Sokal, A. (2017). More fashionable nonsense: Some thoughts on “The Conceptual Penis as a Social Construct” Hoax. Skeptic, 22(3), 54–58. Retrieved September 15, 2018, from EBSCO Online Database Academic Source Ultimate. http://search.ebscohost.com/login.aspx?direct=true&db=asn&AN=128035367&site=ehost-live

Vinyard, M. & Colvin, J. B. (2022). Demystifying Scholarly Metrics: A Practical Guide. Libraries Unlimied.