What’s Online Peer Review For? Guest Post by Stacy Lavin

It’s Peer Review Week, an annual event that brings together individuals, institutions, and organizations committed to sharing the central message that good peer review, whatever shape or form it might take, is critical to scholarly communications. We are pleased to share a guest post by Senior Managing Editor for Journals, Stacy Lavin.

You might look at the title to this post and think: Duh, it’s for peer review. Right? I mean, what is an online peer-review system but a digital version of the analog process that editorial offices have been following for centuries? All anachronisms (and attempts at catchy openers) aside, that’s exactly what I used to think. That is, it’s what I thought online peer review was for before I spent about four years acting as an intermediary between our journals’ editorial offices that use online peer review and the vendor for the system we use (Aries/Editorial Manager). In those four years, I’ve discovered that online peer review has the potential to do much more than streamline the work editorial offices do to vet and select content for publication. It has the potential not only to address the pain points of academic journal editors and their staff but also to serve the broader strategic interests of editorial offices, scholars, and publishers in less obvious ways.

As we’ve been reminded during Peer Review Week 2018, “good peer review, whatever shape or form it might take, is critical to scholarly communications.” This year’s Peer Review Week theme of diversity in peer review has, moreover, prompted us to consider the invisible and indirect barriers to “good peer review,” which resonates with the growing awareness of the importance of actively and systematically developing a culture of inclusion within the community of university presses. As Sandra Korn and Alejandra Mejía astutely observe, writing of their work to coordinate the student intern program in the Books Acquisitions department of Duke University Press, it “is vital to the intellectual work of publishing to have queer students, students of color, students from low socioeconomic backgrounds, and student activists engaging with the literature that is oftentimes theorizing the experiences of their communities.” Alice Meadows, writing just this morning, reminds us that while we’ve known for a while that scholarly publishing overall is an “overwhelmingly white and cis-female industry, with a leadership that is disproportionately white male dominated,” it is becoming clear that peer review is likewise “less diverse and more biased than it could or should be, which is hindering our efforts to ensure an inclusive, ethical, trustworthy scholarly communications ecosystem.”

Getting back to my opening thoughts about online peer review, I would add that cultivating an inclusive culture in all of the spheres of our work as a scholarly publishing community is one of the broader strategic interests that online peer-review systems have the potential to further. For instance, among the eight tactics Meadows recommends in her post as ways to tackle diversity and inclusion in peer review, two of them involve pillars of online peer-review systems: data and software. Tactic number six is “Collect the data.” While there has been limited progress in creating gender equity in referee populations, Meadows suggests that at the very least collecting the data is “a critical first step toward being able to understand, and ultimately resolve, the issue.” Tactic number seven is “Make use of available tools” like editorial management software that can “spot and reduce the risk of bias in the selection of reviewers.”

Now, you might be thinking peer-review systems are hardly built to address diversity and they’re often seen as pretty onerous to manage by users as it is. Good point. Let’s step back for a moment and consider that the main challenges that online peer-review systems originally aimed to address were ironically the high labor requirements of managing the processing and vetting of submissions—inviting reviewers, reminding reviewers, making decisions, sending decision letters, asking for revisions, reminding authors to send in revisions—not to mention increasing concern over reviewer availability and fatigue. Online peer-review systems stepped in to help solve some of these problems (automating reminders, etc.), while more avant-garde services have sought to streamline the work of reviewers themselves (Rubriq, etc.). While the automation and data tracking features of peer-review systems have been useful, reviewer fatigue has been a more elusive and possibly more critical challenge. For instance, there are journals who claim to practice blind review but then—passing the burden of finding a reviewer onto the contributor—require contributors not only to nominate a reviewer for their submission but also obtain the permission of that reviewer to nominate them (which I guess makes it not-so-blind review).

With online peer review, as is often the case with technology, solving some frustrations gave birth to new frustrations. The typical grievances are aversions to non-intuitive interfaces—the tinkering, system-admin-oriented environments where coders and likeminded folks feel right at home but others…don’t (I’ll leave it to Kent Anderson to articulate the way many users feel about manuscript tracking and submission systems). But that’s just one (solvable) problem representing only one facet of the massive capabilities these systems have the potential to materialize to help the scholarly publishing community facilitate a more inclusive peer review culture (not to mention more efficient editorial and production processes). In fact, I am convinced that the biggest opportunity of the current state of online peer-review is also a source of the biggest concerns users have: it can do so much more than they need. They’d prefer not having so many functions, options, buttons, or configurations. They don’t want to have to adapt their workflows to the system. The beauty of it is that they don’t have to. As Meadows notes, we can use existing features of these robust systems to pursue the multifaceted interests of the scholarly communications community. As well-funded companies recognize the value of and acquire online peer-review systems, those systems might become more flexible and capable of developing the tools we need to solve the more elusive problems of peer review like reviewer fatigue and making peer review more inclusive—in fact, solving the latter is key to solving the former. Likewise, established publishing vendors are adding peer review to their suite of services, seeking input from the academic publishing community on how to best meet their needs. Our voices as publishers and potential consumers of their expanded services will carry a lot of weight in those conversations, I would think. As I see it, our job in this climate of consolidation/expansion with respect to online peer-review software is to focus on what these systems can do to help us, not just our workflows but also our broader strategic interests, such as cultivating inclusion in all aspects of what we do. It’s not our job to adapt to these systems (they’re almost infinitely configurable) but theirs to adapt to us in our efforts to actively and systematically foster diversity in scholarly publishing.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s