Courtesy of http://xkcd.com/385/
CC-BY-NC 2.5
If you are a fan of academic banter and hang out on Twitter, there are several irreverent Twitter handles you can follow to get a glimpse into the world of academics. A few of these center around peer review. Handles such as @yourpapersucks or @academicssay deliver fictional and sometimes real and frustrating examples of less than helpful feedback given to authors. Many of these are comedy gold.
Recently, Twitter brought to light the opposite end of the spectrum when two female authors complained about receiving the review comments that follow:
“It would probably also be beneficial to find one or two male biologists to work with (or at least obtain internal peer review from, but better yet as active co-authors), in order to serve as a possible check against interpretations that may sometimes be drifting too far away from empirical evidence into ideologically biased assumptions.”
The authors did not name the journal but with help from the internet sleuths of the world, we learned that the paper was reviewed for PLOS ONE.
As someone who is responsible for overseeing peer review, two immediate thoughts popped into my head: “How could the journal let this happen?” And, “Thank god it wasn’t one of my journals.”
This story hasn’t exactly gone unnoticed and much has been written about it but I think it’s worth unpacking a bit here in the Kitchen.
Added to the list of 82 things publishers of journals do, one responsibility should be to foster a constructive forum for authors to present their work. There is no shortage of blog posts and editorials about why publishers are superfluous to the academic publishing process. Some argue that review by peers can be done outside of any formal system and in fact, there are several new ventures that are trying this out.
There is a problem, however, when “editors” are removed from the process. Editors have a responsibility to ensure that reviewer comments are constructive and free from personal attacks. Suggesting that two female authors add a male author in order to improve the paper is a personal attack.
It really makes no difference whether the paper was good or significantly flawed. The reviewer may have been completely correct in stating that the paper lacked empirical evidence. Why on earth the reviewer thought it was appropriate to suggest that the way to solve this would be to add a male co-author is where the train goes off the track.
This is obviously not the first time sexist comments have been made by reviewers. More often than not, the biases appear to be more subtle. Many studies have been done showing the disproportionate number of papers accepted with male authors as opposed to female authors. Further, editorial boards are often lopsided. Conferences are male dominated. Women have been advised to only include first initials so that their gender is not obvious to reviewers. Gender bias is one of many arguments for double-blind peer review and for open peer review.
So back to my two initial reactions, the “how could the journal let this happen” question takes on a new flavor when you take into account that the journal was PLOS ONE. PLOS ONE employs a process whereby papers are reviewed only for technical merit. Additionally, the editorial hierarchy lacks the same kind of oversight seen at many journals. After a paper passes the initial sanity test, an Academic Editor is assigned. This person has sole oversight of the paper. The academic editor (of which there are over 6,000) decides whether to accept or reject the paper and decides whether to get an additional review. In this case, the academic editor solicited a review, got it, and either agreed with it or didn’t read it and rejected the paper. There is no editor-in-chief nor is there any indication that PLOS ONE staff review the decisions and author letters.
Authors who feel as though a rejected paper was not given a fair review can appeal the decision. There is very little information about what this process entails on the PLOS ONE site.
This leads me to my second reaction, “thank god it wasn’t us.” The truth is, this could have (and probably has) happened to a lot of good journals. Everyone is under pressure to speed things up. Big increases in submissions are putting pressure on volunteer editors. Anyone who has ever managed a journal knows that some handling editors are awesome and some are, well, not. It’s that second group that makes us nervous.
In a more traditional journal review process, there is an editor-in-chief and maybe subeditors (associate editors or section editors) who are tasked with oversight of peer review. The levels present on an editorial board vary depending on the size of the journal and the discipline. In many cases that I am aware of, journals staff also review every decision letter before it goes out to the authors.
I have always expected that the editorial coordinators on my team will take a close look at the reviewer comments before sending them to the author. The coordinators should look for whether the reviewer has accidentally identified themselves (which may or may not be a problem depending on your peer review policy), that the review appears to be for the correct paper (mistakes like this happen all too often), that the reviewer isn’t requesting that the author cite the entire publishing history of the reviewer, and that inflammatory language is never included. There are a dozen reasons that a coordinator could miss comments such as were made in the PLOS ONE review– being interrupted by an author calling for an update on her paper, trying to catch up on decisions that stacked up during a vacation day, not being on the top of one’s game when skimming through dozens of reviews that are technical in nature, etc.
This recent incident at PLOS ONE has afforded us a teaching moment. The scholarly publication process is a human process. People make mistakes, people say dumb things and some people are just jerks. As journal publishers and journal editors, we have a responsibility to protect our authors from the jerks. To be clear, it is okay to tell an author that the paper has serious problems. It is not okay to tell the authors that their fatal flaw is their gender.
As publishers, we also have a responsibility to correct these mistakes, as I believe PLOS ONE has done, albeit a bit slower than the authors may have liked. The authors took their complaints to Twitter after having not gotten a satisfactory response to their appeal request within three weeks. I can’t imagine what took so long. The social media storm that ensued moved things along and PLOS then dismissed the reviewer and removed the review from the system. The “academic editor” who handled the paper has been asked to step down and the paper is being given a new review. All of these individuals remain unidentified.
Issues facing women in academia and scholarly publishing are sadly a reality. Much of the gender bias in the publishing side could be eliminated by diversifying editorial boards, diversifying reviewer pools, zero-tolerance policies against gender bias in review, and review processes that protect authors from unfair treatment (going totally dark with double-blind review or shedding light on all participants with open review).
Journals have witnessed a series of peer review crimes lately—faked reviews, made up papers, papers authored by Maggie Simpson. While these incidents are serious and a black mark on the journal peer review process, they represent clear cut, black and white issues. Gender discrimination in academia and society at large is far more insidious and harder to root out. Hopefully some good will come out of this incident and the resulting raised awareness for editors and reviewers will lead to greater care and scrutiny in the future.