Peer review by mail?

Submitting a paper to a journal is slow.

First you submit. Then you wait. Then submit again because you didn’t follow some copy-editing rule ¹. Then you wait. Then an editor finds the manuscript (if you’re lucky). Then you wait. Then reviewers are found. Then you wait (a lot). Then you receive reviews, work super hard to rebut, and resubmit. Then you wait. Wait, wait, wait ².

Here’s a crazy idea that won’t eliminate all those delays, but could remove some of the bottlenecks.

Authors and reviewers should discuss the paper directly

Whoa now, hold up! “How can they possibly do that? What about blind review? Double-blind?"

Of course. But that’s easy to fix: anonymous, disposable email addresses:

FROM: reviewer-085a6938-1d30-486d-aa34-4ca30ce59bb7
TO:   author-2968511b-a5f6-4614-8881-fde2252b0219
CC:   editor-a83b7c96-b5d7-4530-a3d9-bf46ef516a6e

I don't understand Eq 3 in your paper. Why is sigma always positive?

FROM: author-2968511b-a5f6-4614-8881-fde2252b0219
TO:   reviewer-085a6938-1d30-486d-aa34-4ca30ce59bb7
CC:   editor-a83b7c96-b5d7-4530-a3d9-bf46ef516a6e

Sigma can be zero when measurements are taken using a fluxometer. But we only
ever use a flubometer.

FROM: reviewer-085a6938-1d30-486d-aa34-4ca30ce59bb7
TO:   author-2968511b-a5f6-4614-8881-fde2252b0219
CC:   editor-a83b7c96-b5d7-4530-a3d9-bf46ef516a6e

Oh, of course. You are very smart and your paper is very good. Accept!

(Ok, that last part may be a little bit wish-fulfillment…)

(Also, I’m not discussing the technology much here, the implementation details. Probably the anonymous emails need to be implemented in a webmail system. Not a big deal ³.)

I see more advantages than disadvantages using a rapid-fire, back-and-forth review exchange compared to the traditional review-rebut-review-rebut model.

Pros

• Authors can clarify minor details for a reviewer, saving time for the reviewer, the authors (especially if the paper is rejected just due to a misunderstanding), and even the journal.

• Reviewers may spend less time overall in an email exchange than in preparing a full report. Authors may spend less time emailing than in developing a detailed rebuttal.

• Authors don’t have to wait for the journal to forward the reviewer comments. If reviewers write back at different times, the authors don’t need to respond to everything at once.

• Reviewers can talk with each other as well.

Cons

• This could make room for abuse. A reviewer could make unacceptable comments directly to the author. In the current system, the journal and editor have a chance to step in and prevent that. However, how often does that happen? I’ve gotten some pretty nasty comments over the years. Further, all the throwaway email correspondence should be CCed to the journal anyway, so it can still be monitored.

• Reviewers can talk with each other as well. (It’s common in CS conferences for reviewers to discuss among themselves. But isn’t that a bad idea? Don’t you want diversity in your ensemble?)

Now, this is a crazy idea but I also don’t to be too cavalier.

Peer review is fragile, and there are risks to radical changes. If poorly handled, people could game the system, lousy science could be published, and disadvantaged groups could be further burdened.

Has this been tried before? What am I missing that undermines this idea?