[Peter Frost]

What exactly is our policy on anonymity? I understand the circumstances that lead some reviewers to request anonymity, but someone somewhere should know the real identity of each reviewer, i.e., his/her real name, address, highest academic degree and area of study, etc.

Is this the case?

[Emil]

Authors can be anonymous. Reviewers must keep a steady account, but it can be anonymous. For instance, Dalliard, Chuck, Meng Hu use pseudonyms and are not known to me personally.

[Peter Frost]

Emil,

Is that a sound policy? When I review manuscripts for other journals, my identity is anonymous to the author (unless I choose to reveal it), but the editor knows who I am.

Again, I understand why some reviewers choose to be anonymous, but such anonymity shouldn't be systematic. The editor should know the identity of each reviewer and his/her academic credentials.

[Emil]

It is hard to say. With anonymous reviewing, reviewers are not held responsible except by the editor. If their reviews are public and in their name, this may cause them to write better reviews. Or maybe not according to this study.

There is some data that indicate that lack of anonymization of authors leads to biased reviewing, in that reviewers favor papers sent in from top US universities. I can't find the study right now, but it was from a natural experiment with a medical journal that changed practice at some point and they compare data before and after. Edit: Study here.

There is a review of research into reviewing bias here, generally critical of the evidence offered so far.

In practice, until we get some better software for handling the reviewing process, reviewing will probably have to continue like it does now. This software will have to be written by me or Bo, so it can take a while to get done. We do not get paid for working on this project.

There is no special role for the editor of OpenPsych journals. Reviewers for each journal form a democratic body. They can choose to elect a new reviewer, or fire a current one. I do not want to police reviewers.

Initially, I (and Piffer) had to select some persons to invite to become a reviewer, but this isn't supposed to be how it is going to be in the future.