Last week I wrote about a set of AGU EOS articles from 2003 that focus on anonymity in peer review. A quote from one of the articles really stuck with me regarding the personal decision to sign reviews:
Okal (2003) states that, as an editor of GRL, ~40% of the reviews he sees are signed. As a reviewer, he signs 2/3 of his reviews. And as an author, 1/2 the reviews he receives are signed. His experience suggest that:
“The above numbers — 40%;two-thirds; one- half — suggest that the community is divided, with no overwhelming majority in its attitude toward anonymous versus signed reviews. This diversity may indeed be precious and should be respected. Why not keep the system as it is now, leaving it to the individual reviewer to exercise a free decision regarding waiving anonymity?”
Over the course of the next few weeks I hope to build a fun little toy model of ‘peer reviewing’ agents to see if I can tease out something — is diversity in peer review behavior (re: signed vs blind) in some way ‘precious’?
the rules of the model are:
Each agent (scientist) is set to either sign or blind their reviews.
For each time step:
- Randomly pick the number of scientists (‘P’) out of ‘N’ total scientists who will publish a single paper
- Randomly assign ‘R’ reviewers for each paper
- Nobody can review their own paper
- Writing Sceintists can review
- Scientist can do multiple reviews
- Each reviewer gives a random review score (good or bad)
- Reviews are returned to each writer and writers ‘mood’ changes
- signed + reviews result in + feelings toward the reviewer
- signed – reviews result in – feelings toward the reviewer
- unsigned + reviews result in + feelings toward a random scientist
- unsigned – reviews result in – feelings toward a random scientist
And we see how the feelings of the community (toward one another) develop through time.
The beginning of the code is already up on Github. Feel free to contribute or give an opinion.