The comment section of a peer reviewed journal is set out as a place for arguments or rebuttal of published ideas. Of the geomorphology journals I tend to focus on, only Earth Surface Dynamic (ESurf) and the Journal of Geophysical Research – Earth Surface (JGR-ES) offer a comment and reply section (someone please correct me if I am wrong here, but I couldn’t find anything on the websites of Geomorphology or Earth Surface Processes and Landforms on ‘Comments’ as a manuscript type).
Here I want to focus on the 13 year record of JGR-ES. (ESurf is in another game because the journal is young, and open peer review actually allows for a mechanism of public comment and reply during the review process). For JGR-ES, 11 comment/reply pairs have been published. (Side note — who knows how many were submitted and rejected by Eds…. and the acceptance rate of comments is another question).
From this artisan, hand crafted small batch data come these stats:
In this cohort —
- Manuscripts have a mean of 22.7 citations, with a range of 3-86. (Median citations are 15, shown above in red)
- Comments have a mean of 2.4 citations, with a range of 0-6.
- Replies have a mean of 0.6 citations, with a range of 0-3.
- Comments have 18% of the citations compared to the original article (ranging from 0% to 50%).
These statistics seem meaningless, but the qualitative point is that comments are cited far less than the original manuscript. And replies are cited even less than the comments.
In one sense, this is not surprising — previous work by Banobi et al (2011) remarks that comments are not often linked to the actual paper or embedded within the manuscript pdf (even within a single journal). Readers might not actually know there is a comment.
Just to give JGR-ES some credit here, I did notice that some (but not all) JGR-ES comment-reply pairs appear in the ‘Related Content’ tab shown at the bottom of the JGR-ES web page. But this requires researchers to scroll to the bottom of the page for JGR-ES’ own suggestions for ‘Related Content’.