Peering into peer review: Good quality reviews of research articles require neither writing too much nor taking too long

Abstract

The value of scientific knowledge is highly dependent on the quality of the process used to produce it, namely, the quality of the
peer-review process. This process is a pivotal part of science as it works both to legitimize and improve the work of the scientific
community. In this context, the present study investigated the relationship between review time, length, and feedback quality
of review reports in the peer-review process of research articles. For this purpose, the review time of 313 referee reports from
three Chilean international journals were recorded. Feedback quality was determined estimating the rate of direct requests by
the total number of comments in each report. Number of words was used to describe the average length in the sample. Results
showed that average time and length have little variation across review reports, irrespective of their quality. Low quality reports
tended to take longer to reach the editor, so neither time nor length were related to feedback quality. This suggests that referees
mostly describe, criticize, or praise the content of the article instead of making useful and direct comments to help authors
improve their manuscripts.