Publishers trial paying peer reviewers — what did they find?

Two journals embarked on efforts to compensate reviewers, with different results

Trials suggest that offering payment can increase the chance of a researcher agreeing to review, and in some cases speed up the process. Credit: Catherine Falls Commercial/Getty

A spate of research findings offer fresh evidence to the debate about whether peer reviewers should be paid for their time and expertise — a fraught topic that has provoked discussion among researchers.

This month, two journals released data from their own experiments that suggest that offering payments of around USD$250 to researchers who review manuscripts speeds up the process, without affecting the quality of reviews. But some specialists warn that the practice could have unintended consequences for science and publishing.

Although both trials are small, they are a good start at gathering data on paid peer review, says Balazs Aczel, a psychologist at Eötvös Loránd University in Budapest. But he adds that whether to pay peer reviewers remains “a very complicated question”.

Rewarding reviewers

The peer-review system has come under pressure in recent years as more science is published and scientists face more demands on their time. Journal editors now find it harder to secure reviews and some scientists have questioned the fairness of their voluntary labour being relied upon by some highly profitable publishing companies.

The idea of paying peer reviewers has long been discussed, but few publishers have chosen to go down this route so far. Economics journals have experimented with the idea in the past, and some medical journals pay certain reviewers. Others have adopted less-conventional compensation systems: open-access mega journal PeerJ uses a token system that gives reviewers a discount on publishing fees, whereas another title pays its reviewers in a specially developed cryptocurrency.

Some researchers fear that offering reviewers cash incentives could lessen the quality of reviews or change the landscape of research in other, as-yet unknown ways. But until now, there has been a lack of hard evidence about the potential benefits and drawbacks.

Intrigued about the effect of paying peer reviewers, editors at the journal Critical Care Medicine launched a six-month experiment led by David Maslove, a clinical scientist at Queen’s University in Kingston, Canada. Starting in September 2023, the journal asked 715 researchers to review papers. It offered roughly half of them a US$250 incentive.

The results, published in the journal earlier this month¹, found that paying for reviews moderately improved both the number of accepted invitations and the speed at which reviews were carried out. Some 53% of researchers accepted the invitation to review when offered payment, compared with 48% of those who received a standard, non-paid offer. On average, paid reviews came in one day earlier than unpaid ones. Journal editors assessed reviews from paid and unpaid reviewers and found no difference in quality.

Maslove says that the small size of the effect suggests that money has a limited effect on motivating peer reviewers to change their behaviour. “There could be these other values that peer reviewers have, whether its a sense of responsibility or loyalty or owing to society.”

Speed advantage

A separate experiment at the journal Biology Open, found a larger effect, albeit with fewer reviewers.

For six months starting in July 2024, editors covering two of the journal’s ten subject areas treated reviewers as paid contractors under two systems. Reviewers were either offered a £600 (US$776) retainer to review up to three papers per quarter, or were paid £220 per review. Under this scheme, editors would send freelance reviewers an invitation to review, which they had to accept or decline within one business day. Once accepted, the reviewer had four days to submit their peer-review report. A total of 20 manuscripts were reviewed in this way.