If you read the article, they are suggesting a different approach to peer review, not doing away with it. They want to find ways to build in incentives for reviewers to make it worth their while to review rather than allowing it to continue as something that scientists do out of a sense of obligation.
They have an interesting approach but I think it doesn’t go far enough.
Couldn’t you have researchers who specialize in finding “bugs” in published papers (yes, researchers already do this to each other), like we have QA testers or bounties for finding exploits? Is this too aggressive an approach for science? Should work for hard sciences, though.
This is a great idea I think, but part of the problem with science versus programming is that they’re just very different social environments, so the expectations, norms, and demands on each are very different.
Dependent down a little bit more, most research is done by people with phds or other advanced degrees (or pursuing them) in an academic job, and one of the conditions of attaining or maintaining that job is publishing. And these are the same people doing the peer reviews.
I think what this creates, even aside from the overwhelming volume and complexity of work, is a certain amount of grace amongst academics. That is, I think a fair number of peer reviewers are not only failing to rigorously grapple with the material that they review, but because of the small social mileux and shared incentives, they are incentivized to not be very rigorous in many cases.
Not saying peer review is without value, but how harshly would you want to challenge or critique the work of someone whom you may work alongside or under in the future?
i heard about a woman a while back that did exactly that: she read papers across disciplines and found doctored results etc… she’d found something like 10 papers that had fabricated data
I had a thought and didn’t make it clear, added the last sentence that I’m referring to the new
system allowing for anonymous reviews. That combined with publish before review is making a new system catered to malicious business interests. Tobacco companies would just love this system.
Now the idea of making the whole process more visible to a wider audience? Yeah, that could be a benefit.
If you read the article, they are suggesting a different approach to peer review, not doing away with it. They want to find ways to build in incentives for reviewers to make it worth their while to review rather than allowing it to continue as something that scientists do out of a sense of obligation.
They have an interesting approach but I think it doesn’t go far enough.
Couldn’t you have researchers who specialize in finding “bugs” in published papers (yes, researchers already do this to each other), like we have QA testers or bounties for finding exploits? Is this too aggressive an approach for science? Should work for hard sciences, though.
This is a great idea I think, but part of the problem with science versus programming is that they’re just very different social environments, so the expectations, norms, and demands on each are very different.
Dependent down a little bit more, most research is done by people with phds or other advanced degrees (or pursuing them) in an academic job, and one of the conditions of attaining or maintaining that job is publishing. And these are the same people doing the peer reviews.
I think what this creates, even aside from the overwhelming volume and complexity of work, is a certain amount of grace amongst academics. That is, I think a fair number of peer reviewers are not only failing to rigorously grapple with the material that they review, but because of the small social mileux and shared incentives, they are incentivized to not be very rigorous in many cases.
Not saying peer review is without value, but how harshly would you want to challenge or critique the work of someone whom you may work alongside or under in the future?
i heard about a woman a while back that did exactly that: she read papers across disciplines and found doctored results etc… she’d found something like 10 papers that had fabricated data
I had a thought and didn’t make it clear, added the last sentence that I’m referring to the new system allowing for anonymous reviews. That combined with publish before review is making a new system catered to malicious business interests. Tobacco companies would just love this system.
Now the idea of making the whole process more visible to a wider audience? Yeah, that could be a benefit.