1. All data and parameters associated with any open publication should be available to anyone interested in it. The NIH has mandated this for its grant recipients, but sharing data and parameters should also be a required condition for a publication in any journal. Refusing to do so after a paper is published should be considered scientific misconduct.
2. Anyone knowingly publishing a paper that clearly contradicts the work of another group should be required, also as a condition for publication, to discuss the matter with that group well before publication. In the past, this was considered good
scientific etiquette, but gone, apparently, are those days, so a rule is in order.
3. Journal editors should be more aware of referee conflicts of interest. Reviewers should be required to stipulate any conflict of interest in reviewing a paper, even if it’s simply that they don’t like the authors.
4. No journal editor should be allowed to edit a Comment on a paper he allowed to be published. This is an obvious, unacceptable conflict of interest.
5. Comments should not be required to be so short as to prevent them from making sense. I suggest two journal pages, or, better, three. Or how about this radical idea: they should be as long as it takes to make the point.
6. Crazy rules that allow logically offensive situations, like the one that called for rejecting a Comment because the Reply is unpublishable, should be deleted immediately. And Comments and replies need not, and should not, be published together. Indeed, a Comment on a Reply is a good idea, yielding an interesting ongoing dialog that would benefit the community.
7. The reviewers who review a Comment should also review the Reply. They’re the best qualified, as they’re already familiar with the work. This would prevent the insane situation that occurred here, in which the highest quality review of the Comment was simply lost.
8. Reviews should themselves be reviewable. Currently, reviewers can say whatever they like, and there is no check on them. Authors should be allowed to nominate
irresponsible reviewers, such as Reviewer #2 in the above scenario. Confirmed irresponsible reviewers should then be identified and removed from reviewer databases, which would be shared with other journals. Writing an irresponsible review
should be considered a form of scientific misconduct.
9. While removing unethical reviewers would help, improving reviews of ethical ones is also important. Currently there is no compensation of any sort for reviewers and hence no encouragement to do a good job. I believe that reviewers should be paid for their services. People take paid jobs much more seriously than volunteer efforts. Knowing this, social psychologists pay their subjects simply to fill out questionnaires because it yields much higher-quality results. And what could be more important than the accuracy of the archival scientific literature?
10. Require scientific ethics courses in grad school. Problems like those that I encountered are a proverbial ticking time bomb for science. What if those opposed to taking action against global warming were to make the claim that science shouldn’t be believed in this matter because its process is so rife with poor ethics that it can’t be trusted?
Podcast
12 comments:
A related situation occurred with a paper our group put out several years ago. It was essentially a data-mining paper, in which we examined the clinical and biochemical data from large numbers of patients attending a lipid clinic, and looked for correlations.
Three reviews: 1 "paper must be published now", 1 "good with minor revisions", 1 "this is terrible work". We responded to the reviews, tidied up the paper, tightened the language, and waited.
SIX pages of rebuttal came from reviewer 3. All spurious, vague, and wrong. We wrote five pages of detailed "here's why you're wrong" back, and got NINE pages of the same spurious nonsense back.
It took a long time for the journal to step in, which implied to us that the reviewer was someone 'big'.
Wow that guy sounds pretty pissed. Definitely brings home the relevance of respecting your colleague's published work.
Generally I think it's a bad idea and almost never necessary to make a sort of negative argument in publications...ie "This is NOT true" or "This technique is impossible", etc., etc., or making it the point of your paper to contradict existing publications.
First because I think it is beyond the scope of experimental science to disprove concepts. All you can do is show some evidence of a novel observation and put it in the context of what kind of theoretical model you think it supports. Your data might favor one model over another, but I think it's near impossible to claim to have disproven something...
Second, getting your work published is hard enough without picking fights. If the authors of this paper had been unlucky, and this disgruntled guy had been chosen to review their work, they would've had a massive fight on their hands to get it published. The alternative is that you get "lucky" and avoid reviewers who object to your conclusion and you just end up screwing some people over and making enemies in your field. Why bother? Work that is truly not reproducible is destined to become obsolete in the long run.
One thing I found weird in this guy's rant was where he claimed he had already shown that the authors approach was impossible...really? How do you show that? By trying it yourself and failing? "Your approach can't possibly work because I tried it and it didn't work". Doesn't that make him the first one to pick a fight? If he already voiced this contrary viewpoint at a conference or something, maybe the authors requested that the editors specifically not choose him as a reviewer? In which case he screwed himself out of a chance to avoid this whole situation...
Bayman FAIL!
A theory should be considered scientific IF and only IF it is falsifiable. To falsify a theory is to disprove it. I suggest you read up on Popper...
Popper!?? That guy's dead! And that was before the big-pharma ghost-writers were on the scence...
OK that raises an interesting point, so on a more serious note, Popperian positivism is pretty useless in experimental biomedicine becuase
1) As Hawking says, "If the observations disagree with the predictions, one has to discard or modify the theory. (In practice, people often question the accuracy of the observations and the reliability and moral character of those making the observations)".
There are more than enough technical challenges to fuel the former, and failing that, enough therapeutically biased mercenary scientists to let us comfortably ignore inconvenient results.
2) Biology is about exceptions. The observation of a random mutation does not disprove the Law/Theory of heredity. If biologists listened to Popper, we wouldn't have any theories because every single one has exceptions. So we have to build biological theories by "consensus observations" rather than falsification. Our predictions are probablistic, whereas falsification is only useful if your model is supposed to be deterministic.
Interesting.
Though I will point out that the paper/comment in question was a physics paper so it's conceivable that the 'fundamentally impossible' approach discussed in the OP may well have been mathematically impossible, or contrary to some immutable law (like conservation of energy or something) rather than "I tried it and it didn't work, therefore it's impossible"
the paper/comment in question was a physics paper
Good point. I guess that's why physicists are so militant...
The part that has me scratching my head on this is that he emphasized how his comment was clearly the exception -- that previous comments were allowed more pages, comments published during the saga had more space, etc.
It seemed that he's trying to make two contradictory points -- first, that it's really hard to publish a comment, and second, that his particular comment was specifically picked on. He can't have it both ways. If other comments were published easily, then it isn't hard to publish a comment.
I'm curious why his comment was treated so differently than all the others that came out at the same time in the same journal. He never explains that, and it makes me wonder if his comment was as courteous, clear, and obvious as he thinks.
The list of proposed rules he offers include some that make sense, but several are completely impractical, naive, pointless, and unenforceable. If he's serious about them and thinks they're clear and practical, it makes me hesitant to accept his judgement about his own comment.
I have a hard time deciding whether "blacklisting" certain reviewers would be a good thing or not. Sometimes reviewers' comments clearly indicate they have not read the paper and are not doing their job. But If you flagged every reviewer you don't agree with I'm sure it could quickly get out of hand. Sometimes I rather like the idea of reviewers not being anonymous, just so there is some semblance of accountability on their part... But it would be a career suicide.
The list of proposed rules he offers include some that make sense, but several are completely impractical, naive, pointless, and unenforceable.
I had the same thought as well. However some of his suggestions would be much more plausible under a revised online publishing-reviewing-commenting system...and this gets a bit to AC's point about non-anonymous reviewers.
Non-anonymous reviews would be nice...expressed through a transparent online commenting and reviewing system this would go a long way in preventing disasters like this one. It would speed up the primary review process and make post-publication commenting and discussion easy. I'm thinking PLOS One-esque but more integrated into the current mainstream publishing process. It's gotta happen eventually, if only because the review process is getting so lengthy and painfully redundant.
The strength of such a system would be that the literature dynamically evolves. Which might be an obstacle to getting publishers aboard, since dust-collecting print copies of journals would become worthless...although we're pretty much there already I guess.
To me the whole story is just a bunch of whining unless he cites the original paper and puts his comment on the web somewhere, anywhere. His account should be secondary to those original materials. He could potentially remain anonymous aswell so as not to further adversely affect his/her career. If his story is true he shouldn't be afraid of anything (apparently his lifes work has been disproven). I'm not saying I personally would read the comment but it would make some great blog material on a physics blog.
I'm with iayork on this one. Something doesn't pass the smell test.
If you want to grab big boobs in your hand or want to fuck a massive pussy then book our sexy call girl tonight and encounter wild sex with them.
Dwarka Escorts
Vaishali Escorts Service
Escorts in Hauz Khas
East of Kailash Call Girls
Call Girls in Connaught Place
Post a Comment