Hello,
I am a Masters Student at National Centre for Human Genome Studies and Research, India. I have been discussing experimentation designing for my senior's dissertation when I came across a strange yet deeply purturbing issue which i really want to share with the Scientific community here.
I came across a paper in very reputed scientific journal that had quite questionable data in terms of appropriate Control selection and experiment analysis, yet what bothered me is that inspite of such irregularities, it managed to get published in one of those so called "journals of very high repute!" Moreover, not even once, the author tried to relate/design his experiment to simulate physiological conditions but at the end he is simply extrapolating his data hypothetically to make a statement at physiological levels!
All my concerns regarding that paper were confirmed by senior PI's. I am not in favor of creating any controversies by naming the paper but with such papers keep on getting published in good journals, I am sure the day is not far away when we'll lose our trust in "good" journals?
Kind Regards,
Tarun Gupta
MSc Human Genomic
National Centre for Human Genome Studies and Research
Panjab University
Chandigarh-India
I am a Masters Student at National Centre for Human Genome Studies and Research, India. I have been discussing experimentation designing for my senior's dissertation when I came across a strange yet deeply purturbing issue which i really want to share with the Scientific community here.
I came across a paper in very reputed scientific journal that had quite questionable data in terms of appropriate Control selection and experiment analysis, yet what bothered me is that inspite of such irregularities, it managed to get published in one of those so called "journals of very high repute!" Moreover, not even once, the author tried to relate/design his experiment to simulate physiological conditions but at the end he is simply extrapolating his data hypothetically to make a statement at physiological levels!
All my concerns regarding that paper were confirmed by senior PI's. I am not in favor of creating any controversies by naming the paper but with such papers keep on getting published in good journals, I am sure the day is not far away when we'll lose our trust in "good" journals?
Kind Regards,
Tarun Gupta
MSc Human Genomic
National Centre for Human Genome Studies and Research
Panjab University
Chandigarh-India
Questionable Data in reputed journals. Where are we heading to?
username: Elsie Mon, 04 Feb 2008 10:21 AM
Oh...this is something that happens all the time in science. It is frustrating because even if a paper in a highly respected journal (let's say Science Cell or Nature) is debunked by the community in later publications, the first author still gets to put a "Big 3" paper on his or her resume...and hiring committees and grant reviewers see "Big 3" and go "oooooohhhh!" and don't look any further. It is very frustrating. The lesson to be learned here: A lot of really good, groundbreaking science is published in field specific journals. Judge by the content, not by the location. The best thing you can do (and which is what I have done) is do the experiment right, publish it, refer to the bad paper and in your discussion say something like "our results conflict with ______. One possible explanation is that their controls were screwy" of course you have to say it in a much more professional way.
Questionable Data in reputed journals. Where are we heading to?
username: Justin Mon, 04 Feb 2008 12:37 PM
Happens all the time. It is well-known in the field that ~50% of all material published in Cell, Science, and Nature have glaring problems. We are not talking about inherent ambiguities of science, but the accuracy and quality of the actual printed material. Unfortunately, in the field of publishing, there is a lot more than meets the eye about what gets published. Many times, it's about politics, position, power, or connections.
Interesting Article about "Ethics Educator" Grant
username: Eric Mon, 04 Feb 2008 03:52 PM
Tarun,
Your frustrations are common to us all. Elsie, Justin, and Dave have given excellent assesments of this problem, and there is little else to add to their posts other than to say that science is in reality a very political business far removed from purity. Another problem you may have noticed, one that relates to Elsie's observations, is the tendency of authors to reference lower-quality studies from big-name labs pblished in big-name journals rather than a more appropriate and more scientifically-sound study published in field-specific journals. There is a tendency to assume a Cell/Nature paper must be good science, and a not-so-subtle desire to curry favor with powerful people by giving them undue credit and reference. It happens all the time, and I see no end to the problem short of eliminating this absurd construction called the "impact factor."
Your frustrations are common to us all. Elsie, Justin, and Dave have given excellent assesments of this problem, and there is little else to add to their posts other than to say that science is in reality a very political business far removed from purity. Another problem you may have noticed, one that relates to Elsie's observations, is the tendency of authors to reference lower-quality studies from big-name labs pblished in big-name journals rather than a more appropriate and more scientifically-sound study published in field-specific journals. There is a tendency to assume a Cell/Nature paper must be good science, and a not-so-subtle desire to curry favor with powerful people by giving them undue credit and reference. It happens all the time, and I see no end to the problem short of eliminating this absurd construction called the "impact factor."
Interesting Article about "Ethics Educator" Grant
username: Baoloa Mon, 04 Feb 2008 06:47 PM
An interesting perspective on this issue was recently published in Nature, where Pubmed was trawled for duplicate publications:
http://www.nature.com/nature/journal/v451/n7177/full/451397a.html
A good comment was also made here:
http://pipeline.corante.com/archives/2008/01/30/recycle_reuse_republish.php
However, one would have to be pretty inept to be caught by this method.
http://www.nature.com/nature/journal/v451/n7177/full/451397a.html
A good comment was also made here:
http://pipeline.corante.com/archives/2008/01/30/recycle_reuse_republish.php
However, one would have to be pretty inept to be caught by this method.
Ethics and Misconduct in research
username: Garth Tue, 05 Feb 2008 10:57 AM
The Office of Research Integrity (ORI) is a high-level office in the Department of Health and Human Services (which is the home department of may research funding gov't agencies, including NIH). Its mission is to monitor (and sometimes spearhead) investigations of misconduct and promote responsible conduct of research through policy, education, and regulatory activities. There are regular reports of ethics violations, and established researchers loosing HHS and Fed gov't funding, their positions, and fines against university wide ethics violations.
It is worth checking out the ORI website, and keeping abreast of what ORI does. Most scientists (young and old) are not aware that the Federal Government has an appointed office of scientifically trained (i.e., with PhDs and MS degrees, and years of research/lab experience) investigators to deal with ethics and integrity violations. Being an active member of 'the science community' means promoting ethics and integrity, and ORI is a great way for you to become more involved and aware of what is being done in ethics education for scientist. And besides, if you do not become aware an involved in what ORI does, and pass this on to your friends (or eventually the young scientists you will mentor), then who will?
http://ori.dhhs.gov/
It is worth checking out the ORI website, and keeping abreast of what ORI does. Most scientists (young and old) are not aware that the Federal Government has an appointed office of scientifically trained (i.e., with PhDs and MS degrees, and years of research/lab experience) investigators to deal with ethics and integrity violations. Being an active member of 'the science community' means promoting ethics and integrity, and ORI is a great way for you to become more involved and aware of what is being done in ethics education for scientist. And besides, if you do not become aware an involved in what ORI does, and pass this on to your friends (or eventually the young scientists you will mentor), then who will?
http://ori.dhhs.gov/
Questionable Data in reputed journals. Where are we heading to?
username: Tarun Wed, 06 Feb 2008 10:28 AM
affiliation/organization: National Centre for Human Genome Studies & Research
affiliation/organization: National Centre for Human Genome Studies & Research
Being a new kid in the block, these are real eye openers. I still wonder how such articles escape editors eye! Do they simply believe that if a paper is from so-and-so lab, it must be OK!
As an outset, These publishing houses are more of buisnesses than just science journals. They charge a significant amount of money from its readers and its their responsibility to maintain "quality" of information. They simply can't sell junk for so much money! There must be a stringent "quality control mechanism".
Perhaps, at least "Big3", should take a note of this and include someone from ORI or people with special training in ethical aspects of science. I wonder if editors are reading this!
As an outset, These publishing houses are more of buisnesses than just science journals. They charge a significant amount of money from its readers and its their responsibility to maintain "quality" of information. They simply can't sell junk for so much money! There must be a stringent "quality control mechanism".
Perhaps, at least "Big3", should take a note of this and include someone from ORI or people with special training in ethical aspects of science. I wonder if editors are reading this!
Questionable Data in reputed journals. Where are we heading to?
username: S.H. Wed, 06 Feb 2008 10:37 AM
This thread raised a very valid point. From the past few years, 0nly 10%-20% of the papers presented in my lab's journal club have somehow proved their points by the evidence they provided in the paper. Good quality papers are hard to come across.
Questionable Data in reputed journals. Where are we heading to?
username: Elsie Wed, 06 Feb 2008 11:09 AM
Another good thing to remember is that when you are asked to peer review articles...it is your job to protect the literature! It is easy to fall into the trap of feeling bad for some poor postdoc or graduate student who did a lot of work... but if their data is bad, publishing it validates it and it makes it that much harder for the next poor postdoc or graduate student who might actually be on the right track.
Questionable Data in reputed journals. Where are we heading to?
username: Ale Wed, 06 Feb 2008 01:53 PM
But Elsie, a reviewer can catch up missing controls or retouched pictures, you cannot see pure fake data or the double publications (most of which were found to be submitted at the same time to different journals since teh publication dates were to close).
I also think there is a part of this due to the good reputation of some big senior authors. Revieweers and Editors trust in her internal filter before sending the manuscript out and labs with no reputation (no one knows you) have to prove themselves right on every paper. I agree with Tarun on this.
0 comments:
Post a Comment