Main Area and Open Discussion > Living Room
Peer Review and the Scientific Process
IainB:
Yes, I read about this and remain skeptical in the absence of solid proof, either way.
IainB:
A surprising programme from BBC Radio 4: Everything We Know Is Wrong (click on link to download/hear the programme)
Written notes:
Every day the newspapers carry stories of new scientific findings. There are 15 million scientists worldwide all trying to get their research published. But a disturbing fact appears if you look closely: as time goes by, many scientific findings seem to become less true than we thought. It's called the "decline effect" - and some findings even dwindle away to zero.
A highly influential paper by Dr John Ioannidis at Stanford University called "Why most published research findings are false" argues that fewer than half of scientific papers can be believed, and that the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true. He even showed that of the 49 most highly cited medical papers, only 34 had been retested and of them 41 per cent had been convincingly shown to be wrong. And yet they were still being cited.
Again and again, researchers are finding the same things, whether it's with observational studies, or even the "gold standard" Randomised Controlled Studies, whether it's medicine or economics. Nobody bothers to try to replicate most studies, and when they do try, the majority of findings don't stack up. The awkward truth is that, taken as a whole, the scientific literature is full of falsehoods.
Jolyon Jenkins reports on the factors that lie behind this. How researchers who are obliged for career reasons to produce studies that have "impact"; of small teams who produce headline-grabbing studies that are too statistically underpowered to produce meaningful results; of the way that scientists are under pressure to spin their findings and pretend that things they discovered by chance are what they were looking for in the first place. It's not exactly fraud, but it's not completely honest either. And he reports on new initiatives to go through the literature systematically trying to reproduce published findings, and of the bitter and personalised battles that can occur as a result.
Producer/Presenter: Jolyon Jenkins.
--- End quote ---
Renegade:
Yes, I read about this and remain skeptical in the absence of solid proof, either way.
-IainB (August 27, 2014, 11:45 PM)
--- End quote ---
The issue is mired down with too much baggage. The best one can hope for is just a gamble.
Renegade:
A surprising programme from BBC Radio 4: Everything We Know Is Wrong (click on link to download/hear the programme)
Written notes:
Every day the newspapers carry stories of new scientific findings. There are 15 million scientists worldwide all trying to get their research published. But a disturbing fact appears if you look closely: as time goes by, many scientific findings seem to become less true than we thought. It's called the "decline effect" - and some findings even dwindle away to zero.
A highly influential paper by Dr John Ioannidis at Stanford University called "Why most published research findings are false" argues that fewer than half of scientific papers can be believed, and that the hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true. He even showed that of the 49 most highly cited medical papers, only 34 had been retested and of them 41 per cent had been convincingly shown to be wrong. And yet they were still being cited.
Again and again, researchers are finding the same things, whether it's with observational studies, or even the "gold standard" Randomised Controlled Studies, whether it's medicine or economics. Nobody bothers to try to replicate most studies, and when they do try, the majority of findings don't stack up. The awkward truth is that, taken as a whole, the scientific literature is full of falsehoods.
Jolyon Jenkins reports on the factors that lie behind this. How researchers who are obliged for career reasons to produce studies that have "impact"; of small teams who produce headline-grabbing studies that are too statistically underpowered to produce meaningful results; of the way that scientists are under pressure to spin their findings and pretend that things they discovered by chance are what they were looking for in the first place. It's not exactly fraud, but it's not completely honest either. And he reports on new initiatives to go through the literature systematically trying to reproduce published findings, and of the bitter and personalised battles that can occur as a result.
Producer/Presenter: Jolyon Jenkins.
--- End quote ---
-IainB (August 29, 2014, 08:11 AM)
--- End quote ---
Interesting. I suppose that it sort of explains why medical advice changes every few years. Yesterday X was great, and today X is bad.
IainB:
Yes, it could explain such medical "fads", but the more informed amongst us would probably have woken up to those years ago.
The paper seems to be well substantiated (independently observable evidence and reference sources) and thus generally provable/TRUE. There's nothing especially new about it per se, except that it has been reported on by the BBC - infamous for promulgating their religio-political bias and for pushing pseudo-science (e.g., Rotherham, 28Gate, etc.). I suspect they produced the programme by mistake - it probably missed going through their usual internal censorship gate.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version