Main Area and Open Discussion > Living Room
Peer Review and the Scientific Process
IainB:
@barney: It's not a bad analogy. You seem to have hit the nail on the head.
From experience, it is largely the same for program critique/review in this forum as it usually is elsewhere in mainstream programming.
For example, it was whilst I was lecturing that I learned to code in an assembler programming language, and I had a bunch of useful minds nearby to review/critique my efforts - which, whist being a somewhat humbling experience, helped me a lot and taught me to avoid underestimating my own level of ignorance.
Later on, I became increasingly involved in developing/supporting commercial computer programs used for solving problems in scientific/mathematical/statistical analysis and modelling, mostly written in Fortran. These programs absolutely had to work spot-on and with a clearly-identified margin of error, because the outputs were critically important. For example, including things such as: audience research and identifying TV audience viewing habits so as to target adverts appropriately; the design of a new missile's aerofoil; the optimum mix of materials going into a steel smelting furnace to make a particular kind of steel; predictions about wave height in the North Sea (for scheduling safe voyages to/from oil rigs); the econometric predictions of the UK's economy.
The only way you could be sure to achieve some degree of assurance that a program worked and was fit-for-purpose was through rigorous testing and review by the users - typically Operations Research people, and/or mathematicians/scientists/engineers who used the programs for their work on a daily basis.
IainB:
I was reading an article the other day on the distortion/misuse of science, when I came across a link to an explanation of what "Post-Modern Science" was, and why it is in use today. It is a different and political approach to science, and it could help to explain why the abuse/misuse of science and why the lack of rigorous peer review occurs:
Post Normal Science:
Politicized Science? An excuse to stretch facts?
The Post Normal Science doctrine comes from a 1993 paper by the philosophers Silvio
Functowitz and Jerome Ravetz (refer Funtowicz and Ravetz "Science for the Post-Normal Age", Futures, 25/7 September 1993, 739-755).
Post Normal Science is promoted as appropriate when science is complex, uncertain, and the situation involves high stakes. It may be a mistake to call it science at all, since it appears to be a method for deciding on a course of action. In Post Normal Science, the players in the scientific quest are extended beyond scientists to include an “extended peer group” and even investigative journalists. Post Normal Science might better be referred to as participatory science.” The standards of evidence are relaxed to allow anecdotal information that ordinarily would not be acceptable in formal scientific investigations.
There are problems. Take the case of the possibility of an atomic bomb in 1942. The science was uncertain would it work, could isotope separation be performed? The stakes were high would the Germans get it first? The circumstances met the conditions for using Post Normal Science. But there was no extended peer group and investigative journalism was forbidden . The decisions were made by a small group of politicians, military leaders, and scientists. Lesson: If the stakes are high or there are competitors, secrecy is often essential. ...
...
...Post Normal Science should be seen as a political device that can be used to push a particular policy.
Source: Article here and paper here.
--- End quote ---
As one commenter to the article mentions:
"...a distressing number of posters are using broad brush terms to challenge the integrity of the many many scientists participating in the IPCC process. This is just wrong. Remember that the politicians have the final say in the contents of the final report."
--- End quote ---
That may be a fair comment, but it does not explain why only relatively few of these "many many" scientists seem to be standing up and denouncing what manifests as a generally unprincipled misuse/abuse of science and the peer review process. It seems to be simple dishonesty.
IainB:
On the other hand, here is what seems to be a rather tongue-in-cheek proposal to ban peer review altogether: Peer Evil – the rotten business model of modern science
IainB:
Taki's Magazine has a disturbing post about the demise of not just the peer review process, but of science and rational thought:
(Snippets copied below sans embedded hyperlinks/images.)
A Requiem for Science
As a science geek from way back—Andrade and Huxley were favorite childhood companions—I try to keep tabs on that side of things. This can be disheartening. To quote from that intergalactic bestseller We Are Doomed:
Scientific objectivity is a freakish, unnatural, and unpopular mode of thought, restricted to small cliques whom the generality of citizens regard with dislike and mistrust. Just as religious thinking emerges naturally and effortlessly from the everyday workings of the human brain, so scientific thinking has to struggle against the grain of our mental natures. There is a modest literature on this topic:...
...In a society such as the modern West, where intelligence is declining, where fertility trends are dysgenic, where cognitive elites enforce assent to feel-good ideological claptrap and the mass of citizenry is absorbed in frivolities, science hovers always on the edge of extinction....
--- End quote ---
--- End quote ---
IainB:
Here is something highlighting the need for rigorous Peer Review of the Scientific Process in an area of medicine, but pointing out that it might not be achievable:
Let's make up a fictional story where the main character is a brilliant scientist, a single-handed winner of not just one, but two Nobel prizes (say, one for science, and one for peace), and with scientific and academic accolades aplenty. He is so highly-regarded that "he cannot be wrong" (fallacy of the appeal to authority). He makes some statements about how high-dosage vitamins can increase your health, well-being, and longevity - but this is substantiated by and based on no real or substantive scientific research. Yet he insists that vitamins will cure/prevent all manner of ills, including cancer, the common cold, influenza, heart disease and even psychological disorders - the list is huge. Everybody believes him - why shouldn't they?
A humungus and profitable new market for the production, sale, marketing and consumption of food-additives/vitamins is suddenly born, thus achieving The Holy Grail of Marketing - the creation of an entirely new market.
But then medical research results start to trickle in that seem to consistently indicate a relatively strong correlation between premature death from various causes (including heart disease and cancer) and the high-level consumption of vitamins. Our scientist refutes the results of any research that is contradictory to his statements - his status as a scientist and the sheer force of his personality are sufficient, it seems, to substantiate his statements. There can be no debate.
Time passes, the market for vitamins grows and makes many people rich, because people believe what they are told by the marketers that vitamins can increase their health, well-being, and longevity and even cure/prevent all manner of ills, including cancer, the common cold, influenza, heart disease and even psychological disorders.
But still the medical research results continue to trickle in that seem to consistently indicate a relatively strong correlation between premature death from various causes (including heart disease and cancer) and the high-level consumption of vitamins. The mounting pile of evidence pointing to the conclusion that the high-level consumption of vitamins is not only ineffective in promoting health, but also potentially harmful seems irrefutable.
So medical scientists and doctors who are charged with looking after the public health and who may have taken the Hippocratic oath do not prescribe vitamins to any of their patients, for the simple reason that stochastically, vitamins are proven to be at best useless and at worst potentially harmful and leading to premature death.
Then a farmer lies dying of viral pneumonia from having Swine Flu. He is in hospital in a coma and on life-support, and the prognosis is unavoidable death. The doctors eventually recommend turning off life support as there is no hope of recovery.
His family have read about how someone with a similar condition was restored to health by IV of high levels of vitamin C. They ask the doctors to give him the injections, before turning off life support. The doctors refuse - they have to.
The family bullies them into eventually doing it anyway - "What difference does it make?" (as Mrs Clinton might have put it). The patient starts to recover. After he comes out of the coma, the doctors initially refuse to give him any more IV of vitamin C, but again, under pressure from the family, they relent but just give him a harmless minimal dosage, so the family resort to smuggling in and secretly feeding the patient high-dosage oral vitamin C. He fully recovers, goes home to his farm and continues recuperating with high-dosage oral vitamin C.
Do you think that this fiction would make for a good story? I do, but, oddly it is all true.
The scientist was Linus Pauling, and you can read something about him here: The Vitamin Myth: Why We Think We Need Supplements
The patient was the subject of a New Zealand TV documentary in 2010, and you can read about it here: Vitamin C Recovery From Viral Pneumonia in New Zealand Farmer
So, evidently, it would seem that there are circumstances/conditions where IV of high levels of vitamin C and ingestion of high levels of oral vitamin C can be beneficial - even life-saving.
What would be expected from this is some solid medical research to establish exactly what those circumstances/conditions are, and why the vitamin C is effective in those cases, whereas it might be ineffective or harmful in most other cases.
The research would, of course, be subject to extensive and rigorous peer review before any results were published.
However, I would predict that that research is unlikely to take place at more than a glacial speed - simply because most research is funded by Big Pharma who are only likely to be interested in funding research which leads to a new, profitable patented drug or medical procedure. You can't patent vitamins that occur in nature - though I recall reading elsewhere that Big Parma had lobbied some US Senators to pass a bill that might allow them just that, making the current method of production of vitamins to the food-additive market illegal.
To a large extent, in medicine and in other areas, genuine scientific research and the scientific process seem to have been hijacked and monopolised by powerful commercial interests.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version