Author Topic: Peer Review and the Scientific Process (Read 146196 times)

IainB · « **Reply #25 on:** June 19, 2013, 11:00 PM »

@barney: It's not a bad analogy. You seem to have hit the nail on the head.
From experience, it is largely the same for program critique/review in this forum as it usually is elsewhere in mainstream programming.
For example, it was whilst I was lecturing that I learned to code in an assembler programming language, and I had a bunch of useful minds nearby to review/critique my efforts - which, whist being a somewhat humbling experience, helped me a lot and taught me to avoid underestimating my own level of ignorance.
Later on, I became increasingly involved in developing/supporting commercial computer programs used for solving problems in scientific/mathematical/statistical analysis and modelling, mostly written in Fortran. These programs absolutely had to work spot-on and with a clearly-identified margin of error, because the outputs were critically important. For example, including things such as: audience research and identifying TV audience viewing habits so as to target adverts appropriately; the design of a new missile's aerofoil; the optimum mix of materials going into a steel smelting furnace to make a particular kind of steel; predictions about wave height in the North Sea (for scheduling safe voyages to/from oil rigs); the econometric predictions of the UK's economy.
The only way you could be sure to achieve some degree of assurance that a program worked and was fit-for-purpose was through rigorous testing and review by the users - typically Operations Research people, and/or mathematicians/scientists/engineers who used the programs for their work on a daily basis.

IainB · « **Reply #26 on:** July 01, 2013, 01:32 AM »

I was reading an article the other day on the distortion/misuse of science, when I came across a link to an explanation of what "Post-Modern Science" was, and why it is in use today. It is a different and political approach to science, and it could help to explain why the abuse/misuse of science and why the lack of rigorous peer review occurs:

Post Normal Science:
Politicized Science? An excuse to stretch facts?

The Post Normal Science doctrine comes from a 1993 paper by the philosophers Silvio
Functowitz and Jerome Ravetz (refer Funtowicz and Ravetz "Science for the Post-Normal Age", Futures, 25/7 September 1993, 739-755).

Post Normal Science is promoted as appropriate when science is complex, uncertain, and the situation involves high stakes. It may be a mistake to call it science at all, since it appears to be a method for deciding on a course of action. In Post Normal Science, the players in the scientific quest are extended beyond scientists to include an “extended peer group” and even investigative journalists. Post Normal Science might better be referred to as participatory science.” The standards of evidence are relaxed to allow anecdotal information that ordinarily would not be acceptable in formal scientific investigations.

There are problems. Take the case of the possibility of an atomic bomb in 1942. The science was uncertain would it work, could isotope separation be performed? The stakes were high would the Germans get it first? The circumstances met the conditions for using Post Normal Science. But there was no extended peer group and investigative journalism was forbidden . The decisions were made by a small group of politicians, military leaders, and scientists. Lesson: If the stakes are high or there are competitors, secrecy is often essential. ...
...
...Post Normal Science should be seen as a political device that can be used to push a particular policy.

Source: Article here and paper here.

As one commenter to the article mentions:

"...a distressing number of posters are using broad brush terms to challenge the integrity of the many many scientists participating in the IPCC process. This is just wrong. Remember that the politicians have the final say in the contents of the final report."

That may be a fair comment, but it does not explain why only relatively few of these "many many" scientists seem to be standing up and denouncing what manifests as a generally unprincipled misuse/abuse of science and the peer review process. It seems to be simple dishonesty.

IainB · « **Reply #27 on:** July 01, 2013, 08:39 AM »

On the other hand, here is what seems to be a rather tongue-in-cheek proposal to ban peer review altogether: Peer Evil – the rotten business model of modern science

IainB · « **Reply #28 on:** July 02, 2013, 03:26 AM »

Taki's Magazine has a disturbing post about the demise of not just the peer review process, but of science and rational thought:
(Snippets copied below sans embedded hyperlinks/images.)

A Requiem for Science
As a science geek from way back—Andrade and Huxley were favorite childhood companions—I try to keep tabs on that side of things. This can be disheartening. To quote from that intergalactic bestseller We Are Doomed:
Scientific objectivity is a freakish, unnatural, and unpopular mode of thought, restricted to small cliques whom the generality of citizens regard with dislike and mistrust. Just as religious thinking emerges naturally and effortlessly from the everyday workings of the human brain, so scientific thinking has to struggle against the grain of our mental natures. There is a modest literature on this topic:...

...In a society such as the modern West, where intelligence is declining, where fertility trends are dysgenic, where cognitive elites enforce assent to feel-good ideological claptrap and the mass of citizenry is absorbed in frivolities, science hovers always on the edge of extinction....

IainB · « **Reply #29 on:** August 01, 2013, 01:02 AM »

Here is something highlighting the need for rigorous Peer Review of the Scientific Process in an area of medicine, but pointing out that it might not be achievable:
Let's make up a fictional story where the main character is a brilliant scientist, a single-handed winner of not just one, but two Nobel prizes (say, one for science, and one for peace), and with scientific and academic accolades aplenty. He is so highly-regarded that "he cannot be wrong" (fallacy of the appeal to authority). He makes some statements about how high-dosage vitamins can increase your health, well-being, and longevity - but this is substantiated by and based on no real or substantive scientific research. Yet he insists that vitamins will cure/prevent all manner of ills, including cancer, the common cold, influenza, heart disease and even psychological disorders - the list is huge. Everybody believes him - why shouldn't they?
A humungus and profitable new market for the production, sale, marketing and consumption of food-additives/vitamins is suddenly born, thus achieving The Holy Grail of Marketing - the creation of an entirely new market.

But then medical research results start to trickle in that seem to consistently indicate a relatively strong correlation between premature death from various causes (including heart disease and cancer) and the high-level consumption of vitamins. Our scientist refutes the results of any research that is contradictory to his statements - his status as a scientist and the sheer force of his personality are sufficient, it seems, to substantiate his statements. There can be no debate.

Time passes, the market for vitamins grows and makes many people rich, because people believe what they are told by the marketers that vitamins can increase their health, well-being, and longevity and even cure/prevent all manner of ills, including cancer, the common cold, influenza, heart disease and even psychological disorders.
But still the medical research results continue to trickle in that seem to consistently indicate a relatively strong correlation between premature death from various causes (including heart disease and cancer) and the high-level consumption of vitamins. The mounting pile of evidence pointing to the conclusion that the high-level consumption of vitamins is not only ineffective in promoting health, but also potentially harmful seems irrefutable.
So medical scientists and doctors who are charged with looking after the public health and who may have taken the Hippocratic oath do not prescribe vitamins to any of their patients, for the simple reason that stochastically, vitamins are proven to be at best useless and at worst potentially harmful and leading to premature death.

Then a farmer lies dying of viral pneumonia from having Swine Flu. He is in hospital in a coma and on life-support, and the prognosis is unavoidable death. The doctors eventually recommend turning off life support as there is no hope of recovery.
His family have read about how someone with a similar condition was restored to health by IV of high levels of vitamin C. They ask the doctors to give him the injections, before turning off life support. The doctors refuse - they have to.
The family bullies them into eventually doing it anyway - "What difference does it make?" (as Mrs Clinton might have put it). The patient starts to recover. After he comes out of the coma, the doctors initially refuse to give him any more IV of vitamin C, but again, under pressure from the family, they relent but just give him a harmless minimal dosage, so the family resort to smuggling in and secretly feeding the patient high-dosage oral vitamin C. He fully recovers, goes home to his farm and continues recuperating with high-dosage oral vitamin C.

Do you think that this fiction would make for a good story? I do, but, oddly it is all true.

The scientist was Linus Pauling, and you can read something about him here: The Vitamin Myth: Why We Think We Need Supplements

The patient was the subject of a New Zealand TV documentary in 2010, and you can read about it here: Vitamin C Recovery From Viral Pneumonia in New Zealand Farmer

So, evidently, it would seem that there are circumstances/conditions where IV of high levels of vitamin C and ingestion of high levels of oral vitamin C can be beneficial - even life-saving.
What would be expected from this is some solid medical research to establish exactly what those circumstances/conditions are, and why the vitamin C is effective in those cases, whereas it might be ineffective or harmful in most other cases.
The research would, of course, be subject to extensive and rigorous peer review before any results were published.

However, I would predict that that research is unlikely to take place at more than a glacial speed - simply because most research is funded by Big Pharma who are only likely to be interested in funding research which leads to a new, profitable patented drug or medical procedure. You can't patent vitamins that occur in nature - though I recall reading elsewhere that Big Parma had lobbied some US Senators to pass a bill that might allow them just that, making the current method of production of vitamins to the food-additive market illegal.
To a large extent, in medicine and in other areas, genuine scientific research and the scientific process seem to have been hijacked and monopolised by powerful commercial interests.

tomos · « **Reply #30 on:** August 01, 2013, 05:57 AM »

The scientist was Linus Pauling, and you can read something about him here: The Vitamin Myth: Why We Think We Need Supplements
-IainB (August 01, 2013, 01:02 AM)

-
~~I'm only half way through~~ the Atlantic article, but it is a real eye-opener! Haven't taken vitamins myself for donkeys years now, but that's not been from any kind of belief (or disbelief). I guess I do reckon though that if you want vitamin C, you could eat more brocoli (or whatever) ;-)

But still the medical research results continue to trickle in that seem to consistently indicate a relatively strong correlation between premature death from various causes (including heart disease and cancer) and the high-level consumption of vitamins. The mounting pile of evidence pointing to the conclusion that the high-level consumption of vitamins is not only ineffective in promoting health, but also potentially harmful seems irrefutable.
-IainB (August 01, 2013, 01:02 AM)

-
I was wondering was that anecdotal but it seem to have been shown in tests according to the article. Unfortunately info is given about these is incomplete. I see the article is an excerpt from a book. May be more info there. Maybe not. I'm presuming The Atlantic checked out his sources...

In 2007, researchers from the National Cancer Institute examined 11,000 men who did or didn't take multivitamins. Those who took multivitamins were twice as likely to die from advanced prostate cancer.

-
this I found not so convincing. It could be another factor related to the 'type' of person who tends to take vitamins. Are they 'worriers'? More stressed? Different lifestyles?

========

Would be an interesting scenario if vitamins ended up getting a health warning like with cigarettes.

Renegade · « **Reply #31 on:** August 01, 2013, 08:02 AM »

However, I would predict that that research is unlikely to take place at more than a glacial speed - simply because most research is funded by Big Pharma who are only likely to be interested in funding research which leads to a new, profitable patented drug or medical procedure. You can't patent vitamins that occur in nature - though I recall reading elsewhere that Big Parma had lobbied some US Senators to pass a bill that might allow them just that, making the current method of production of vitamins to the food-additive market illegal.
To a large extent, in medicine and in other areas, genuine scientific research and the scientific process seem to have been hijacked and monopolised by powerful commercial interests.
-IainB (August 01, 2013, 01:02 AM)

+1

I read "The Atlantic" article. I'm not convinced.

There are important distinctions between natural and synthetic compounds. Often it is impractical to create synthetic duplicates of natural compounds, or to create a synthetic mix of compounds. Scientists simply are not capable of explaining what a lot of things do, then they publish poo-poo articles dissing this or that because they screwed the pooch by limiting their range... oh bother. Iain can explain better than I can.

As silly little example of how things matter, the scent of dill and spearmint share the same chemical formula. They smell different because they are mirror images of each other though.

Would be an interesting scenario if vitamins ended up getting a health warning like with cigarettes.
-tomos (August 01, 2013, 05:57 AM)

They will be illegal soon. The UN is working on that. Codex Alimentarius is a part of the UN's Agenda 21 and aims to make vitamins either illegal, by prescription, or limited to clinically ineffective (low) dosages. e.g. The Codex Alimentarius recommendation for vitamin D (or E - my memory is fuzzy at the moment) is 10 IU. 10,000 IU is where you make contact with reality. (Actually, that could be 1,000 IU - I forget. I heard a doctor talk about it, and remember that she takes 10,000 IU of vitamin D (E?) daily, but not 100% sure about what she said was the minimum.)

If there's a hit piece on vitamins (or just about anything else in the MSM), I figure that I should do the opposite of what it recommends.

IainB · « **Reply #32 on:** August 01, 2013, 10:36 AM »

Please don't take what I posted as my making a recommendation for or against taking high dosage vitamins - I am not making any such recommendation.
What I am interested in is the medical research that needs to be done - to the satisfaction and scientific standards required by the doctors - to establish if/when high dosage vitamins are beneficial, or otherwise. This does not seem to be something that can be entrusted to Big Pharma, as they would likely kill the idea anyway, only to revive it if they get the legal copyright/monopoly on vitamins and food additives. We just need to understand more about it first.
We also need to keep our heads firmly screwed on and retain a skeptical stance at all times:

The Atlantic is presumably an unbiased organ, but you never know - for example, they could have been funded for writing the article about the myth of vitamins, by Big Pharma.
The article in ZIMBIO re the NZ farmer's recovery documentary is in the "Natural Alternative Health Therapies" section, apparently written by an American MD who seems to work in the area of "natural medicine" and who may be effectively promoting the book Curing the Incurable by Dr Thomas Levy. That rings all kinds of alarm bells with me, as it is a domain that is typically packed jam full of sundry quacks, frauds and crackbrained theorists, one of its longest-running scams being homeopathy. (My favourite is still phrenology.)

I retain an open mind on high dosage vitamins, and await some scientific proof either way - proof that the medical fraternity can/will accept.
Similarly, I retain an open mind on the farmer's spectacular recovery, but I regard the statement that:

...the doctors attributed the dramatic improvement to "turning patient into a prone position", and not to the IV vitamin C.

- as laughable. (I was gobsmacked by this when I heard it, watching the documentary in 2010.)
I mean, fair enough, all you have is only 1 case of empiric evidence, and it would be irrational to take it as proof and to attribute the recovery to the IV vitamin C, but to attribute the dramatic improvement to "turning patient into a prone position" seems egregiously irrational. For example, if that was all it took to remove the patient from death's door, then why didn't they do that (as a recognised medical procedure) before he went into coma and after, instead of saying it was time to take the now comatose patient off life support and let him die? It's moronic. I suspect they were probably desperately clutching at straws to avoid conceding that it might have been the IV vitamin C and wanted to avoid accusations of malpractice. However, what they did/said actually could potentially raise genuine questions about malpractice.

I think there are some valid questions that the doctors need to answer. As it says at Laleva.org - Vitamin C Saves Man Dying of Viral Pneumonia (my emphasis)
(Copied below sans embedded hyperlinks/images.)

...Interviewed in part two was the Principle Advisor to the Health Ministry and Senior Intensive Care Specialist, David Galler who denied that the intravenous Vitamin C was a contributing factor in the Allan Smith's recovery. He proclaimed that the recovery could have been just as likely from a "bus driving by" as the high dose Vitamin C . When asked what he would need as proof to that Vitamin C is effective, he replied he would need a randomized controlled trial, such as those for new drug approval funded by a pharmaceutical company.

Three Randomized Placebo Controlled Studies
Apparently Dr Galler is unaware of three double blind placebo controlled studies of IV Vitamin C in critically ill patients in the ICU. These studies were published in Dr. Galler's own peer reviewed specialty medical literature. (1-5)

These three studies showed reduced mortality and reduced time on ventilators for septic and critically ill patients in the ICU setting. In addition, numerous other studies have measured blood vitamin C levels in critically ill patients in the hospital showing Vitamin C is typically depleted with levels below 25 % of healthy individuals.( Nathens et al)

(6-11) As Dr Levy points out in Part Four of the Series (see below), there are thousands of studies over 70 years in the medical literature showing effectiveness, and safety of Vitamin C for viral illness. Dr Levy's book cites 1200 such articles supporting the use of Vitamin C.

Denying the Blatantly Obvious
Dr Galler appeared on New Zealand television claiming to be an authority and medical expert in the care of the ICU critically ill patient. To then make statements amounting to a public admission of ignorance of his own specialty literature is a profound embarrassment to him and to the Ministry of Health that appointed him Advisor. For Allan Smith's ICU doctors to witness a patient's dramatic recovery from sure death, and then deny the effectiveness of the treatment is astounding display of denying the obvious, and an embarrassment to the medical system in New Zealand. This is tantamount to holding up a hand in front of a person's face who then steadfastly denies a hand is in front of his face. It can also be compared to the ridiculous scenario of "denying" that parachutes are lifesaving, and insisting on "proof" by requiring a placebo controlled study. Two men jump out of a plane, one with a parachute and one without a parachute, to "prove" parachutes are effective.

I would like to know more of Dr David Galler's views on this case. He will presumably have some rational reasons for saying what he did, and for refuting the three studies referred to (and any others) - assuming that he knew of them. If he did not know of them, then I think the taxpayers should be told why - especially in this age of enlightened government accountability and transparency.

Renegade · « **Reply #33 on:** August 01, 2013, 10:58 AM »

The phrase "presstitute" was coined because the MSM are mostly corporate whores. I have zero faith in the media anymore.

And I seriously doubt that we will get any reliable information from the medical establishment on vitamins. Ever. Manipulating studies is very easy to do when you sub out things that work and sub in things that don't work and use a bastardized generic description to cover both and mislead people.

These kinds of tactics are par for the course in more than just medicine. We see them all the time in other completely unrelated issues. It's just meant to confuse an issue.

The "turning patient into a prone position" nonsense nearly made me vomit.

However, I don't think I would be so kind in my assessment of what's going on there. I'll skip that for the sake of civility here.

As we will not get decent answers from a deeply conflicted industry, I can only come to the conclusion that the best course of action is to do the opposite of their recommendations in every case where it makes sense, e.g. wrt vitamins, etc. It's a pragmatic approach. I simply do not have the time and resources to get 50 billion PhDs in medical douchewaddery.

I can also go on past experience where I've found traditional western allopathic medicine to be at best useless. Again, I'll skip that for the sake of being civil.

I WISH we had actual science in medicine. We don't. It's a joke.

Stoic Joker · « **Reply #34 on:** August 01, 2013, 11:32 AM »

Would be an interesting scenario if vitamins ended up getting a health warning like with cigarettes.
-tomos (August 01, 2013, 05:57 AM)

Okay...Then I'd be pissing myself laughing.

Renegade · « **Reply #35 on:** August 01, 2013, 11:35 AM »

Would be an interesting scenario if vitamins ended up getting a health warning like with cigarettes.
-tomos (August 01, 2013, 05:57 AM)

Okay...Then I'd be pissing myself laughing.
-Stoic Joker (August 01, 2013, 11:32 AM)

Search for info on Agenda 21 and Codex Alimentarius. You'll not just piss yourself, you'll shit yer droors as well.

Renegade · « **Reply #36 on:** August 03, 2013, 01:30 AM »

More funny stuff from politicians about vitamins:

http://healthimpactn...reatens-supplements/

IainB · « **Reply #37 on:** August 03, 2013, 10:15 PM »

Good news:
(Copied below sans embedded hyperlinks/images.)

University of California to allow open access to new academic papers
On November 1, faculty will be automatically enrolled in the UC's open access policy.
by Megan Geuss - Aug 3, 2013 8:45 pm UTC

The University of California—an enormous institution that encompasses 10 campuses and over 8,000 faculty members—introduced an Open Access Policy late last week. This policy grants the UC a license to its faculty's work by default, and requires them to provide the UC with copy of their peer-reviewed papers on the paper's publication date. The UC then posts the paper online to eScholarship, its open access publishing site, where the paper will be available to anyone, free of charge.

Making the open access license automatic for its faculty leverages the power of the institution—which publishes over 40,000 scholarly papers a year—against the power of publishers who would otherwise lock content behind a paywall. “It is much harder for individuals to negotiate these rights on an individual basis than to assert them collectively,” writes the UC. “By making a blanket policy, individual faculty benefit from membership in the policy-making group, without suffering negative consequences. Faculty retain both the individual right to determine the fate of their work, and the benefit of making a collective commitment to open access.”

Faculty members will be allowed to opt out of the scheme if necessary—if they have a prior contract with a journal, for example. Academic papers published in traditional journals before the enactment of this policy will not be made available on eScholarship at this time.

“As faculty members, we are asserting our control over the publication of scholarly research and recognize the responsibility for making that process sustainable and true to the intentions of scholars,” explained the UC on a FAQ page. “The faculty are also sending a strong collective message to publishers about the values and the system we would like in the future.”

The move comes at a time when the US federal government is heavily promoting open access. In February 2013, the White House announced that all science papers produced through federal funding would be made available to the public one year after their publication, and the Obama Administration is working to extend that policy to cover the information published by all federal agencies. Many other institutions have adopted open access policies, including 177 other universities and the World Bank.

As Chris Kelty, associate professor at the Department of Information Studies at UCLA, explained in a series of videos on the UC's eScholarship site: ”Everybody benefits from this really, the faculty benefit from this because their work's more widely available, it might come in for higher citations. The University benefits because the profile of the University is higher and it might send a message to Sacramento about our commitment to research. And the public benefits—whether you're a K-12 teacher, or someone in an emergency room looking for an article, or someone in business trying to get a patent, everyone in the public benefits from wider availability of our research.” In addition, Kelty explained, publishers “are quite reconciled to this” after seeing 177 other universities take a similar path.

Renegade · « **Reply #38 on:** August 03, 2013, 10:40 PM »

Good news:
-IainB (August 03, 2013, 10:15 PM)

Very good news.

I briefly browsed and looked into a few articles.

One recently published one that I glanced at was simply terrifying. What the authors were advocating was horrific. And that was just the first paragraph.

I think it will be a very good thing to have these kinds of papers exposed to the light of day, rather than mulled over in dark corners by policy makers and implemented without anyone being able to even know what's going on.

IainB · « **Reply #39 on:** August 17, 2013, 12:19 PM »

^^ Yes, the horrifying ideas that some so-called "scientists" and "progressives" seem to come up with from time to time that look as though they might almost have been deliberately designed to drag us backwards in time, into a kind of barbarism.
There was an interesting link about this at Bishop Hill, referring to the hijacking of science, motivated by power/political interests in the Energy sector:

Delingpole on shale
Aug 17, 2013 Energy: gas Royal Society

James Delingpole has a perceptive piece on shale gas and the parallels with Ayn Rand's Atlas Shrugged:

One of the things she foresaw was the current nonsensical, dishonest, canting campaign against shale gas. In Atlas Shrugged it takes the form of Rearden Metal, the miracle technology which is going to transform the US economy if only the progressives will let it. But of course, Rand’s fictional progressives don’t want Reardon Metal to succeed any more than their modern, real-life equivalents want shale gas to succeed. Why not? For the same rag-bag of made-up, disingenuous reasons which progressives have used to justify their war on progress since time immemorial: it’s unfair, it uses up scarce resources, it might be dangerous. Rand doesn’t actually use the phrase “the precautionary principle.” But this is exactly what she is describing in the book when various vested interests – the corporatists in bed with big government, the politicised junk-scientists at the Institute of Science (aka, in our world, the National Academy of Sciences or the Royal Society), the unions – try to close down the nascent technology using the flimsiest of excuses.
_________________________
Although it has been pointed out that the Royal Society have been broadly supportive of shale developments, the parallels that James points out are rather striking.

Read the whole thing

IainB · « **Reply #40 on:** September 04, 2013, 03:08 AM »

Why do some scientists break with the scientific process?
Hat tip to: http://wattsupwithth...bers-on-bad-science/

Interesting report of an analysis from a survey carried out by clinicalpsychology.net (February 2012):

Bad Science – The Psychology Behind Bad Research
Scientists are some of our most trusted members of society. We depend on them for a great deal of what we know about the world. Unfortunately, recent looks into the world of scientific research and reporting has discovered that many scientists are not as trustworthy as we would like to believe. By engaging in various kinds of scientific misconduct, such as falsifying or fabricating data, scientists are getting the results they want without the honesty and integrity that we expect of the scientific institution. Some fields are worse than others as well, with clinical psychology being a notoriously troublesome area. How do we fix it? Read the infographic above to find out.

The infographic shows that, of biomedical research trainees at the University of California San Diego:

5% of those surveyed admitted to having modified the results of their research
81% of those surveyed said they would modify or fabricate results to win a grant or publish a paper.

Perhaps unsurprisingly (because it reveals something about the academic robustness/integrity/ethics of the biomedical research trainees at the University of California San Diego in particular, and of the profession of psychologists in general), the infographic seems to have been disappeared (404) at the original link:
Here it is anyway: (click to enlarge)

Peer Review and the Scientific Process

Below is the header to that, where it says:

BAD SCIENCE
THE PSYCHOLOGY BEHIND EXAGGERATED & FALSE RESEARCH
We often thnk that scientists are the most honest people around, and assume that scientific findings are reliable and true. But several new studies have revealed that an enormous number of researchers cut corners, cook data,
and lie about results when conducting experiments.
This is the world of bad science.

(click to enlarge)

Peer Review and the Scientific Process

Renegade · « **Reply #41 on:** September 04, 2013, 04:44 AM »

^ NO! <sniff> Shut up! Peer review! Double-blind! Logic! <sniff, sniff>

Placebo controlled! <waaah!> Gravity. deGrasse Tyson! Pictures of animals. <wahhh~~> Research grants! <sniff> Star pictures and galaxies.

Bill Nye. <waaahhh~~> Science Guy... <sniff> Shut up!

HAHAHAH~! I FRIGGIN' LOVE THAT GRAPHIC!

I especially like the note about medico-pharma-whores.

The WUWT article links to more fun as well, but, I'll skip some of the fun there for reasons you might well guess about here, but I can't resist this:

There are only two warnings in that speech for God’s sake, if you’re going to honor a historical document maybe somebody could at least read it, and maybe for once in almost fifty years remind us of Ike’s second warning: “…that public policy could itself become the captive of a scientific-technological elite.”

And there, the natural course of conversation would jump from Ike to Icke...

IainB · « **Reply #42 on:** October 04, 2013, 07:03 AM »

I am beginning to wonder whether we have forgotten what the point of peer review is.
After posting this on 2013-10-02 I was pretty annoyed:

I think this kind of pay-walling stinks. It shows a complete lack of ethics and professional scientific integrity and it goes hand-in-hand with the equally odious practice of deliberate restriction of access - by blocking FOI access, or locking-up and in some cases deletion - of/to data/information used in dubious scientific/academic research which has been partly/wholly funded by the public purse. This seems to be invariably attributable to a desperate need to avoid critical and open review leading to the very real risk that the research can be falsifiable - e.g., (QED) Climategate, S-E Anglia CRU research FOIA and the now apparently discredited hockey-stick chart from Mann (Penn.U.).

These all seem to be reflections of the same thing: a complete lack of ethics and professional scientific integrity.
________________________
-IainB (October 02, 2013, 06:46 PM)

- but I then became more annoyed after reading in washingtonpost.com that a medical doctor-scientist and CEO of a biotech company is facing some pretty stiff criminal charges after apparently falsely inflating the statistical health improvement outcomes (the research for which had been peer reviewed) for his drug, with the motivation apparently being money ($200 million, or something - i.e., lots of it):
The press-release conviction of a biotech CEO and its impact on scientific research

I then just now finally got around to watching the vidcast of Prof./PhD Don Easterbrook testifying before a hearing for a senate commission in Washington on 2013-03-26. He is a geologist. Start watching at 10 minutes and 30 seconds. Basically, using just raw, unadulterated data, Easterbrook explains to the senators all about "climate change" and why the theories, models and manipulated data (GISS, NASA, CRU) used by IPCC/CAGW alarmists are bunk. It's like watching a curious and highly rational child knock down a row of standing dominoes, each one onto the next.
Scam exposed. Time spent: approx. 1:20hrs, including Q&A.

Towards the end of it, even though he has kept mentioning that this or that point has been substantiated/verified by other scientists with whom he works, Easterbrook is asked if his work has been peer reviewed, and he says "Everything I have spoken about today, all this work, has been peer reviewed by other scientists, astronomers, physicists" (OWTTE).
He is also asked if he can explain how the IPCC with its peer-reviewed material can come to such different conclusions, and he politely says he can't explain it.

By the way, is there some way I could record that video? I don't know how, but I would love to have a copy. The guy has a giant intellect. Reminds me of WE Deming, whom I once had the opportunity to learn from for 4 days in succession, which experience changed me and my life.

Renegade · « **Reply #43 on:** October 04, 2013, 07:59 AM »

By the way, is there some way I could record that video? I don't know how, but I would love to have a copy. The guy has a giant intellect. Reminds me of WE Deming, whom I once had the opportunity to learn from for 4 days in succession, which experience changed me and my life.
-IainB (October 04, 2013, 07:03 AM)

Holy cripes.

These a$$sholes have split it into almost 600 chunks. (573)

See if this works in a mass downloader (1 segment per line):

Spoiler

Renegade · « **Reply #44 on:** October 04, 2013, 08:00 AM »

And here are the rest because the forum wouldn't handle that much data in 1 post... and that was JUST the URLs!

Spoiler

IainB · « **Reply #45 on:** October 04, 2013, 09:13 AM »

Thanks! I shall try that later. I use FlasgGot + Getright. I was download ing 230 or so items the other night and to my great surprise GetRight repeatedly crashed. I wonder how it will cope with 600 or so?

xtabber · « **Reply #46 on:** October 04, 2013, 05:13 PM »

I then just now finally got around to watching the vidcast of Prof./PhD Don Easterbrook testifying before a hearing for a senate commission in Washington on 2013-03-26. He is a geologist. Start watching at 10 minutes and 30 seconds. Basically, using just raw, unadulterated data, Easterbrook explains to the senators all about "climate change" and why the theories, models and manipulated data (GISS, NASA, CRU) used by IPCC/CAGW alarmists are bunk. It's like watching a curious and highly rational child knock down a row of standing dominoes, each one onto the next.
Scam exposed. Time spent: approx. 1:20hrs, including Q&A.

Towards the end of it, even though he has kept mentioning that this or that point has been substantiated/verified by other scientists with whom he works, Easterbrook is asked if his work has been peer reviewed, and he says "Everything I have spoken about today, all this work, has been peer reviewed by other scientists, astronomers, physicists" (OWTTE).
He is also asked if he can explain how the IPCC with its peer-reviewed material can come to such different conclusions, and he politely says he can't explain it.
-IainB (October 04, 2013, 07:03 AM)

I'm not a geologist or a climatologist, so I won't address specifics, but watching the Easterbrook testimony reminded me of many people I have come across who have built what they believe to be an irrefutable case for some crackpot theory or another. Unless you have a really deep understanding of the issues involved, it can be hard to challenge them because they know enough to sound as if they really do know what they are talking about.

As it happens, the geology department of Western Washington University, from which Dr. Easterbrook retired some time ago, issued a statement dissociating themselves from his testimony, which they describe as "filled with misrepresentations, misuse of data and repeated mixing of local vs. global records." I'd suggest reading that before accepting his ideas as valid.

Renegade · « **Reply #47 on:** October 05, 2013, 06:16 AM »

I then just now finally got around to watching the vidcast of Prof./PhD Don Easterbrook testifying before a hearing for a senate commission in Washington on 2013-03-26. He is a geologist. Start watching at 10 minutes and 30 seconds. Basically, using just raw, unadulterated data, Easterbrook explains to the senators all about "climate change" and why the theories, models and manipulated data (GISS, NASA, CRU) used by IPCC/CAGW alarmists are bunk. It's like watching a curious and highly rational child knock down a row of standing dominoes, each one onto the next.
Scam exposed. Time spent: approx. 1:20hrs, including Q&A.
-IainB (October 04, 2013, 07:03 AM)

Darn thing doesn't want to play for me. Either stalls or just won't play. I'll have to check back later.

IainB · « **Reply #48 on:** October 05, 2013, 06:33 AM »

I'm not a geologist or a climatologist, so I won't address specifics, but watching the Easterbrook testimony reminded me of many people I have come across who have built what they believe to be an irrefutable case for some crackpot theory or another. Unless you have a really deep understanding of the issues involved, it can be hard to challenge them because they know enough to sound as if they really do know what they are talking about.
As it happens, the geology department of Western Washington University, from which Dr. Easterbrook retired some time ago, issued a statement dissociating themselves from his testimony, which they describe as "filled with misrepresentations, misuse of data and repeated mixing of local vs. global records." I'd suggest reading that before accepting his ideas as valid.
______________________________
-xtabber (October 04, 2013, 05:13 PM)

@xtabber: Thankyou for that comment. Being something of an information/data junkie by training and inclination, I followed up the link and other, related references on the issue that I could find. I found your comments and those at the link rather illuminating.
Your comment was in response to mine (above) where I refer to three instances of apparently peer-reviewed research seemingly being abused by publication bias (for whatever reason) or being otherwise abused, buried or even avoided so as to (deliberately or otherwise) obfuscate or pervert the valid conclusions of science and/or avoid testing and the risk of falsifiability:

(a) in the fields of space-related science and climate science.
(b) in the field of scientific medical research.
(c) in the field of climate science.

Points that should probably be made here:

1. The subject of this thread: is categorically about Peer Review and the Scientific Process.
Thus, the issue is not "whether what Easterbrook says is true" but more like "whether this adds to our knowledge and understanding of the use of peer-reviews in the scientific process".
Whilst you profess ignorance of the facts or specific issues involved, your comments above would nevertheless seem to be arguing against or pointing to other people's "arguments" against the truth of what Easterbrook says in his presentation.
This would seem to be irrelevant to the subject - ignoratio elenchi (a "red herring" or genetic fallacy).
Furthermore you seem to have introduced:
- - argumentum ad hominem (argument against the person).
- - argumentum ad ignorantiam (forwarding a proposition without any certain proof).
- - argumentum ad populum (appeal to the people/consensus, popular sentiment - appeal to the majority; appeal to loyalty).
- - argumentum ad verecundiam (appeal to authority; conventional propriety).
- - non sequitur ("it does not follow"; or irrelevant conclusion: diverts attention away from a fact in discussion rather than addressing it directly.
  ______________________
Examples of outcomes of fallacy in "scientific" or "rational" thought:
Spoiler
"Man will never reach the moon regardless of all future scientific advances."
-- Dr. Lee DeForest, "Father of Radio & Grandfather of Television."

"The bomb will never go off. I speak as an expert in explosives."
-- Admiral William Leahy , US Atomic Bomb Project

"There is no likelihood man can ever tap the power of the atom."
-- Robert Millikan, Nobel Prize in Physics, 1923

"Computers in the future may weigh no more than 1.5 tons."
-- Popular Mechanics, forecasting the relentless march of science, 1949

"I think there is a world market for maybe five computers."
-- Thomas Watson, chairman of IBM, 1943

"I have traveled the length and breadth of this country and talked with the best people, and I can assure you that data processing is a fad that won't last out the year."
--The editor in charge of business books for Prentice Hall, 1957

"But what is it good for?"
-- Engineer at the Advanced Computing Systems Division of IBM, 1968, commenting on the microchip.

"640K ought to be enough for anybody."
-- Bill Gates, 1981

"This 'telephone'has too many shortcomings to be seriously considered as a means of communication. The device is inherently of no value to us,"
-- Western Union internal memo, 1876.

"The wireless music box has no imaginable commercial value. Who would pay for a message sent to nobody in particular?"
-- David Sarnoff's associates in response to his urgings for investment in the radio in the 1920s.

"The concept is interesting and well-formed, but in order to earn better than a 'C,' the idea must be feasible,"
-- A Yale University management professor in response to Fred Smith's paper proposing reliable overnight delivery service. (Smith went on to found Federal Express Corp.)

"I'm just glad it'll be Clark Gable who's falling on his face and not Gary Cooper,"
--Gary Cooper on his decision not to take the leading role in "Gone With The Wind."

"A cookie store is a bad idea. Besides, the market research reports say America likes crispy cookies, not soft and chewy cookies like you make,"
-- Response to Debbi Fields' idea of starting Mrs. Fields' Cookies.

"We don't like their sound, and guitar music is on the way out,"
-- Decca Recording Co. rejecting the Beatles, 1962.

"Heavier-than-air flying machines are impossible,"
-- Lord Kelvin, president, Royal Society, 1895.

"If I had thought about it, I wouldn't have done the experiment. The literature was full of examples that said you can't do this,"
-- Spencer Silver on the work that led to the unique adhesives for 3-M "Post-It" Notepads.

"Drill for oil? You mean drill into the ground to try and find oil? You're crazy,"
-- Drillers who Edwin L. Drake tried to enlist to his project to drill for oil in 1859.

"Stocks have reached what looks like a permanently high plateau."
-- Irving Fisher, Professor of Economics, Yale University , 1929.

"Airplanes are interesting toys but of no military value,"
-- Marechal Ferdinand Foch, Professor of Strategy, Ecole Superieure de Guerre , France .

"Everything that can be invented has been invented,"
-- Charles H. Duell, Commissioner, US Office of Patents, 1899.

"The super computer is technologically impossible. It would take all of the water that flows over Niagara Falls to cool the heat generated by the number of vacuum tubes required."
-- Professor of Electrical Engineering, New York University

"I don't know what use any one could find for a machine that would make copies of documents. It certainly couldn't be a feasible business by itself."
-- the head of IBM, refusing to back the idea, forcing the inventor to found Xerox.

"Louis Pasteur's theory of germs is ridiculous fiction."
-- Pierre Pachet, Professor of Physiology at Toulouse , 1872

"The abdomen, the chest, and the brain will forever be shut from the intrusion of the wise and humane surgeon,"
-- Sir John Eric Ericksen, British surgeon, appointed Surgeon-Extraordinary to Queen Victoria 1873.

"There is no reason anyone would want a computer in their home."
-- Ken Olson, president, chairman and founder of Digital Equipment Corp., 1977.

I would therefore recommend the approach that I try to take, which is that one takes responsibility for making the effort to do one's own thinking, using the available data, rather than deferring to or allowing the thinking of others to be a substitute for one's own thinking.

In other words, one follows the Royal Society's motto: "Nullius in verba/verbo." Literally, "Take nobody's word for it; see for yourself".
Spoiler
"Nullius in verba/verbo." Motto of the Royal Society, London. Literally, "Take nobody's word for it; see for yourself".
This motto indicates that currently, legitimate science seems to be based on the rejection of trust.
Thus, saying something purely on the basis of trust does not resemble genuine knowledge.
This is a new paradigm from the old, where scientific method can be seen to have developed from the 16th century perspective (Montaigne - no harm in the fact that "almost all the opinions we have are taken on authority and credit".) to the 17th century perspective (Gilbert, Bacon, Descartes and Boyle) where the approach is to take nothing on trust/authority.
So today we seek natural knowledge founded in evidence in nature - using individual reason - NOT in authority of tradition.
Thus real knowledge is NOT based on trust but on direct experience. - because reliance on the views of others produces errors.
The best scientist is thus incapable of functioning as a member of society.
The puzzle is that objective truth may exist, but human nature may preclude us from being able to experience it.

More on broken trust in peer review and how to fix it:
Spoiler
Shoring Up the Mantra of Science: Take Nobody's Word for It

Shoring Up the Mantra of Science: Take Nobody's Word for It
Ronald Bailey|Aug. 17, 2012 12:34 pm

Broken trust in peer review
The cited mantra is a general translation of "Nullius in verba," the motto of the British Royal Society, one of the world's first scientific organizations. Real science does not credit arguments from authority, but accepts the results from experiment and demonstration. The idea is that other researchers would check each others results to see if they could be reproduced. In the modern world, there's a lot less experimental replication and the result is lots of unreproduced experimental results are strewn throughout the scientific literature.

Earlier this year, two cancer researchers reported that that nine out of 10 preclinical peer-reviewed cancer research studies cannot be reproduced. As I explained in my column, "Can Most Cancer Research Be Trusted?":

   The academic system encourages the publication of a lot of junk research, and former vice president for oncology research at the pharmaceutical company Amgen Glenn Begley and M.D. Anderson Cancer Center researcher Lee Ellis agree. “To obtain funding, a job, promotion or tenure, researchers need a strong publication record, often including a first-authored high-impact publication,” they note. And journal editors and grant reviewers make it worse by pushing researchers to produce “a scientific finding that is simple, clear and complete—a ‘perfect’ story.” This pressure induces some researchers massage data to fit an underlying hypothesis or even suppress negative data that contradicts the favored hypothesis. In addition, peer review is broken. If an article is rejected by one journal, very often researchers will ignore the comments of reviewers, slap on another cover letter and submit to another journal. The publication process becomes a lottery; not a way to filter out misinformation.

The company Science Exchange has proposed its "Reproducibility Initiative" as an innovative way to fix this problem at the heart of experimental science. As Science Daily reports:

   Scientists who want to validate their findings will be able to apply to the initiative, which will choose a lab to redo the study and determine whether the results match.

   The project sprang from the growing realization that the scientific literature - from social psychology to basic cancer biology - is riddled with false findings and erroneous conclusions, raising questions about whether such studies can be trusted. Not only are erroneous studies a waste of money, often taxpayers', but they also can cause companies to misspend time and resources as they try to invent drugs based on false discoveries.

   "‘Published' and ‘true' are not synonyms," said Brian Nosek, a psychology professor at the University of Virginia in Charlottesville and a member of the initiative's advisory board....

   The initiative's 10-member board of prominent scientists will match investigators with a lab qualified to test their results, said Elizabeth Iorns, Science Exchange's co-founder and chief executive officer. The original lab would pay the second for its work. How much depends on the experiment's complexity and the cost of study materials, but should not exceed 20 percent of the original research study's costs. Iorns hopes government and private funding agencies will eventually fund replication to improve the integrity of scientific literature.

   The two labs would jointly write a paper, to be published in the journal PLoS One, describing the outcome. Science Exchange will issue a certificate if the original result is confirmed.

Here's hoping that lots of researchers will take advantage of this new initiative. For more background check out epidemiologist John Ioannides' 2005 classic article, "Why Most Published Research Findings are False" at PLoS Medicine.

I would also suggest that, in some contradiction to what you suggest, the evidence would seem to demonstrate that one categorically does not necessarily need to be a geologist or a climatologist - or, for that matter a member of any secular priesthood - to be able to contribute rational discussion on the subject of peer reviews or outcomes of climate science research, or other science for that matter. For example, many of the IPCC report authors/contributors over the years are or have been similarly and unashamedly not necessarily qualified and/or nor from what you might consider to be relevant scientific disciplines, but that apparently does not preclude their contribution - mistaken or otherwise (QED) - though some of them apparently do seem to consider themselves to be select members of a priesthood (QED per Climategate emails).
2. The need for critical thinking, reason/rationality when discussing the subject of peer-review in science:
Given the above, having even (say) 2 or so classical logical fallacies in a row would seem to be bad enough, but 5 or more would probably generally be regarded as going a tad too far and a pretty poor showing for the principles of rational thought.
Furthermore, as supposed substantiation of what you say or refer to, material from the link you provide would seem to employ the use of some of these fallacies - apparently by scientists/academics.
Just to recap on the importance of this: Critical thinking helps us to look for the presence of a fallacy in a rational argument, which would indicate an invalid point in its logical structure. If a single point in a logical structure is invalid, then the whole structure is deemed invalid (not true). If groups of people employ or use the same logical fallacy/fallacies, then that does not substantiate or reduce the fallacy nor validate what they say - e.g., including groups advocating witch-burning, Lysenkoism, McCarthyism, Climate Catastrophism, Heaven's Gate, and Phrenology - though it might appeal to our confirmational bias or inherent irrationality (beliefs).
3. The implications of this for peer review in the scientific process:
The link you pointed to - statement - was illuminating, not so much for what refutation or truth it might have contained but for the light it shed on the use of:
- - argumentum ad hominem (argument against the person).
- - argumentum ad ignorantiam (forwarding a proposition without any certain proof).
- - argumentum ad populum (appeal to the people/consensus, popular sentiment - appeal to the majority; appeal to loyalty).
- - argumentum ad verecundiam (appeal to authority; conventional propriety).
- - non sequitur ("it does not follow"; or irrelevant conclusion: diverts attention away from a fact in discussion rather than addressing it directly.
- - peer review consensus ( argumentum ad populum and argumentum ad verecundiam)
- as mechanisms for making something out to be absolutely and unquestionably true because unfortunately they were for some reason unable to provide proof to do that (though the proof might have been available nonetheless).

Some conclusions we could arrive at here would include:

A. Truth: You can't make something true out of a collection of logical fallacies. That would be an assault upon reason. Once you accept one invalid premise, you can accept infinitely more.
However, the depressing reality seems too often to be that many people are so unable to think rationally for themselves that they seem gullible to this kind of barrage of logical fallacy. One's head would be full of a confusing and probably conflicting mass of invalid premises, with ergo no real knowledge or understanding of truth.
B. Peer review per se is not crucial as it cannot and does not certainly establish truth: We have already seen, in this discussion thread and others - e.g., including, the thread on CAGW, Thermageddon? Postponed! - that there is plenty of evidence to demonstrate pretty conclusively that peer review is an unreliable instrument for determining truth, as it can be and has been, and probably will continue to be used/abused to rationalise whatever careless or unethical/misguided scientists might want, because they cannot otherwise scientifically prove a pet theory or preferred/biased conclusion.
This is also well-documented in the literature - e.g., including as referred to in one of the spoilers above("…on broken trust in peer review and how to fix it").
C. Falsifiability is crucial:
Falsifiability or refutability is the property of a statement, hypothesis, or theory whereby it could be shown to be false if some conceivable observation were true. In this sense, falsify is synonymous with nullify, meaning not "to commit fraud" but "show to be false". Science must be falsifiable. - Wikipedia.

As to a discussion of whether the IPCC theory/research and conclusions of CAGW are provable, or if the contrary facts/conclusions that Easterbrook was providing that it was falsifiable stood up to scrutiny - i.e., whether what the IPCC or Easterbrook are saying can be discredited/refuted on a factual and rational footing - I think that an evidence-based discussion of that would be very interesting, but is probably best be left to another, more relevant discussion thread, and so I shall continue with it in the discussion Thermageddon? Postponed!
(I hope that's OK.)

IainB · « **Reply #49 on:** October 14, 2013, 08:09 PM »

If that part of the scientific process called "peer review" is per se not crucial, as it cannot and does not certainly establish truth (QED), then of what use is it?
From personal experience of having worked in a consulting environment where all one's technical reports had to undergo collegial peer review and standards review prior to going out to the client, my view is that peer review can be an extremely helpful process for giving one's output a final, rigorous sanity check. One is provided by a brief review report from each reviewer, which highlights any errors or omissions of fact or approach, which enables you to correct the report. We all make mistakes, so this was a useful checkpoint.
It was also just good risk-avoidance. As consultants, we were being billed out at a rate of (typically) $1,000 to $1,500 per day. A client might often make significant expenditure based on the recommendations/advice in our reports at the end of an assignment. If we gave duff advice, then we could have been liable for the consequences, so everything had to be rational, factual, substantiated and the potential errors/risks had to be stated. We had insurance to cover us for professional negligence or mistake, but if we ever had to claim against it, then the subsequent premiums would have skyrocketed and could have driven us out of business.

In science, the (non-crucial) peer review part of the process would seem to have not only a potential (desirable) governing effect, but also could have an enabling effect on the (undesirable) propensity for inherent bias/error/fraud - and the latter might not become apparent unless you checked the science for falsifiability - which is crucial (QED).
This necessarily forces you back to the Royal Society's motto: "Nullius in verba/verbo." Literally, "Take nobody's word for it; see for yourself".
The signs of bad/bogus science are only likely to be revealed if you follow the Royal Society's motto - and remember, the rule is: science must be falsifiable.

Interestingly, this discussion thread mentions some evidence (as "bad science") to show that for a great number of years, parts of the scientific community may have been only too well aware of the potential of and propensity for inherent bias/error/fraud in peer review (above), and of how to use it to promote lobbying for preferred religio-political or other biased conclusions in their "science".

Checking the science for falsifiability would seem to be about the only real test that we have at our disposal, and where scientists seem determined to avoid the risk of research being exposed to that test (e.g., refusing to reveal methods, or refusing/obfuscating FOI requests), one can presume that something professionally unethical is probably going on.
_____________________________

"The rule of thumb is that, if a business process can not stand the hard light of scrutiny, then there is probably something unethical about it". - Sir Adrian Cadbury (Chairman of the then Quaker family-owned Cadbury's) in his prize-winning article on Business Ethics for Harvard Business Review circa 1984.