Main Area and Open Discussion > Living Room

Peer Review and the Scientific Process

<< < (16/47) > >>

IainB:
Here's a  :Thmbsup: to integrity and this group of 30 UK scientists who may have risked their careers by putting their names to this open letter.
(Copied below sans embedded hyperlinks/images.)
30 Scientists Call For More Challenges To ‘Prevailing Orthodoxies’
Date: 20/03/14     The Guardian

Government agencies use peer review to discourage open-ended inquiries and serious challenges to prevailing orthodoxies.

“Science is the belief in the ignorance of experts,” said Richard Feynman in the 1960s. But times change. Before about 1970, academics had access to modest funding they could use freely. Industry was similarly enlightened. Their results included the transistor, the maser-laser, the electronics and telecommunications revolutions, nuclear power, biotechnology and medical diagnostics galore that enriched the lives of virtually everyone; they also boosted 20th-century economic growth.

After 1970, politicians substantially expanded academic sectors. Peer review’s uses allowed the rise of priorities, impact etc, and is now virtually unavoidable. Applicants’ proposals must convince their peers that they serve national policies and are the best possible uses of resources. Success rates are about 25%, and strict rules govern resubmissions. Rejected proposals are usually lost. Industry too has lost its taste for the unpredictable. The 500 major discoveries, almost all initiated before about 1970, challenged mainstream science and would probably be vetoed today. Nowadays, fields where understanding is poor are usually neglected because researchers must convince experts that working in them will be beneficial.

However, small changes would keep science healthy. Some are outlined in Donald Braben’s book, Promoting the Planck Club: How Defiant Youth, Irreverent Researchers and Liberated Universities Can Foster Prosperity Indefinitely. But policies are deeply ingrained. Agencies claiming to support blue-skies research use peer review, of course, discouraging open-ended inquiries and serious challenges to prevailing orthodoxies. Mavericks once played an essential role in research. Indeed, their work defined the 20th century. We must relearn how to support them, and provide new options for an unforeseeable future, both social and economic. We need influential allies. Perhaps Guardian readers could help?

Donald W Braben University College London
John F Allen Queen Mary, University of London
William Amos University of Cambridge
Richard Ball University of Edinburgh
Tim Birkhead FRS University of Sheffield
Peter Cameron Queen Mary, University of London
Richard Cogdell FRS University of Glasgow
David Colquhoun FRS University College London
Rod Dowler Industry Forum, London
Irene Engle United States Naval Academy, Annapolis
Felipe Fernández-Armesto University of Notre Dame
Desmond Fitzgerald Materia Medica
Pat Heslop-Harrison University of Leicester
Dudley Herschbach Harvard University, Nobel Laureate
H Jeff Kimble Caltech, US National Academy of Sciences
Sir Harry Kroto FRS Florida State University, Tallahassee, Nobel Laureate
James Ladyman University of Bristol
Nick Lane University College London
Peter Lawrence FRS University of Cambridge
Angus MacIntyre FRS Queen Mary, University of London
John Mattick Garvan Institute of Medical Research, Sydney
Beatrice Pelloni University of Reading
Martyn Poliakoff FRS University of Nottingham
Douglas Randall University of Missouri
David Ray Bio Astral Limited
Sir Richard J Roberts FRS New England Biolabs, Nobel Laureate
Ken Seddon Queen’s University of Belfast
Colin Self University of Newcastle
Harry Swinney University of Texas, US National Academy of Sciences
Claudio Vita-Finzi FBA Natural History Museum

The Guardian, 18 March 2014

GWPF Newsletter   
©2013 The GWPF. All Rights Reserved. Information published on this website is for educational use only.   
_______________________

--- End quote ---

References:

* The original Guardian article: We need more scientific mavericks | @guardianletters | Science | The Guardian
* Supporting Gardian notes: Freedom for scientific mavericks | @guardianletters | Science | The Guardian

IainB:
A curious paper on psychology by one Prof Lewandowsky at the Univ. of Western Australia was recently retracted by its publishers, Frontiers, and for cogent reasons.
As reported here, the Lewandowsky retraction raises issues on Peer Review.
Climate Papers Without Peer
by Tony Thomas 6-4-14

Want your, er, highly innovative research to get lots of attention, the sort that keeps those grants coming? You could do worse than start with some kind words from a peer-reviewer whose work is glowingly cited in your own paper. After that, apply for the next batch of grants

Peer review is claimed to be the gold standard for scientific papers. Yet in the establishment climate science world, “peer review” operates differently. Professor Stephan Lewandowsky’s now-retracted paper Recursive Fury, about conspiracy-mindedness of “deniers”, raises a few issues about peer reviewing.
(Read the rest at the link.)

--- End quote ---

IainB:
Interesting twitter discussion: What is peer review exactly?

IainB:
I recall having read something about this "new discovery" earlier this year with considerable skepticism, and wondering whether it wasn't just another case of rushing something newsworthy/marketable (good for raising research funds) into print in Nature before properly peer reviewing it - especially as some of the media reports had pictures of this attractive girly-girl Japanese PhD student in a white lab coat, surrounded by Hello Kitty memorabilia or something, in her lab/office. She had apparently made the discovery and was grateful that her professor had "believed" in her despite the rest of the department's scientists apparently thinking she was a harebrained crackpot, or something.

So I was somewhat unsurprised to read in slashdot.org:
Japanese Stem Cell Debacle Could Bring Down Entire Center
(Copied below sans embedded hyperlinks/images.)
sciencehabit (1205606) writes
Shutting down the research center at the heart of an unfolding scientific scandal may be necessary to prevent a recurrence of research misconduct, according to a report released at a press conference in Tokyo today. A committee reviewing conduct at the RIKEN Center for Developmental Biology (CDB) in Kobe, Japan, found lax oversight and a failure on the part of senior authors of two papers in Nature outlining a surprisingly simple way of reprogramming mature cells into stem cells. The committee surmised that a drive to produce groundbreaking results led to publishing results prematurely. "It seems that RIKEN CDB had a strong desire to produce major breakthrough results that would surpass iPS cell research," the report concludes, referring to another type of pluripotent stem cell. "One of our conclusions is that the CDB organization is part of the problem," said committee chair Teruo Kishi Kishi. He recommends a complete overhaul of CDB, including perhaps restructuring it into a new institute. "This has to be more than just changing the nameplate."

--- End quote ---

IainB:
There has been some discussion in this thread regarding the publication of "bad"/fraudulent science, and how peer review seems incapable of improving the situation and in some cases may actually aggravate it.
In the UK, the MPs in the House of Commons are having a "links day" in which MPs will get together with scientists to discuss the issue of trust in science. Though somewhat belated, this is arguably a Very Good Thing for science.
Trust largely arose as an issue due to the revelations of "Climategate" in 2009 (and again in 2011) - the publication of swathes of emails hacked from the servers at the UK's UEA CRU (University of East Anglia Climate Research Unit) which highlighted what were described as somewhat unscientific/fraudulent goings-on, and the scientists apparently narrowly avoided being put through a Royal Commission of Enquiry on the matter. One outcome from this was a relatively serious collapse of the public's trust in the scientists involved - in the UK and abroad. (This is discussed in its many aspects in the DC Forum in the Basement section - Thermageddon? Postponed!.)

The Bishop Hill blog has a post describing the proposed Links, and in it makes some very pertinent points about the relevance/use of peer review in science and especially the implications where outputs from that science are subsequently fed, along with statistically insignificant data, into a policy development process.
(Copied below sans embedded hyperlinks/images.)
Parliamentary links day
The House of Commons is having a "links day" in which MPs will get together with scientists to discuss the issue of trust in science. Mark Walport and Paul Nurse will be speaking. I've been following the tweets on the #linksday2014 hashtag and they are a mixed bunch so far.

For example, we learn that Nicola Gulley, the editorial director of the Institute of Physics opined that:

    ...peer review key to maintaining trust in science. No crisis but a lack of understanding of this process.

You can see why someone working in the peer-reviewed journal sector might be keen on peer reviewed science, but for many readers at BH and many others uninvolved with the climate debate, peer review - its ineffectiveness, the superficial aura of "correctness" it gives, and the problem of gatekeeping - are the source of mistrust in science not a solution to it.

On the other hand Mark Walport has apparently been emphasising that science is only one input into the policy process, which is undoubtedly true and a rebuke to the scientivists and activists who constantly criticise politicians for "ignoring" scientists.

When you think about it, there's a link between these two themes. As we know, the peer reviewed evidence demonstrates conclusively that peer review is virtually useless at finding error and fraud (see discussion in The Hockey Stick Illusion). Clearly then, those advocating use of peer reviewed science in the policymaking process must be incorporating non-peer-reviewed elements into their thinking in order to overcome the peer-reviewed evidence that peer-reviewed evidence is no better than non-peer-reviewed evidence.

I'm not sure what conclusions we should draw from this though.
________________________________________
Update on Jun 24, 2014 by Registered CommenterBishop Hill
Paul Nurse is currently speaking at the event. Somewhat predictably he is still airing his GWPF conspiracy theories, as revealed in a tweet by James Wilsdon:

    Paul Nurse warns against taking science advice from “shadowy organisations who refuse to declare their funders.”

Readers will recall the Met Office's scientific advice to Parliament: first telling them that recent changes in global mean surface temperature were statistically significant and then, when pressed for their calculations, prevaricating, then confessing that the changes were not statistically significant and then without apparently batting an eyelid claiming that they did not rely on such statistical analyses.

No doubt these are the kinds of people that Sir Paul feels should be giving scientific advice to politicians.
________________________________________
Update on Jun 24, 2014 by Registered CommenterBishop Hill
More from Sir Paul via the twitter hashtag:

    Parliamentarians, journalists & scientists need to work better together; will increase public trust in science.

(If the press don't hold anyone to account, we can persuade the public of anything).

    "Trust in science key to maintaining democracy" SirPaulNurse @royalsociety rounding off discussions on science & public trust.

(Huh?)

    Uncertainty in scientific issues must be reflected in policy advice says Sir Paul Nurse.

(I'm looking forward to the Royal Society saying that the rise in surface temperatures is not statistically significant).

--- End quote ---

This Links thing could be encouraging in terms of showing a will to get to the truth which is at the heart of the matter, and it would seem like a good opportunity to take the bull by the horns. However, given the changeable political situation in the UK and the preparatory positioning statements of some of the players (as indicated in the post), I am unsure as to whether the MPs will actually have the bottle to do it. We shall see.

Navigation

[0] Message Index

[#] Next page

[*] Previous page