Welcome Guest.   Make a donation to an author on the site August 22, 2014, 04:41:11 PM  *

Please login or register.
Or did you miss your validation email?


Login with username and password (forgot your password?)
Why not become a lifetime supporting member of the site with a one-time donation of any amount? Your donation entitles you to a ton of additional benefits, including access to exclusive discounts and downloads, the ability to enter monthly free software drawings, and a single non-expiring license key for all of our programs.


You must sign up here before you can post and access some areas of the site. Registration is totally free and confidential.
 
Read the full one-year retrospective report on DonationCoder.com.
   
   Forum Home   Thread Marks Chat! Downloads Search Login Register  
Pages: Prev 1 2 3 [4]   Go Down
  Reply  |  New Topic  |  Print  
Author Topic: Peer Review and the Scientific Process  (Read 11457 times)
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #75 on: March 30, 2014, 10:40:33 PM »

Here's a  Thmbsup to integrity and this group of 30 UK scientists who may have risked their careers by putting their names to this open letter.
(Copied below sans embedded hyperlinks/images.)
Quote
30 Scientists Call For More Challenges To ‘Prevailing Orthodoxies’
Date: 20/03/14     The Guardian

Government agencies use peer review to discourage open-ended inquiries and serious challenges to prevailing orthodoxies.

“Science is the belief in the ignorance of experts,” said Richard Feynman in the 1960s. But times change. Before about 1970, academics had access to modest funding they could use freely. Industry was similarly enlightened. Their results included the transistor, the maser-laser, the electronics and telecommunications revolutions, nuclear power, biotechnology and medical diagnostics galore that enriched the lives of virtually everyone; they also boosted 20th-century economic growth.

After 1970, politicians substantially expanded academic sectors. Peer review’s uses allowed the rise of priorities, impact etc, and is now virtually unavoidable. Applicants’ proposals must convince their peers that they serve national policies and are the best possible uses of resources. Success rates are about 25%, and strict rules govern resubmissions. Rejected proposals are usually lost. Industry too has lost its taste for the unpredictable. The 500 major discoveries, almost all initiated before about 1970, challenged mainstream science and would probably be vetoed today. Nowadays, fields where understanding is poor are usually neglected because researchers must convince experts that working in them will be beneficial.

However, small changes would keep science healthy. Some are outlined in Donald Braben’s book, Promoting the Planck Club: How Defiant Youth, Irreverent Researchers and Liberated Universities Can Foster Prosperity Indefinitely. But policies are deeply ingrained. Agencies claiming to support blue-skies research use peer review, of course, discouraging open-ended inquiries and serious challenges to prevailing orthodoxies. Mavericks once played an essential role in research. Indeed, their work defined the 20th century. We must relearn how to support them, and provide new options for an unforeseeable future, both social and economic. We need influential allies. Perhaps Guardian readers could help?

Donald W Braben University College London
John F Allen Queen Mary, University of London
William Amos University of Cambridge
Richard Ball University of Edinburgh
Tim Birkhead FRS University of Sheffield
Peter Cameron Queen Mary, University of London
Richard Cogdell FRS University of Glasgow
David Colquhoun FRS University College London
Rod Dowler Industry Forum, London
Irene Engle United States Naval Academy, Annapolis
Felipe Fernández-Armesto University of Notre Dame
Desmond Fitzgerald Materia Medica
Pat Heslop-Harrison University of Leicester
Dudley Herschbach Harvard University, Nobel Laureate
H Jeff Kimble Caltech, US National Academy of Sciences
Sir Harry Kroto FRS Florida State University, Tallahassee, Nobel Laureate
James Ladyman University of Bristol
Nick Lane University College London
Peter Lawrence FRS University of Cambridge
Angus MacIntyre FRS Queen Mary, University of London
John Mattick Garvan Institute of Medical Research, Sydney
Beatrice Pelloni University of Reading
Martyn Poliakoff FRS University of Nottingham
Douglas Randall University of Missouri
David Ray Bio Astral Limited
Sir Richard J Roberts FRS New England Biolabs, Nobel Laureate
Ken Seddon Queen’s University of Belfast
Colin Self University of Newcastle
Harry Swinney University of Texas, US National Academy of Sciences
Claudio Vita-Finzi FBA Natural History Museum

The Guardian, 18 March 2014

GWPF Newsletter   
©2013 The GWPF. All Rights Reserved. Information published on this website is for educational use only.   
_______________________

References:
Logged
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #76 on: April 26, 2014, 11:09:06 AM »

A curious paper on psychology by one Prof Lewandowsky at the Univ. of Western Australia was recently retracted by its publishers, Frontiers, and for cogent reasons.
As reported here, the Lewandowsky retraction raises issues on Peer Review.
Quote
Climate Papers Without Peer
by Tony Thomas 6-4-14

Want your, er, highly innovative research to get lots of attention, the sort that keeps those grants coming? You could do worse than start with some kind words from a peer-reviewer whose work is glowingly cited in your own paper. After that, apply for the next batch of grants

Peer review is claimed to be the gold standard for scientific papers. Yet in the establishment climate science world, “peer review” operates differently. Professor Stephan Lewandowsky’s now-retracted paper Recursive Fury, about conspiracy-mindedness of “deniers”, raises a few issues about peer reviewing.
(Read the rest at the link.)
Logged
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #77 on: May 26, 2014, 09:54:26 AM »

Interesting twitter discussion: What is peer review exactly?
Logged
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #78 on: June 14, 2014, 06:41:44 AM »

I recall having read something about this "new discovery" earlier this year with considerable skepticism, and wondering whether it wasn't just another case of rushing something newsworthy/marketable (good for raising research funds) into print in Nature before properly peer reviewing it - especially as some of the media reports had pictures of this attractive girly-girl Japanese PhD student in a white lab coat, surrounded by Hello Kitty memorabilia or something, in her lab/office. She had apparently made the discovery and was grateful that her professor had "believed" in her despite the rest of the department's scientists apparently thinking she was a harebrained crackpot, or something.

So I was somewhat unsurprised to read in slashdot.org:
Japanese Stem Cell Debacle Could Bring Down Entire Center
(Copied below sans embedded hyperlinks/images.)
Quote
sciencehabit (1205606) writes
Shutting down the research center at the heart of an unfolding scientific scandal may be necessary to prevent a recurrence of research misconduct, according to a report released at a press conference in Tokyo today. A committee reviewing conduct at the RIKEN Center for Developmental Biology (CDB) in Kobe, Japan, found lax oversight and a failure on the part of senior authors of two papers in Nature outlining a surprisingly simple way of reprogramming mature cells into stem cells. The committee surmised that a drive to produce groundbreaking results led to publishing results prematurely. "It seems that RIKEN CDB had a strong desire to produce major breakthrough results that would surpass iPS cell research," the report concludes, referring to another type of pluripotent stem cell. "One of our conclusions is that the CDB organization is part of the problem," said committee chair Teruo Kishi Kishi. He recommends a complete overhaul of CDB, including perhaps restructuring it into a new institute. "This has to be more than just changing the nameplate."
Logged
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #79 on: June 24, 2014, 09:24:40 PM »

There has been some discussion in this thread regarding the publication of "bad"/fraudulent science, and how peer review seems incapable of improving the situation and in some cases may actually aggravate it.
In the UK, the MPs in the House of Commons are having a "links day" in which MPs will get together with scientists to discuss the issue of trust in science. Though somewhat belated, this is arguably a Very Good Thing for science.
Trust largely arose as an issue due to the revelations of "Climategate" in 2009 (and again in 2011) - the publication of swathes of emails hacked from the servers at the UK's UEA CRU (University of East Anglia Climate Research Unit) which highlighted what were described as somewhat unscientific/fraudulent goings-on, and the scientists apparently narrowly avoided being put through a Royal Commission of Enquiry on the matter. One outcome from this was a relatively serious collapse of the public's trust in the scientists involved - in the UK and abroad. (This is discussed in its many aspects in the DC Forum in the Basement section - Thermageddon? Postponed!.)

The Bishop Hill blog has a post describing the proposed Links, and in it makes some very pertinent points about the relevance/use of peer review in science and especially the implications where outputs from that science are subsequently fed, along with statistically insignificant data, into a policy development process.
(Copied below sans embedded hyperlinks/images.)
Quote
Parliamentary links day
The House of Commons is having a "links day" in which MPs will get together with scientists to discuss the issue of trust in science. Mark Walport and Paul Nurse will be speaking. I've been following the tweets on the #linksday2014 hashtag and they are a mixed bunch so far.

For example, we learn that Nicola Gulley, the editorial director of the Institute of Physics opined that:

    ...peer review key to maintaining trust in science. No crisis but a lack of understanding of this process.


You can see why someone working in the peer-reviewed journal sector might be keen on peer reviewed science, but for many readers at BH and many others uninvolved with the climate debate, peer review - its ineffectiveness, the superficial aura of "correctness" it gives, and the problem of gatekeeping - are the source of mistrust in science not a solution to it.

On the other hand Mark Walport has apparently been emphasising that science is only one input into the policy process, which is undoubtedly true and a rebuke to the scientivists and activists who constantly criticise politicians for "ignoring" scientists.

When you think about it, there's a link between these two themes. As we know, the peer reviewed evidence demonstrates conclusively that peer review is virtually useless at finding error and fraud (see discussion in The Hockey Stick Illusion). Clearly then, those advocating use of peer reviewed science in the policymaking process must be incorporating non-peer-reviewed elements into their thinking in order to overcome the peer-reviewed evidence that peer-reviewed evidence is no better than non-peer-reviewed evidence.

I'm not sure what conclusions we should draw from this though.
________________________________________
Update on Jun 24, 2014 by Registered CommenterBishop Hill
Paul Nurse is currently speaking at the event. Somewhat predictably he is still airing his GWPF conspiracy theories, as revealed in a tweet by James Wilsdon:

    Paul Nurse warns against taking science advice from “shadowy organisations who refuse to declare their funders.”

Readers will recall the Met Office's scientific advice to Parliament: first telling them that recent changes in global mean surface temperature were statistically significant and then, when pressed for their calculations, prevaricating, then confessing that the changes were not statistically significant and then without apparently batting an eyelid claiming that they did not rely on such statistical analyses.

No doubt these are the kinds of people that Sir Paul feels should be giving scientific advice to politicians.
________________________________________
Update on Jun 24, 2014 by Registered CommenterBishop Hill
More from Sir Paul via the twitter hashtag:

    Parliamentarians, journalists & scientists need to work better together; will increase public trust in science.

(If the press don't hold anyone to account, we can persuade the public of anything).

    "Trust in science key to maintaining democracy" SirPaulNurse @royalsociety rounding off discussions on science & public trust.

(Huh?)

    Uncertainty in scientific issues must be reflected in policy advice says Sir Paul Nurse.

(I'm looking forward to the Royal Society saying that the rise in surface temperatures is not statistically significant).

This Links thing could be encouraging in terms of showing a will to get to the truth which is at the heart of the matter, and it would seem like a good opportunity to take the bull by the horns. However, given the changeable political situation in the UK and the preparatory positioning statements of some of the players (as indicated in the post), I am unsure as to whether the MPs will actually have the bottle to do it. We shall see.
Logged
Renegade
Charter Member
***
Posts: 11,110



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #80 on: June 29, 2014, 09:21:12 PM »

Again about trust...

http://www.psmag.com/navi...ustry-gums-science-84564/

Quote
How the Sugar Industry Gums Up Science

It goes over a bit how "science" is a bitter joke in the hands of the sugar industry shills.

The article links here:

http://blog.ucsusa.org/ad...ing-debate-at-the-fda-564

Where, thankfully, we can see that there are people that actually care about shilling in science.

Still, I think the most important methodology in modern science is "follow the money", which is an incredibly sad thing.
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #81 on: July 12, 2014, 02:01:00 AM »

More of this makes a mockery of "science".
(Copied below sans embedded hyperlinks/images.)
Quote
Scholarly journal retracts 60 articles, smashes ‘peer review ring’ - The Washington Post
By Fred Barbash July 10
Updated

Every now and then a scholarly journal retracts an article because of errors or outright fraud. In academic circles, and sometimes beyond, each retraction is a big deal.

Now comes word of a journal retracting 60 articles at once.

The reason for the mass retraction is mind-blowing: A “peer review and citation ring” was apparently rigging the review process to get articles published.

You’ve heard of prostitution rings, gambling rings and extortion rings. Now there’s a “peer review ring.”

The publication is the Journal of Vibration and Control (JVC). It publishes papers with names like “Hydraulic engine mounts: a survey” and “Reduction of wheel force variations with magnetorheological devices.”

The field of acoustics covered by the journal is highly technical:

    Analytical, computational and experimental studies of vibration phenomena and their control. The scope encompasses all linear and nonlinear vibration phenomena and covers topics such as: vibration and control of structures and machinery, signal analysis, aeroelasticity, neural networks, structural control and acoustics, noise and noise control, waves in solids and fluids and shock waves.

JVC is part of the SAGE group of academic publications.

Here’s how it describes its peer review process:

[The journal] operates under a conventional single-blind reviewing policy in which the reviewer’s name is always concealed from the submitting author.
All manuscripts are reviewed initially by one of the Editors and only those papers that meet the scientific and editorial standards of the journal, and fit within the aims and scope of the journal, will be sent for peer review.  Generally, reviews from two independent referees are required.

An announcement from SAGE published July 8 explained what happened, albeit somewhat opaquely.

In 2013, the editor of JVC, Ali H. Nayfeh, became aware of people using “fabricated identities” to manipulate an online system called SAGE Track by which scholars review the work of other scholars prior to publication.

Attention focused on a researcher named Peter Chen of the National Pingtung University of Education (NPUE) in Taiwan and “possibly other authors at this institution.”

After a 14-month investigation, JVC determined the ring involved “aliases” and fake e-mail addresses of reviewers — up to 130 of them — in an apparently successful effort to get friendly reviews of submissions and as many articles published as possible by Chen and his friends. “On at least one occasion, the author Peter Chen reviewed his own paper under one of the aliases he created,” according to the SAGE announcement.

The statement does not explain how something like this happens. Did the ring invent names and say they were scholars? Did they use real names and pretend to be other scholars? Doesn’t anyone check on these things by, say, picking up the phone and calling the reviewer?

In any case, SAGE and Nayfeh confronted Chen to give him an “opportunity to address the accusations of misconduct,” the statement said, but were not satisfied with his responses.

In May, “NPUE informed SAGE and JVC that Peter Chen had resigned from his post on 2 February 2014.”

Each of the 60 retracted articles had at least one author and/or one reviewer “who has been implicated in the peer review” ring, said a separate notice issued by JVC.

Efforts by The Washington Post to locate and contact Chen for comment were unsuccessful.

The whole story is described in a publication called “Retraction Watch” under the headline: “SAGE Publications busts ‘peer review and citation ring.’”

“This one,” it said, “deserves a ‘wow.’”

Update: Some additional information from the SAGE statement: “As the SAGE investigation drew to a close, in May 2014 Professor Nayfeh’s retirement was announced and he resigned his position as Editor-in-Chief of JVC….Three senior editors and an additional 27 associate editors with expertise and prestige in the field have been appointed to assist with the day-to-day running of the JVC peer review process. Following Professor Nayfeh’s retirement announcement, the external senior editorial team will be responsible for independent editorial control for JVC.”

Note to readers: Thanks for pointing out my grammatical error. No excuses.

There’s a follow to this story here.
Logged
Renegade
Charter Member
***
Posts: 11,110



Tell me something you don't know...

see users location on a map View Profile WWW Give some DonationCredits to this forum member
« Reply #82 on: July 12, 2014, 03:02:41 AM »

Yay. The supposed results of the skeptical process need to be skeptically questioned because... just... ::facepalm::
Logged

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker
IainB
Supporting Member
**
Posts: 4,620


Slartibartfarst

see users location on a map View Profile Give some DonationCredits to this forum member
« Reply #83 on: August 18, 2014, 03:44:15 AM »

Interesting item in Hacker News from peerj.com. It's a link to a .PDF file.
Quote
A surge of p-values between 0.040 and 0.049 in recent decades (but negative results are increasing rapidly too).

It is known that statistically significant results are more likely to be published than results that are not statistically significant.  However, it is unclear whether negative results are disappearing from papers, and whether there exists a ‘hierarchy of sciences’ with the social sciences publishing more positive results than the physical sciences.  Using Scopus, we conducted a search in the abstracts of papers published between 1990 and 2014, and calculated the percentage of papers reporting marginally positive results (i.e., p-values between 0.040 and 0.049) versus the percentage of papers reporting marginally negative results (i.e., p-values between 0.051 and 0.060).  The results indicate that negative results are not disappearing, but have actually become 4.3 times more prevalent since 1990.  Positive results, on the other hand, have become 13.9 times more prevalent since 1990.  We found no consistent support for a ‘hierarchy of sciences’.  However, we did find large differences in reporting practices between disciplines, with the reporting of p-values being 60.6 times more frequent in the biological sciences than in the physical sciences.  We argue that the observed longitudinal trends may be caused by negative factors, such as an increase of questionable research practices, but also by positive factors, such as an increasingly quantitative research focus. ...
Logged
Pages: Prev 1 2 3 [4]   Go Up
  Reply  |  New Topic  |  Print  
 
Jump to:  
   Forum Home   Thread Marks Chat! Downloads Search Login Register  

DonationCoder.com | About Us
DonationCoder.com Forum | Powered by SMF
[ Page time: 0.053s | Server load: 0.1 ]