Main Area and Open Discussion > Living Room

Peer Review and the Scientific Process

<< < (42/47) > >>

Renegade:
http://theweek.com/articles/618141/big-science-broken

Big Science is broken


Science is broken.

That's the thesis of a must-read article in First Things magazine, in which William A. Wilson accumulates evidence that a lot of published research is false. But that's not even the worst part.

Advocates of the existing scientific research paradigm usually smugly declare that while some published conclusions are surely false, the scientific method has "self-correcting mechanisms" that ensure that, eventually, the truth will prevail. Unfortunately for all of us, Wilson makes a convincing argument that those self-correcting mechanisms are broken.

For starters, there's a "replication crisis" in science. This is particularly true in the field of experimental psychology, where far too many prestigious psychology studies simply can't be reliably replicated. But it's not just psychology. In 2011, the pharmaceutical company Bayer looked at 67 blockbuster drug discovery research findings published in prestigious journals, and found that three-fourths of them weren't right. Another study of cancer research found that only 11 percent of preclinical cancer research could be reproduced. Even in physics, supposedly the hardest and most reliable of all sciences, Wilson points out that "two of the most vaunted physics results of the past few years — the announced discovery of both cosmic inflation and gravitational waves at the BICEP2 experiment in Antarctica, and the supposed discovery of superluminal neutrinos at the Swiss-Italian border — have now been retracted, with far less fanfare than when they were first published."

What explains this? In some cases, human error. Much of the research world exploded in rage and mockery when it was found out that a highly popularized finding by the economists Ken Rogoff and Carmen Reinhardt linking higher public debt to lower growth was due to an Excel error. Steven Levitt, of Freakonomics fame, largely built his career on a paper arguing that abortion led to lower crime rates 20 years later because the aborted babies were disproportionately future criminals. Two economists went through the painstaking work of recoding Levitt's statistical analysis — and found a basic arithmetic error.

Then there is outright fraud. In a 2011 survey of 2,000 research psychologists, over half admitted to selectively reporting those experiments that gave the result they were after. The survey also concluded that around 10 percent of research psychologists have engaged in outright falsification of data, and more than half have engaged in "less brazen but still fraudulent behavior such as reporting that a result was statistically significant when it was not, or deciding between two different data analysis techniques after looking at the results of each and choosing the more favorable."

Then there's everything in between human error and outright fraud: rounding out numbers the way that looks better, checking a result less thoroughly when it comes out the way you like, and so forth.

--- End quote ---

More at the link.

Link to the article at First Things:

http://www.firstthings.com/article/2016/05/scientific-regress

SCIENTIFIC REGRESS

he problem with ­science is that so much of it simply isn’t. Last summer, the Open Science Collaboration announced that it had tried to replicate one hundred published psychology experiments sampled from three of the most prestigious journals in the field. Scientific claims rest on the idea that experiments repeated under nearly identical conditions ought to yield approximately the same results, but until very recently, very few had bothered to check in a systematic way whether this was actually the case. The OSC was the biggest attempt yet to check a field’s results, and the most shocking. In many cases, they had used original experimental materials, and sometimes even performed the experiments under the guidance of the original researchers. Of the studies that had originally reported positive results, an astonishing 65 percent failed to show statistical significance on replication, and many of the remainder showed greatly reduced effect sizes.

Their findings made the news, and quickly became a club with which to bash the social sciences. But the problem isn’t just with psychology. There’s an ­unspoken rule in the pharmaceutical industry that half of all academic biomedical research will ultimately prove false, and in 2011 a group of researchers at Bayer decided to test it. Looking at sixty-seven recent drug discovery projects based on preclinical cancer biology research, they found that in more than 75 percent of cases the published data did not match up with their in-house attempts to replicate. These were not studies published in fly-by-night oncology journals, but blockbuster research featured in Science, Nature, Cell, and the like. The Bayer researchers were drowning in bad studies, and it was to this, in part, that they attributed the mysteriously declining yields of drug pipelines. Perhaps so many of these new drugs fail to have an effect because the basic research on which their development was based isn’t valid.

When a study fails to replicate, there are two possible interpretations. The first is that, unbeknownst to the investigators, there was a real difference in experimental setup between the original investigation and the failed replication. These are colloquially referred to as “wallpaper effects,” the joke being that the experiment was affected by the color of the wallpaper in the room. This is the happiest possible explanation for failure to reproduce: It means that both experiments have revealed facts about the universe, and we now have the opportunity to learn what the difference was between them and to incorporate a new and subtler distinction into our theories.

The other interpretation is that the original finding was false. Unfortunately, an ingenious statistical argument shows that this second interpretation is far more likely. First articulated by John Ioannidis, a professor at Stanford University’s School of Medicine, this argument proceeds by a simple application of Bayesian statistics. Suppose that there are a hundred and one stones in a certain field. One of them has a diamond inside it, and, luckily, you have a diamond-detecting device that advertises 99 percent accuracy. After an hour or so of moving the device around, examining each stone in turn, suddenly alarms flash and sirens wail while the device is pointed at a promising-looking stone. What is the probability that the stone contains a diamond?

Most would say that if the device advertises 99 percent accuracy, then there is a 99 percent chance that the device is correctly discerning a diamond, and a 1 percent chance that it has given a false positive reading. But consider: Of the one hundred and one stones in the field, only one is truly a diamond. Granted, our machine has a very high probability of correctly declaring it to be a diamond. But there are many more diamond-free stones, and while the machine only has a 1 percent chance of falsely declaring each of them to be a diamond, there are a hundred of them. So if we were to wave the detector over every stone in the field, it would, on average, sound twice—once for the real diamond, and once when a false reading was triggered by a stone. If we know only that the alarm has sounded, these two possibilities are roughly equally probable, giving us an approximately 50 percent chance that the stone really contains a diamond.

--- End quote ---

More at that link as well.

IainB:
@Renegade: ^^ Those are very interesting links, thanks.
True, but somewhat depressing though.

Mikky111:
IainB  ;D ;D

Renegade:
Scientists Say Fraud Causing Crisis of Science - #NewWorldNextWeek



Story #1: 40% of Scientists Admit Fraud “Always or Often” Contributes to Irreproducible Research
http://bit.ly/1XkpU1b
How the National Academy of Sciences Misled the Public Over GMO Food Safety
http://bit.ly/1TY7cZ9
Portland, Oregon School Board Promotes Climate Justice, Bans Books That Deny Climate Change
http://bit.ly/1UwJBPf

--- End quote ---

IainB:
@Renegade: Yes, I had read about all three of those ^^.
"Ministry of Truth" here we come.
We have to realise that these things are accompanied by - could not happen without - an abandonment of reason. It's not technocracy per se, its political fascism based on whatever "scientific" model they find most convenient or expedient to promulgate their objectives (e.g., the school board banning books that risk factually contradicting/refuting the politically correct and unsubstantial propaganda or religio-political ideology).

It's not much different to people wanting to ban the teaching of evolutionary theory in schools because it conflicts with their preferred religio-political ideology. (And please don't try to tell me that the Old Testament is a theoretical treatise.)

Judging by some of the nonsense that some people spout, I often wonder whether we as a species are hard-wired, as it were, so that we are unable to resist believing in imaginary things - be they fairies, djins, God, the non-existence of God, life after death, non-life after death, Heaven and Hell, or whatever - to the extent that we feel obliged to convert others to our beliefs and are intellectually intolerant of anyone holding any conflicting views (theory or belief).

The documented history of wars from ancient to modern times shows an essential and unpleasant truth that, whenever one sees a situation where a power group holds an absolute intolerance of other groups'/people's credible reasoning, views or beliefs, it was generally followed by some form of sacrifice and forceful reinforcement of "the correct" views/behaviours, punishable by death to the unreformable unbelievers.

There is a near-perfect model of this kind of dichotomy in Nature. If you've ever seen a video of a cuckoo chick in its host's nest, pushing out everything - including the eggs of the host birds and/or their hatched young - then you have witnessed a highly successful survival behaviour. It is instinctive. The chick cannot abide having anything else in the nest, and even has a specially-hollowed out dent in its back to assist in the expulsion of other objects from the nest. The tiny host-parents wear themselves out trying to feed the voracious giant parasite that they believe is their offspring.

Now consider the paradigm in one's mind as being somewhat akin to that cuckoo chick. Within the radius of control of the mind, there must be no conflicting paradigms.
Except that, with humans, it doesn't stop there - the radius of control expands and extends far outside of one's head and other people must think similarly, or else there'll be trouble. Typically that's likely to mean that someone's probably going to have to be jailed, beaten up or worse, even killed.

At root, it's simply another manifestation of Ahamkara

If anything has been demonstrated in this discussion thread, it is that the truth of science has continually and consistently been perverted by fraud and/or religio-political ideology and unreason, to the extent that no-one can read about scientific research any more without a huge dollop of skepticism.
A corrupt peer review of a corrupt science cannot somehow "make it true", no matter how hard and shrill the desperate cry that "There is consensus. See? We are all agreed! And it's all been peer reviewed, so it must be true!"

Yeah, right.

Navigation

[0] Message Index

[#] Next page

[*] Previous page