topbanner_forum
  *

avatar image

Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
  • Tuesday April 16, 2024, 11:54 am
  • Proudly celebrating 15+ years online.
  • Donate now to become a lifetime supporting member of the site and get a non-expiring license key for all of our programs.
  • donate

Last post Author Topic: Peer Review and the Scientific Process  (Read 145963 times)

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #100 on: December 01, 2014, 07:51 AM »
Did a quick skim of the bios of the staff and then read a half-dozen articles on American Thinker.

From the site:

About Us

American Thinker is a daily internet publication devoted to the thoughtful exploration of issues of importance to Americans. Contributors are accomplished in fields beyond journalism and animated to write for the general public out of concern for the complex and morally significant questions on the national agenda.

There is no limit to the topics appearing on American Thinker. National security in all its dimensions -- strategic, economic, diplomatic, and military -- is emphasized. The right to exist and the survival of the State of Israel are of great importance to us. Business, science, technology, medicine, management, and economics in their practical and ethical dimensions are also emphasized, as is the state of American culture.

I have a certain Uncle. He's a highly educated and well-intentioned individual. But somewhere along the line, he got fed up with many of things he was seeing in the news and became very angry. Angry to the point of where it affected his better nature and intellectual judgement and created a politico-moral lens (with a narrow angle and shallow depth of field) which he now views the world through. Beneath his visible calm, and love for quoting sources, the rage that's boiling within can clearly be seen. Another good man driven half-mad by the complexities and ambiguities of the world he lives in.

The writers at American Thinker remind me very much of him.

Maybe it's me. but do I detect a certain middle-American and strongly 'conservative' (dare I say closet far right?) agenda at work in American Thinker?  ;)

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #101 on: December 01, 2014, 10:12 AM »
^^ ...erm, thanks, that's an interesting critique of American Thinker (though I personally couldn't care less about its political leanings), but did you happen to notice the bit about the Secret Science Reform Act of 2014?
I mean, I thought it was an absolute hoot. Or are you suggesting that I have been taken in by a spoof? If so, I apologise for being gullible, but when it popped up in my feed reader it did seem to be like a real report about a real thing.    :(

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #102 on: December 02, 2014, 12:48 PM »
Maybe it's me. but do I detect a certain middle-American and strongly 'conservative' (dare I say closet far right?) agenda at work in American Thinker?  ;)

Probably. But in order to be a far-right extremist, you only have to say something like "saving money is good" or "being in debt is bad." 

But the article that IainB posted is completely relevant to the thread.

It shouldn't be any kind of surprise that "science" is regularly tarnished with incompetence and fraud. While incompetence is one thing, fraud isn't so easily forgiveable.

Given the current climate, I would go so far as to say that any research that isn't completely and 100% open should be rejected out of hand and completely ignored.

If you don't have the information necessary to replicate it, then it's not worth anything. Transparency is everything.

That includes raw data.

A CDC whistleblower recently came out and exposed deliberate fraud in previous studies. They had excluded data to make a product "safe" when they knew it wasn't. Raw data matters. Transparency is everything.

There is a crisis in peer review, and the crisis isn't a "science" issue, it's a "human" issue, and largely of greed. That's neither right-wing nor left-wing.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #103 on: December 02, 2014, 01:48 PM »
But in order to be a far-right extremist, you only have to say something like "saving money is good" or "being in debt is bad."

Not true. But you have your agenda I suppose. :P

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #104 on: December 02, 2014, 03:49 PM »
Talk about "red whales"...

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #105 on: December 02, 2014, 08:00 PM »
But in order to be a far-right extremist, you only have to say something like "saving money is good" or "being in debt is bad."

Not true. But you have your agenda I suppose. :P

Try discussing monetary policy, currency, etc. You'll very quickly find that only "far-right extremists" advocate things like that.

Here's a quick example from a professor of economics at Oxford:

http://mainlymacro.b...esian-economics.html

So my argument is that Keynesian theory is not left wing...

Etc. etc.

Keynesian economics is framed as centrist. He puts fiscal conservatism on the far right. And he's far from alone.

Now, keep in mind chapter 2 of "The Communist Manifesto" and this:

https://www.marxists...t-manifesto/ch02.htm

Nevertheless, in most advanced countries, the following will be pretty generally applicable.
...
2. A heavy progressive or graduated income tax.
...
5. Centralisation of credit in the hands of the state, by means of a national bank with State capital and an exclusive monopoly.

Which are also central tenants of Keynesian economics. i.e. Progressive taxation and central banks.

I could blather on about this for quite some time, but I think I've sufficiently made my point: All you need to do to be a right-wing extremist is to advocate fiscal responsibility outside of a Keynesian/communist model.

NOTE: I've picked economics for the example deliberately as there is a stronger argument to be made that economics is a science compared to some other areas where someone might also be called "right-wing", though with much greater justification.

For a quick diversion there:

http://www.theguardi...ience-robert-shiller

The advance of behavioural economics is not fundamentally in conflict with mathematical economics, as some seem to think, though it may well be in conflict with some currently fashionable mathematical economic models. And, while economics presents its own methodological problems, the basic challenges facing researchers are not fundamentally different from those faced by researchers in other fields. As economics develops, it will broaden its repertory of methods and sources of evidence, the science will become stronger, and the charlatans will be exposed.

• Robert J. Shiller, a 2013 Nobel laureate in economics, is Professor of Economics at Yale University.

And just some random, dissenting opinion... And one more from The Harvard Crimson...

Also, since the article itself is about "traditional hard science", it makes little sense to frame the example in the same terms. As to whether economics is a science, that all depends on who you listen to. The mainstream or establishment view is that it is a science. This is debatable. Karl Popper's "The Poverty of Historicism" (1936~1957 [a bit complicated - it was first a reading, then a full book but lacked publication for a number of years]) helps to clarify how the point can be contested. Part of the inspiration for the book was Popper wanting to illustrate how both communism and fascism drew inspiration from historicism. Expanding on the quote above:

Nevertheless, in most advanced countries, the following will be pretty generally applicable.

1. Abolition of property in land and application of all rents of land to public purposes.
2. A heavy progressive or graduated income tax.
3. Abolition of all rights of inheritance.
4. Confiscation of the property of all emigrants and rebels.
5. Centralisation of credit in the hands of the state, by means of a national bank with State capital and an exclusive monopoly.
6. Centralisation of the means of communication and transport in the hands of the State.
7. Extension of factories and instruments of production owned by the State; the bringing into cultivation of waste-lands, and the improvement of the soil generally in accordance with a common plan.
8. Equal liability of all to work. Establishment of industrial armies, especially for agriculture.
9. Combination of agriculture with manufacturing industries; gradual abolition of all the distinction between town and country by a more equable distribution of the populace over the country.
10. Free education for all children in public schools. Abolition of children’s factory labour in its present form. Combination of education with industrial production, &c, &c.

Each point made is clearly within the domain of economics, with only point #10 being remotely contestable.

So at a minimum we can see how Marxism (and thus Marxist economics) falls into that category of historicism that Popper wishes to attack. The same holds for fascism.

It might be useful to note that the first reading of the paper "The Poverty of Historicism" was at the invite of Friedrich von Hayek, a classical liberal economist, who at the time would have been considered more "centrist" than today. Here's a fun tidbit to help make that point:

http://en.wikipedia....wiki/Friedrich_Hayek

In 1984, he was appointed a member of the Order of the Companions of Honour by Queen Elizabeth II on the advice of Prime Minister Margaret Thatcher for his "services to the study of economics".

I think we know where most people put Thatcher. :)

But, why would I spend so long blather on about Karl Popper?

http://plato.stanfor....edu/entries/popper/

Additionally, Peter Medawar, John Eccles and Hermann Bondi are amongst the distinguished scientists who have acknowledged their intellectual indebtedness to his work, the latter declaring that 'There is no more to science than its method, and there is no more to its method than Popper has said.'

Because in a thread about science it's useful to understand the man (and his writings) who basically defined science.

But back to the example being one of economics...

If we are to take science as neutral, but economics as political, can we take economics as science? That seems like a hard pill to swallow. Or do we take economics as potentially science, and potentially political, with some criteria by which we can separate the two? I would say that it lies entirely in the answer to whether or not its statements can be falsified.

If I can briefly rephrase my original 2 statements:

Original:
saving money is good

Rephrased:
It is advantageous to have resources to freely draw upon at will

Original:
being in debt is bad

Rephrased:
It is disadvantageous to have future labour allocated to uses that have no personal benefit (this could be better phrased, but close enough)

Whether or not those statements are falsifiable may be open to debate, but that saying them will end up with you being called "right-wing" isn't really up for debate because it happens. Regularly. The SPLC and Mark Potok are great examples there. :)

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Stretching the "peer reviewed" brand until it snaps.
« Reply #106 on: January 07, 2015, 07:30 AM »
I'm not sure whether the term "peer review" carries any real weight nowadays - or whether it retains any scientific credibility or has any real meaning for science.
Interesting review of the state of affairs:
(Copied below sans embedded hyperlinks/images.)
DSHR's Blog: Stretching the "peer reviewed" brand until it snaps
Tuesday, January 6, 2015
Stretching the "peer reviewed" brand until it snaps
The very first post to this blog, seven-and-a-half years and 265 posts ago, was based on an NSF/JISC workshop on scholarly communication. I expressed skepticism about the value added by peer review, following Don Waters by quoting work from Diane Harley et al:

    They suggest that "the quality of peer review may be declining" with "a growing tendency to rely on secondary measures", "difficult[y] for reviewers in standard fields to judge submissions from compound disciplines", "difficulty in finding reviewers who are qualified, neutral and objective in a fairly closed academic community", "increasing reliance ... placed on the prestige of publication rather than ... actual content", and that "the proliferation of journals has resulted in the possibility of getting almost anything published somewhere" thus diluting "peer-reviewed" as a brand.

My prediction was:

    The big problem will be a more advanced version of the problems currently plaguing blogs, such as spam, abusive behavior, and deliberate subversion.

Since then, I've returned to the theme at intervals, pointing out that reviewers for top-ranked journals fail to perform even basic checks, that the peer-reviewed research on peer review shows that the value even top-ranked journals add is barely detectable, even before allowing for the value subtracted by their higher rate of retraction, and that any ranking system for journals is fundamentally counter-productive. As recently as 2013 Nature published a special issue on scientific publishing that refused to face these issues by failing to cite the relevant research. Ensuring relevant citation is supposed to be part of the value top-ranked journals add.

Recently, a series of incidents has made it harder for journals to ignore these problems. Below the fold, I look at some of them.

In November, Ivan Oransky at Retraction Watch reported that BioMed Central (owned by Springer) recently found about 50 papers in their editorial process whose reviewers were sock-puppets, part of a trend:

    Journals have retracted more than 100 papers in the past two years for fake peer reviews, many of which were written by the authors themselves.

Many of the sock-puppets were suggested by the authors themselves, functionality in the submission process that clearly indicates the publisher's lack of value-add. Nature published an overview of this vulnerability of peer review by Cat Ferguson, Adam Marcus and Oransky entitled Publishing: The peer-review scam that included jaw-dropping security lapses in major publisher's systems:

    [Elsevier's] Editorial Manager's main issue is the way it manages passwords. When users forget their password, the system sends it to them by e-mail, in plain text. For PLOS ONE, it actually sends out a password, without prompting, whenever it asks a user to sign in, for example to review a new manuscript.

In December, Oransky pointed to a study published in PNAS by Kyle Silera, Kirby Leeb and Lisa Bero entitled Measuring the effectiveness of scientific gatekeeping. They tracked 1008 manuscripts submitted to three elite medical journals:

    Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research.

Desk-rejected papers never even made it to review by peers. Its fair to say that Silera et al conclude:

    Despite this finding, results show that in our case studies, on the whole, there was value added in peer review.

These were elite journals, so a small net positive value add matches earlier research. But again, the fact that it was difficult to impossible for important, ground-breaking results to receive timely publication in elite journals is actually subtracting value. And, as Oransky says:

    Perhaps next up, the authors will look at why so many “breakthrough” papers are still published in top journals — only to be retracted. As Retraction Watch readers may recall, high-impact journals tend to have more retractions.

Also in December, via Yves Smith, I found Scholarly Mad Libs and Peer-less Reviews in which Marjorie Lazoff comments on the important article For Sale: “Your Name Here” in a Prestigious Science Journal from December's Scientific American (owned by Nature Publishing). In it Charles Seife investigates sites such as:

    MedChina, which offers dozens of scientific "topics for sale" and scientific journal "article transfer" agreements.

Among other services, these sites offer "authorship for pay" on articles already accepted by journals. He also found suspicious similarities in wording among papers, including:

    "Begger's funnel plot" gets dozens of hits, all from China.“Beggers funnel plot” is particularly revealing. There is no such thing as a Beggers funnel plot. ... "It's difficult to imagine that 28 people independently would invent the name of a statistical test,"

Some of the similarities may be due to authors with limited English using earlier papers as templates when reporting valid research, but some such as the Begger's funnel plot papers are likely the result of "mad libs" style fraud. And Lazoff points out they likely used sockpuppet reviewers:

    Last month, Retraction Watch published an article describing a known and partially-related problem: fake peer reviews, in this case involving 50 BioMed Central papers. In the above-described article, Seife referred to this BioMed Central discovery; he was able to examine 6 of these titles and found that all were from Chinese authors, and shared style and subject matter to other “paper mill-written” meta-analyses.

Lazoff concludes:

    Research fraud is particularly destructive given traditional publishing’s ongoing struggle to survive the transformational Electronic Age; the pervasive if not perverse marketing of pharma, medical device companies, and self-promoting individuals and institutions using “unbiased” research; and today’s bizarrely anti-science culture. 

but goes on to say:

    Without ongoing attention and support from the entire medical and science communities, we risk the progressive erosion of our essential, venerable research database, until it finally becomes too contaminated for even our most talented editors to heal.

I'm much less optimistic. These recent examples, while egregious, are merely a continuation of a trend publishers themselves started many years ago of stretching the "peer reviewed" brand by proliferating journals. If your role is to act as a gatekeeper for the literature database, you better be good at being a gatekeeper. Opening the gate so wide that anything can get published somewhere is not being a good gatekeeper.

The fact that even major publishers like Nature Publishing are finally facing up to problems with their method of publishing that the scholars who research such methods have been pointing out for more than seven years might be seen as hopeful. But even if their elite journals could improve their ability to gatekeep, the fundamental problem remains. An environment where anything will get published, the only question is where (and the answer is often in lower-ranked journals from the same publishers), renders even good gatekeeping futile. What is needed is better mechanisms for sorting the sheep from the goats after the animals are published. Two key parts of such mechanisms will be annotations, and reputation systems.

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #107 on: January 07, 2015, 09:49 AM »
I'm not sure whether the term "peer review" carries any real weight nowadays - or whether it retains any scientific credibility or has any real meaning for science.
Interesting review of the state of affairs:

I lost faith a couple decades ago. I've since only received confirmation in my apostasy.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #108 on: January 30, 2015, 05:23 AM »
In this discussion thread, in a response to @xtabber here, I drew three conclusions:
Some conclusions we could arrive at here would include:
  • A. Truth: You can't make something true out of a collection of logical fallacies. That would be an assault upon reason. Once you accept one invalid premise, you can accept infinitely more.
    However, the depressing reality seems too often to be that many people are so unable to think rationally for themselves that they seem gullible to this kind of barrage of logical fallacy. One's head would be full of a confusing and probably conflicting mass of invalid premises, with ergo no real knowledge or understanding of truth.

  • B. Peer review per se is not crucial as it cannot and does not certainly establish truth: We have already seen, in this discussion thread and others - e.g., including, the thread on CAGW, Thermageddon? Postponed! - that there is plenty of evidence to demonstrate pretty conclusively that peer review is an unreliable instrument for determining truth, as it can be and has been, and probably will continue to be used/abused to rationalise whatever careless or unethical/misguided scientists might want, because they cannot otherwise scientifically prove a pet theory or preferred/biased conclusion.
    This is also well-documented in the literature - e.g., including as referred to in one of the spoilers above("…on broken trust in peer review and how to fix it").

  • C. Falsifiability is crucial:
    Falsifiability or refutability is the property of a statement, hypothesis, or theory whereby it could be shown to be false if some conceivable observation were true. In this sense, falsify is synonymous with nullify, meaning not "to commit fraud" but "show to be false". Science must be falsifiable. - Wikipedia.
__________________________________

One of the things that often puzzles me is how easily we seem to be conned by false peer reviews and how we are seemingly so wilfully blind to the truth in things, and so I was very interested to read the lifehacker post Carl Sagan's Best Productivity Tricks, where it says:
(Copied below sans embedded hyperlinks/images.)
...Hone Your "Baloney Detection Kit"
Sagan was first and foremost a scientist, and that means he had a very specialized outlook on the world. In his book, The Demon Haunted World, he outlines what he calls his "baloney detection kit." The kit is essentially a means to test arguments and find fallacies. It's a great toolset for skeptical thinking. Here's part of his kit:
  • Wherever possible there must be independent confirmation of the "facts."
  • Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
  • Arguments from authority carry little weight — "authorities" have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that in science there are no authorities; at most, there are experts.
  • Spin more than one hypothesis. If there's something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each of the alternatives. What survives, the hypothesis that resists disproof in this Darwinian selection among "multiple working hypotheses," has a much better chance of being the right answer than if you had simply run with the first idea that caught your fancy.
  • Try not to get overly attached to a hypothesis just because it's yours. It's only a way station in the pursuit of knowledge. Ask yourself why you like the idea. Compare it fairly with the alternatives. See if you can find reasons for rejecting it. If you don't, others will.
Sagan's kit here isn't just for science, of course. It's great for everything, from presidential debates to statistics. When you challenge those biases, you walk away with a better point of view. It's also a good toolset if you're making an argument at work, giving a presentation in school, or even just taking on a lively debate at the dinner table. The better you are at detecting baloney, the better your arguments will be in the long run. ...
(Read the rest at the link.)

It's all "habits of mind" really - thinking skills (De Bono).

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #109 on: January 30, 2015, 06:32 AM »
Regarding "Spin more than one hypothesis," academia is a complete laughingstock. Just look at how doctoral dissertations are done - create a hypothesis, then show it. This is simply idiotic. A thought out and careful hypothesis that proves to be false is also information, i.e. we now know that X is false, or, limiting the scope or field is also a valid assertion. But this doesn't happen when everyone tries to prove that their hypothesis is "true". Certainly, 'true' is sexier, but 'false' can also be useful, especially when we know why it is false.

But, just for kicks, here's a statement for people to mull over for a second, assess, and then click the spoiler. ;)

Statement) There are no studies that show that tobacco causes cancer.

Spoiler
That's actually true.

The studies are statistical and do not control for tobacco vs. tobacco laced with chemicals, which almost all tobacco is. e.g. Cigarettes are laced with ammonia because it freebases the nicotine and makes it 35x more powerful, and more addictive. There are many more chemicals added as well.

The question about what part of the cigarette causes cancer is unanswered. Is it the tobacco? Or something else?

Please - Do not read into that more than I've stated. I have not said "cigarettes are good and rainbow farting unicorns". Is it likely that tobacco by itself causes cancer? Probably. I don't know. I have no evidence to say one way or another, other than the flawed studies on it.


Oh, and just for fun... 1 more...

There are no peer-reviewed, double-blind, placebo-controlled studies to prove that large caliber bullets shot into the head cause death. :P

I like to pull that one out for those who are religious about science. They invariably very quickly retreat from their faith just long enough to start calling me names, then they return to their faith just as quickly. I find it probably just as amusing as they find it infuriating. :)

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #110 on: February 05, 2015, 08:33 AM »
Some very uncomfortable damnation for "peer review" and "science"...

FAIR WARNING: This will upset some people. There are many stories in the MSM revolving around this broader issue at the moment.

Fraud at Merck.

Link to source document (PDF).

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #111 on: February 05, 2015, 09:27 AM »
@Ren - just out of curiosity, what grounds or expertise do you fall back on when it comes to accepting the truth and accuracy of these things you cite? Not being a chemist or a biologist or geneticist myself, I could be duped fairly easily by either side since I don't have enough background or education to evaluate and have an informed "scientific" opinion about most of what is being said here.

When it comes to people like me (i.e. fairly bright and well-read but NOT possessing any real expertise - and having, at best, a cursory knowledge of subject matter itself) I could easily be bamboozled by a cherry-picked yet utterly bogus dissenting opinion. Nor would I know enough to be able to determine the validity of the experiment design used in the study in question. Nor do I have access to the raw data. So despite being pretty damn good when it comes to statistics, I can't even check the validity of any statistically-based conclusion(s) being drawn.

Googling the web and reading isolated studies and opinions isn't the same thing as "getting educated" or becoming knowledgeable about the subject matter in question. That's why people go to school and study the broader base of knowledge needed. So they may someday be able to confidently hold a truly informed opinion.

While I don't automatically dismiss a lot of what you're saying or arguing for out of hand, I'm still less confident than you are about the validity of a lot of the 'evidence' being presented. And I think I may be just a little more aware of the possibility that what I personally wish to be true may seriously affect my ability (and willingness) to see what actually is true - if I'm not extremely careful.
 :)
« Last Edit: February 05, 2015, 09:35 AM by 40hz »

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #112 on: February 05, 2015, 11:25 AM »
@40 - I said exactly nothing towards one side of the debate on the topic. I merely pointed out that a researcher at Merck has pointed out that they were involved in scientific fraud. I think that I'm educated enough to point out that a scientific researcher has blown the whistle on fraud. This isn't really an issue of qualifications. ;)

Given that this researcher was directly involved in "revising" the data, I kinda think he might have a thing or two to say on the issue/fraud.

But, y'know... Like I said... It's going to upset some people. ;)

While I don't automatically dismiss a lot of what you're saying or arguing for out of hand, I'm still less confident than you are about the validity of a lot of the 'evidence' being presented.

Are you high? I post drunk sometimes, and I do go off the rails every now and then, but this is pretty clear cut -- a researcher at Merck has blown the whistle that they published fraudulent information about one of their products. That's all. It's a simple matter of fact. Are you trying to say that Stephen A Krahling doesn't actually exist, or that he didn't blow the whistle? ;)

Or, are you trying to say that one of the researchers involved in the product that has blown the whistle isn't qualified to talk about what he was researching? ;)

In case you missed it, here's the document for the United States District Court for the Eastern District of Pennsylvania where the action took place:

* Merck-False-Claims-Act.pdf (944.86 kB - downloaded 223 times.)

I suppose that someone could look it up here:

https://www.paed.uscourts.gov

But... from here: https://www.paed.usc...urts.gov/us01001.asp

What You Need
  • Personal computer
  • Compatible Web browser (CM/ECF has been tested and works correctly with Netscape 4.7x and 7.0x. and Internet Explorer 5.5 and 6.0)
  • Internet access
[/size]

I'm not that confident that they're very competent. ;)

I told you it would be uncomfortable. :D
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #113 on: February 05, 2015, 12:50 PM »
Are you high? I post drunk sometimes, and I do go off the rails every now and then, but this is pretty clear cut

Thanks for that. Although I've noted you do (by your own admission) post in a less than ideal state of mind sometimes, I'd never think to call you drunk in a discussion.

I also think I see where some confusion enters into the picture. I wasn't responding to your post immediately above mine. It was a more general question - as in I don't know much about a lot of this, so do you have some good in-depth sources (like an online university level course or two) you'd care to share that puts you in a better position than me to say what's real when it comes to this stuff?

I probably should have been more explicit. I wasn't calling BS on you. I was just looking for some direction or suggestions since you're far more into science controversies than I am. And I am very aware of how easy it is to fool ourselves - so it's a very big concern of mine that I don't do that to myself..

Apologies if I wasn't clear and you felt offended. (Re-reading my post lets me see it could easily have be taken the way you did.) Chalk it up to somebody who had to go out on icy roads in 6 degree F weather last night, drive 28 miles to replace a failed RAID element on a client's server, sit there until about 6am to make sure everything was back to normal - then drive home with a major headache.

I probably should have taken a nap or drank more coffee before I came here and posted. Oh well. At least I took some aspirin. :)

tomos

  • Charter Member
  • Joined in 2006
  • ***
  • Posts: 11,961
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #114 on: February 05, 2015, 02:13 PM »
@40, this does not seems to me to be about sides. If the scientists were working with vaccines, they are presumably pro-vaccine. As Ren says, this is whistle blowing.

And let's face it, who can be genuinely surprised about *any* corporation fixing the books where they are given the chance to do so? That's what is most disturbing - that they are given that chance:
it sounds like the company is testing the efficacy of its own product for the FDA. And that the product got approval based on those tests. And if the reports are correct, that the test guidelines were not adhered to, and that the FDA didnt check that - in spite of irregularities being reported.

[this not directed at any of the posts above] The danger of taking 'sides' re vaccines, is that one can get defensive and lose objectivity. Which can contribute to allowing stuff like this to happen. (FWIW I'm broadly pro-vaccine myself, but dubious about the methods used to force it on the populace, and dubious about relations between the FDA and the corporations developing the vaccines.)
Tom

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #115 on: February 05, 2015, 02:49 PM »
^@tomos - See my post just above yours. I think there was a disconnect between what I thought I was asking Ren, and what Ren thought I was inferring. If there was any confusion there, the fault was entirely mine.

And I agree. None of this is (or should be) about taking sides.  :)
« Last Edit: February 05, 2015, 03:23 PM by 40hz »

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #116 on: February 05, 2015, 06:49 PM »
Apologies if I wasn't clear and you felt offended.

Me? Hell no! :) I just figure that you like to poke me a bit every now and then to see how I react (or, go off the rails as it were). ;D I'd be remiss if I didn't! 8)


I also think I see where some confusion enters into the picture. I wasn't responding to your post immediately above mine. It was a more general question - as in I don't know much about a lot of this, so do you have some good in-depth sources (like an online university level course or two) you'd care to share that puts you in a better position than me to say what's real when it comes to this stuff?

There's not much that anyone needs to know. A guy blew the whistle.

Now, for the underlying issue, yes - we would need a good amount of additional information, e.g. university level courses, to comment on the "science". However, I don't think that this is one topic that we should get into as it is simply far too hot.

There are quite a few lectures (from experts in the field) that you can view online about the underlying issue. The range of opinions spans the full gambit from super-uber-pro-vaccine, down through the range of healthy skepticism, and way on over to rabidly-anti-vaccine.

Tomos hits exactly what we should be concerned about:

(FWIW I'm broadly pro-vaccine myself, but dubious about the methods used to force it on the populace, and dubious about relations between the FDA and the corporations developing the vaccines.)

But, those issues are about 1) forced medication, and 2) corruption, which while a degree in philosophy or law is certainly an advantage in discussing them, there's enough wiggle room for input from others. :)

However, that's off topic. My original point in the post above was to point out a whistle blower in the scientific community. This goes to illustrate:

1) "Scientists" are human.
2) Humans are corruptible.

It is that bit of corruption that is a serious issue for those that claim "peer review" is some kind of holy cow in science.

 
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #117 on: February 05, 2015, 08:37 PM »
...It is that bit of corruption that is a serious issue for those that claim "peer review" is some kind of holy cow in science.

Yes, that seems quite true - and of course there should be no "holy cows" in science. As Feynman and other scientists have pointed out, there should be just science that is transparent and open to the ultimate test of falsifiability, so that anyone can repeat the experiments with a view to seeing whether they can get the same consistently repeatable results. If they can't, then it is falsifiable, and the hypothesis is invalidated, so we have learned something that we might not have known before - it's a step forwards, not backwards. This is what Thomas Edison referred to regarding all his failures before he finally arrived at a working concept of the light bulb.

Though there is nothing in the scientific process itself that mentions or necessitates something called "peer review", I have occasionally read about people (who are presumably unfamiliar with the scientific process) saying dismissively about some scientific paper or other "Oh yes, that's all very well, but has it been peer reviewed? - because it's not scientifically verified until it has been.". That of course is quite incorrect.

My view is that, whereas peer review could be expected to be a process of considerable use in potentially improving the quality of science papers, it has unfortunately been so much abused (by now), and by so many unscrupulous individuals/groups, as a handy device to cover up the falsity of their preferred science outcomes (QED) - so as to substantiate some ulterior purpose or other - that one can no longer safely assume to have trust in either the peer review process or the scientists (where they are scientists) employing the process to verify some piece of science. You are obliged to to check it for yourself - "Nullius in verba". The reality is that you can't take someone's word for anything if you want to get at the uncorrupted truth (and that goes double for irrational pronouncements coming out of the mouths of ostensible representatives of the Royal Society).

As a case in point, I have coincidentally only just today posted this (below) about the apparently deliberate falsification of scientific data on what looks like a wholesale basis: Forensic analysis of stochastic chicanery in climate temp time series data
...Things are not always as people would have you see.
One of the greatest pleasures and what I have always enjoyed about statistics is that the data doesn't lie - it just is  - and I find that it is invariably "trying to tell us something". That is, there is some truth for us to discover.
I was making notes on an interesting report posted - what looks like genuine investigative journalism - on the Daily Telegraph website, about data tampering. Because the report provides quite a good summation of the scale of the apparent fraud(s), I have copied my notes below as an image, and copied the raw text to a spoiler below the image.
Everything in the article checks with "peer reviewed" research and can be independently verified (however, being skeptical, I have verified it myself anyway - "Nullius in verba").
What concerns me greatly about this is that, not only has this this scientific data fraud apparently been quite extensive and going on for years, peer reviewed and approved, but it has been done quite deliberately - and undisclosed/concealed.

The DT post URL is: http://www.telegraph...-global-warming.html

The DT post makes reference to the source of analysis of where some pretty significant data tampering has been going on which has only recently been discovered by forensic statistical analysis.
The source is Notalotofpeopleknowthat ( http://notalotofpeop...owthat.wordpress.com )
The specific source article and URL is:
Massive Tampering With Temperatures In South America
https://notalotofpeo...es-in-south-america/
...

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #118 on: February 05, 2015, 09:04 PM »
On the topic of data fraud, Benford's law is extremely interesting. It has to do with the distribution of digits. Here's a nice explanation of it:



And the associated blog post:

http://periodicvideo...-youtube-videos.html

Of course, if you already know this, you can still create fraudulent data that conforms to Benford's law; that merely indicates a higher level of sophistication on the part of the fraudster.
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #119 on: February 06, 2015, 12:24 PM »
I've actually tried to apply Benford's law on an exercise in accountancy forensic auditing a while back, just out of interest. It did not add any/much value to the exercise at the time, as I recall, as it was not really applicable.
Was interesting though, as it could maybe detect the smoke, if not the gun.

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #120 on: February 06, 2015, 01:45 PM »
The sad thing is it isn't the scientific process or peer review that's the problem. It's the subversion and perversion of them that's causing the huge amount of recent public distrust in things "scientific." And whenever there's doubt, there's always someone ready and willing to step in and twist it to their own advantage.

Be it politics, economics, religion, or science - once the practitioners of a given discipline abandon their high road, a mounting wave of public distrust soon follows. And surfing that wave are all the self-nominated pundits and opportunists. Each with an agenda.

To my mind, that's the real tragedy - and danger - in all of this.

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #121 on: February 06, 2015, 09:33 PM »
^ As you've pointed out many times in other contexts... "It's a people problem."  :Thmbsup:
Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

Renegade

  • Charter Member
  • Joined in 2005
  • ***
  • Posts: 13,288
  • Tell me something you don't know...
    • View Profile
    • Renegade Minds
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #122 on: February 07, 2015, 08:41 AM »
Anyone ready for a real lovely little mindf**k? Get out the intellectual KY jelly and lube your mind. It's going to get a bit rough! ;)

To save time, start here: http://youtu.be/JKHUaNAxsTg?t=10m

That's the 10 minute mark.



He points out how some scientific constants fluctuate, and specifically the speed of light and the universal gravitation constant. More in the video.

If he's right, "peer review" has had a seriously massive cluster-f**k of a disaster of cosmic proportions. Beyond that, the implications of the incompetence of those involved is simply astounding.

Slow Down Music - Where I commit thought crimes...

Freedom is the right to be wrong, not the right to do wrong. - John Diefenbaker

40hz

  • Supporting Member
  • Joined in 2007
  • **
  • Posts: 11,858
    • View Profile
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #123 on: February 07, 2015, 04:28 PM »
I'm not surprised it wasn't aired. I watched it twice and still think it's pretty lame.

Here's Mr. Sheldrake's write-up in Wikipedia:

Rupert Sheldrake


Alfred Rupert Sheldrake is an English scientist,[3][4] author,[3] public speaker,[5] and researcher in the field of parapsychology,[6] known for his "morphic resonance" concept.[7] He worked as a biochemist and cell biologist at Cambridge University from 1967 to 1973[3] and as principal plant physiologist at the International Crops Research Institute for the Semi-Arid Tropics until 1978.[8]

Conceived during Sheldrake's time at Cambridge, morphic resonance posits that "memory is inherent in nature"[3][9] and "natural systems, such as termite colonies, or pigeons, or orchid plants, or insulin molecules, inherit a collective memory from all previous things of their kind".[9] Sheldrake proposes that it is also responsible for "telepathy-type interconnections between organisms".[4] His advocacy of the idea encompasses paranormal subjects such as precognition, telepathy and the psychic staring effect[10][11] as well as unconventional explanations of standard subjects in biology such as development, inheritance, and memory.[12]

Morphic resonance is not accepted by the scientific community as a real phenomenon and Sheldrake's proposals relating to it have been characterized as pseudoscience. Critics cite a lack of evidence for morphic resonance and an inconsistency of the idea with data from genetics and embryology, and also express concern that popular attention from Sheldrake's books and public appearances undermines the public's understanding of science.[a]

Despite the negative reception Sheldrake's ideas have received from the scientific community, they have found support in the New Age movement,[26] such as from Deepak Chopra.[27][28] Sheldrake argues that science should incorporate alternative medicine, psychic phenomena, and a greater focus on holistic thinking...

In short, a whole lotta New Age magical and wishful thinking being peddled.

Just because something would be really cool if it existed (or worked a certain way), it's still a very far cry from establishing that it's real. Science says if there's no concrete and verifiable evidence to support a hypothesis, it remains an unproven hypothesis - at best. New Age says: the absence of verifiable evidence is immaterial because that invariably means you're looking in the wrong place - or at it in the wrong way. In short, you lack the "gift," or the "awareness," or are not "sufficiently evolved" to see things the way they do.

I personally heard nothing new in what he's saying. I'm a child of the 60/70s and I've heard almost all of this talk in one form or another before. Those 60/70s were fun times. Anything was possible! And the incontrovertible proof and validation of all those marvellous and mystical "alternate realities" were (supposedly) just around the corner. They'd all "eventually" (that favorite word of dreamers) become established fact and "western so-called science" would finally get its well-deserved comeuppance! Yup! Any day "real soon now."

And just a short forty years later I'm pleased to report...uh...we're still waiting??? WTF?  :huh:

When he opened up with the usual shop-worn New Age straw man attacks on science (which does not make the claims he says it does), my eyes already started to glaze over. A fun thing to watch perhaps. But hardly "an idea worth sharing." Unless maybe you're into the whole post-modern/deconstructionist way of 'thinking.'

Also, the characterization that his talk was "banned" and "censored" is not accurate. That was the characterization made by the person (revolutioneevolve) who posted the video under their own YT account. TED had merely decided the presentation did not measure up to their standards for inclusion in the TED Talks and removed it from their own TED channel. They have not issued DMCA notices to have it taken down from anybody else's channel. So I hardly see where there's some vast conspiracy to suppress and silence (another favorite New Age accusation trotted out whenever anyone disagrees with a favorite New Age guru) Mr. Sheldrake. And they have a whole page on the Open for Discussion blog at TED that explains the whys and wherefores of their action - along with counter arguments and objections by Robert Sheldrake himself. So he's hardly being censored - not that TED actually has the authority or clout to do something like that to begin with. They're not a government or church. So how could they possibly "censor" anything?

When you get past his (and his supporters') hyperbole, I think it's clear that the people who bring us TED were not impressed with Mr. Sheldrake's talk and decided they'd rather not host it on their main channel as a result. And it might possibly also be worth remembering that deciding what TED wants their name associated with is something that is well within their rights considering how TED is a private organization rather than a public institution. I can't see many here disputing that right. At least not in good faith considering we're (mostly) the good liberal/libertarian/anarcho-leaning (pick a flavor!) thinkers we are. Right?

This is where the people at TED explain their position and decision on Mr. Shendrake's talk in greater depth. It hardly sounds like the Spanish Inquisition to me.
 ;) 8)
« Last Edit: February 08, 2015, 08:34 PM by 40hz »

IainB

  • Supporting Member
  • Joined in 2008
  • **
  • Posts: 7,540
  • @Slartibartfarst
    • View Profile
    • Read more about this member.
    • Donate to Member
Re: Peer Review and the Scientific Process
« Reply #124 on: February 08, 2015, 06:43 AM »
Never mind the New Age movement, astrology, or ectoplasmic production, peer review of alchemy gets you no further, even if Sir Isaac Newton is doing the review. Peer review of Isaac Newton’s physics was rather hard as he was, and remains, peerless in that field.