Main Area and Open Discussion > Living Room
Peer Review and the Scientific Process
Renegade:
Here's an interesting bit on bias:
http://grist.org/politics/science-confirms-politics-wrecks-your-ability-to-do-math/
Everybody knows that our political views can sometimes get in the way of thinking clearly. But perhaps we don’t realize how bad the problem actually is. According to a new psychology paper, our political passions can even undermine our very basic reasoning skills. More specifically, the study finds that people who are otherwise very good at math may totally flunk a problem that they would otherwise probably be able to solve, simply because giving the right answer goes against their political beliefs.
The study, by Yale law professor Dan Kahan and his colleagues, has an ingenious design. At the outset, 1,111 study participants were asked about their political views and also asked a series of questions designed to gauge their “numeracy,” that is, their mathematical reasoning ability. Participants were then asked to solve a fairly difficult problem that involved interpreting the results of a (fake) scientific study. But here was the trick: While the fake study data that they were supposed to assess remained the same, sometimes the study was described as measuring the effectiveness of a “new cream for treating skin rashes.” But in other cases, the study was described as involving the effectiveness of “a law banning private citizens from carrying concealed handguns in public.”
--- End quote ---
More at the link.
IainB:
^^ Nice find, but watch that space. In the US it will almost inevitably be used/abused to attempt to irrationally load one's argument in favour of the proponents of this or that religio-political ideology, and ad hom the proponents of contradictory religio-political ideologies.
In any event, there's little new about it. Edward De Bono had already identified the problem years ago in his book "Teaching Thinking" - he called it "intellectual deadlock", and pointed out how it effectively disables one from developing/using thinking skills.
He also pointed out in that and/or a later book that the practice of "taking positions" in debate was likely to be one of the single greatest inhibiting factors to our development, leading to wars and holding back man's evolution. You can substitute "adopting a religio-political ideology" for "taking positions". It sets one's paradigms rock solid so that - regardless of verifiable observational evidence - you can't see or think with any other so-called "truth" (belief/dogma) except that which your paradigm allows you to see.
Now try and prove that peer review can actually add any objective truth to or objectively validate any part of the scientific process.
I predict it is likely to be impossible.
If we are interested in Truth, and if we wish to be something more than unthinking parrots reciting some moronic dogma of a religio-political ideology (system of belief) for most of our lives (which I would argue is realising at best only a sub-human potential), then it seems that one has to fall back on "Nullius in verba/verbo." Motto of the Royal Society, London. Literally, "Take nobody's word for it; see for yourself".
If you need an example of what I mean, watch this depressing piece of video footage of an interview, and weep: (I only came across this by accident yesterday)
Edit 2018-06-12 - original YouTube link no longer available, so inserted link from Wayback.
Original link: <http://www.youtube.com/watch?v=KvufOvneJMk>
Wayback link: <http://web.archive.org/web/20100329013133/https://www.youtube.com/watch?v=KvufOvneJMk>
Though it is from 2010, it is just as significant today as it was then.
The interviewer, who is evidently well-informed, just stands there politely asking simple, very factual questions based on independently verifiable data (no trick questions), and the interviewee - who should be well-informed - answers them to the best of her ability.
I felt acutely embarrassed for her.
Renegade:
^^ Nice find, but watch that space. In the US it will almost inevitably be used/abused to attempt to irrationally load one's argument in favour of the proponents of this or that religio-political ideology, and ad hom the proponents of contradictory religio-political ideologies.
-IainB (October 20, 2013, 05:05 PM)
--- End quote ---
Yes. Someone will come up with some skewed "study", then use that to badger people into accepting their skewed conclusion.
Dressing pigs (fraud) up in pink dresses (as science) seems to be far too common.
In a few cases, you can trace problems back to deeper problems in science where accepted science simply doesn't pan out the way it is purported to.
He also pointed out in that and/or a later book that the practice of "taking positions" in debate was likely to be one of the single greatest inhibiting factors to our development, leading to wars and holding back man's evolution. You can substitute "adopting a religio-political ideology" for "taking positions". It sets one's paradigms rock solid so that - regardless of verifiable observational evidence - you can't see or think with any other so-called "truth" (belief/dogma) except that which your paradigm allows you to see.
-IainB (October 20, 2013, 05:05 PM)
--- End quote ---
One of the nice things about "debate" is that they are set up as FOR/AGAINST, which entirely misses the point about finding some sort of truth for most topics. The topic itself needs to naturally fit into that or else you're simply left with a false dilemma, which again, is perfect for the religio-political zealots.
If we are interested in Truth, and if we wish to be something more than unthinking parrots reciting some moronic dogma of a religio-political ideology (system of belief) for most of our lives (which I would argue is realising at best only a sub-human potential), then it seems that one has to fall back on "Nullius in verba/verbo." Motto of the Royal Society, London. Literally, "Take nobody's word for it; see for yourself".
-IainB (October 20, 2013, 05:05 PM)
--- End quote ---
This is far more difficult than it sounds at first glance.
At some point we need to trust someone else to have studied a topic properly, and to have presented the evidence objectively and fairly.
Like you mentioned elsewhere - one lifetime is not nearly enough.
I felt acutely embarrassed for her.
-IainB (October 20, 2013, 05:05 PM)
--- End quote ---
Oh good grief...
"We're on different planets."
Yes. You are on a different planet.
"I have a job and I don't have time to check..."
Well, there's a large part of the problem. In a world where 2 incomes are often needed for a family to survive, it isn't really justifiable to blame people for prioritizing putting food on the table and trusting so-called "experts" vs. verifying data themselves.
Then again, if you don't have the time to check at least at a basic level, you might want to consider just keeping your trap shut.
barney:
Well, there's a large part of the problem. In a world where 2 incomes are often needed for a family to survive, it isn't really justifiable to blame people for prioritizing putting food on the table and trusting so-called "experts" vs. verifying data themselves.
Then again, if you don't have the time to check at least at a basic level, you might want to consider just keeping your trap shut.
-Renegade (October 20, 2013, 06:54 PM)
--- End quote ---
Been following this thread for a while, but I must admit to a[n apparent] total lack of understanding. Since any review is a peer review (some peers being greater (or lesser) than others, of course, and some being councils rather than individuals), we must accept such reviews at face value or perform the reviewed activity/action/report ourselves. Shamefully, I'm not qualified to review certain nuclear/bio-molecular/physical reports myself (missed those particular days in class, donchano), but, being of [somewhat] rational mind, I do have the capacity of judgement according to mine own particular/weird assumptions and assessments.
That condition, presumably, exists in all reasoning beings - at least to some extent.
Hence, what is the problem with peer review, per se? Bogus judgements? See 'em all the time. This seems nothing more than a complaint against human nature.
xtabber:
Peer Review has a very specific meaning in academia.
Academic journals use panels of reviewers with experience in the fields they cover to screen manuscripts submitted to them and determine whether they are worthy of being published by the journal. These reviewers are the "peers" of the authors and papers they accept for publication are said to have been "peer reviewed."
Note that acceptance for publication does not necessarily mean that the reviewers agree with a paper, just that they find it to be good enough to be published in that journal. Thus "peer review' is (or at least should be - there are many abuses in practice) more of a triage process than an endorsement. The main purpose is to weed out the cranks and charlatans who would otherwise flood the journal just as they do the comments sections of many blogs.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version