Main Area and Open Discussion > Living Room
Peer Review and the Scientific Process
IainB:
I wondered when the UK would take some action to stop the evident decline in academic research standards achieved over the last few years, as displayed in some sub-standard output from some UK universities and some other UK research bodies (e.g., in the domains of climate science and medical research). Looks like they have started to confront/address the issues by focussing on aligning funding allocation with research integrity. (As the saying goes, "Follow the money"?)
Better late than never, I suppose. Yay for Britain! :up:
Let's hope it has some good effect. We shall see.
Britain’s Bad Science Scandal: UK Research Position Threatened By Fact-Fabricators
Date: 18/06/13
John Lawless, The Independent
Britain’s leading science institutions will be told on Monday that they will be stripped of many millions of pounds in research grants if they employ rogue researchers who fake the results of experiments, The Independent has learnt.
The clampdown comes as retractions of scientific claims by medical journals are on course to top 500 for the first time in 2013 – having been just 20 a year in the late 1990s, when Andrew Wakefield notoriously claimed that the MMR vaccine caused autism in children. In April, the UK’s first researcher was jailed for falsifying data over a prolonged period.
The Government is concerned that Britain’s prized second place in global research behind the US will be at threatened if more fact-fabricators are exposed. It knows that hundreds of thousands of jobs could easily go to foreign rivals if British laboratories do not keep coming up with new product ideas, to be made by major multinational companies in UK factories.
All of the country’s 133 universites and colleges of higher education are being forced to sign a new Concordat for Research Integrity – having been warned by major fund providers that those who do not will be refused access to more than £10 billion in research grants funded each year by British taxpayers – and as much again from the private sector.
A spokesman for Universities UK, which chaired negotations with the grant providers, said: “From next year, universities in the UK will have to prove compliance with the research integrity concordat in order to receive research grant. They are doing this to help demonstrate to government, business, international partners and the wider public that they can continue to have confidence in the research.”
Full article in The Independent: The bad science scandal: how fact-fabrication is damaging UK's global name for research.
--- End quote ---
TaoPhoenix:
Separately from the money side, I have pondered a bit on different types of "weighted rating" systems for papers, then the reviewers. So very basically, Researcher A produces a Good Paper, and gets a "Good Researcher" score. Reviewer A reviews it correctly, and gets a Good Reviewer score. So in a simple world, if you get a paper from those two, you can generally "trust it".
But if one or the other goes rogue, then they will begin to collect "bad scores", and eventually like crying wolf game theory, they won't be believed even if later they start to turn around.
I know, everything can be gamed, but at the top of the basic theory, it becomes a shorthand for quality.
barney:
Separately from the money side, I have pondered a bit on different types of "weighted rating" systems for papers, then the reviewers. So very basically, Researcher A produces a Good Paper, and gets a "Good Researcher" score. Reviewer A reviews it correctly, and gets a Good Reviewer score. So in a simple world, if you get a paper from those two, you can generally "trust it".
But if one or the other goes rogue, then they will begin to collect "bad scores", and eventually like crying wolf game theory, they won't be believed even if later they start to turn around.
I know, everything can be gamed, but at the top of the basic theory, it becomes a shorthand for quality.
-TaoPhoenix (June 18, 2013, 06:49 PM)
--- End quote ---
Personalities weigh in as well, sometimes quite heavily. For instance, say I dislike TaoPhoenix. Any paper submitted with that name on it - even if it was research done by someone else! - is likely to get a bad review from me ... assuming I bother to read it! Peer review has a place, as does any other form of review, but consideration has to be afforded the attitudes of the reviewers, regardless their position(s).
mahesh2k:
I believe there are absolute limits to human ability, and that there are phenomena in the universe that are not susceptible to scientific investigation, however advanced our tools become.
--- End quote ---
If we set such goalpost at first while making any theory and then try to base our assumptions thereafter, then it doesn't become critical thinking theory but more of theory based on beliefs and the goal post always changes when any proof threatens the goalpost placed earlier. Such reviews then become biased rather than critical. If the root of any theory isn't challenged it becomes biased and more personal.
barney:
Y'know, it strikes me that DC is a source of much peer review. Someone requests a coding snack, for instance. One (1) of you takes up the challenge, offers a product. Then, usually, several folk in addition to the requester critique the product. Then the creator revises it, republishes it - often with code that purveyors can examine - and it goes through review again, often, multiple times.
Yeah, I realize that's not the greatest analogy, but the process is not dissimilar. We have out naysayers - a few - and we have our innovators and we have our reviewers (wallflowers like me don't count :P.). There's not a significant difference, save for the range of recognition, methinks.
Navigation
[0] Message Index
[#] Next page
[*] Previous page
Go to full version