Home » Archive

Articles tagged with: peer review

Blog, Featured »

[28 Oct 2010 | One Comment | ]
Binary decisions are a real problem in a grey-scale world

I recently made the most difficult decision I’ve had to take thus far as a journal editor. That decision was ultimately to accept the paper; that probably doesn’t sound like a difficult decision until I explain that I made this decision despite a referee saying I should reject the paper with no opportunity for resubmission not once, but twice. One of the real problems I have with traditional pre-publication peer review is the way it takes a very nuanced problem around a work which has many different parts and demands that you take a hard yes/no decision.

Presentations »

[12 Sep 2010 | 3 Comments | ]

A talk given in two slightly different forms at the NFAIS annual meeting 2010 (where I followed Clay Shirkey, hence the title) and at the Society for General Microbiology in Edinburgh in March. In the first case the talk was part of a panel of presentations intended to give the view of “scholars” to the information professionals. In the second it was part of a session looking at the application of web based tools to research and education.
Now, about that filter..
View more presentations from Cameron Neylon.

Abstract (NFAIS meeting): There was …

Blog, Featured »

[10 Aug 2010 | 15 Comments | ]

The online maths community has lit up with excitement as a document, claiming to prove one of the major outstanding theorems in maths has been circulated. In response an online peer review process has swung into action that is very similar to the kind of post-publication peer review that many of us have advocated. Is this a one of, a special case? Or does it point the way towards successfully using the web to find a way of doing peer review effectively and efficiently?

Blog, Featured, Headline »

[8 Jul 2010 | 3 Comments | ]
It’s not information overload, nor is it filter failure: It’s a discovery deficit

The idea that “it’s not information overload, it’s filter failure” combined with the traditional process of filtering scholarly communication by peer review prior to publication seems to be leading towards the idea that we need to build better filters by beefing up the curation of research output before it is published. Here I argue that this is backwards and that the ‘filter failure’ soundbite is maybe unfortunate in the context of scholarly communications. The web won’t reduce the cost of curation, but it has reduced the cost of publication. This means that instead of building filters to prevent stuff getting on the web it is more productive to focus on enhancing discovery. A focus on enabling discovery can both deliver for researchers and provide business models that are more aligned with the way the web works.

Blog, Featured »

[29 Apr 2010 | 14 Comments | ]
In defence of author-pays business models

There has been an awful lot recently written and said about author-pays business models for scholarly publishing and a lot of it has focussed on PLoS ONE. Most recently Kent Andersen has written a piece on Scholarly Kitchen that contains a number of fairly serious misconceptions about the processes of PLoS ONE. This is a shame because I feel this has muddled the much more interesting question that was intended to be the focus of his piece. Nonetheless here I want to give a robust defence of author pays models and of PLoS ONE in particular.

Blog, Featured »

[5 Feb 2010 | 27 Comments | ]
Peer review: What is it good for?

Image by Gideon Burton via Flickr

It hasn’t been a real good week for peer review. In the same week that the Lancet fully retract the original Wakefield MMR article (while keeping the retraction behind a login screen – way to go there on public understanding of science), the main stream media went to town on the report of 14 stem cell scientists writing an open letter making the claim that peer review in that area was being dominated by a small group of people blocking the publication of innovative work. …

Blog »

[7 Sep 2009 | 24 Comments | ]

I have long being sceptical of the costs and value delivered by our traditional methods of peer review. This is really on two fronts, firstly that the costs, where they have been estimated are extremely high, representing a multi-billion dollar subsidy by governments of the scholarly publishing industry. Secondly the value that is delivered through peer review, the critical analysis of claims, informed opinion on the quality of the experiments, is largely lost. At best it is wrapped up in the final version of the paper. At worst it is …

Blog »

[20 Jun 2009 | 24 Comments | ]

I think it is fair to say that even those of us most enamored of post-publication peer review would agree that its effectiveness remains to be demonstrated in a convincing fashion. Broadly speaking there are two reasons for this; the first is the problem of social norms for commenting. As in there aren’t any. I think it was Michael Nielsen who referred to the “Kabuki Dance of scientific discourse”. It is entirely allowed to stab another member of the research community in the back, or indeed the front, but there …

Blog »

[12 Mar 2009 | 12 Comments | ]

This post is both a follow up to last week’s post on the cost’s of peer review and a response to Duncan Hull‘s post of nine or so months ago proposing a game of “Fantasy Science Funding“. The game requires you to describe how you would distribute the funding of the BBSRC if you were a benign (or not so benign) dictator. The post and the discussion should be read bearing in mind my standard disclaimer.
Peer review is in crisis. Anyone who tells you otherwise either has their head …

Blog »

[3 Mar 2009 | 26 Comments | ]

Late last year the Research Information Network held a workshop in London to launch a report, and in many ways more importantly, a detailed economic model of the scholarly publishing industry. The model aims to capture the diversity of the scholarly publishing industry and to isolate costs and approaches to enable the user to ask questions such as “what is the consequence of moving to a 95% author pays model” as well as to simply ask how much money is going in and where it ends up. I’ve been meaning …