One of the things we want the Open Research Computation journal to do is bring more of the transparency and open critique that characterises the best Open Source Software development processes into the scholarly peer review process. But you can talk about changing the way peer review works and you can actively do something about. Michael Barton and Hazel Barton have taken matters into their own hands and thrown the doors completely open. They have submitted a paper to ORC and in parallel asked the community on the BioStar site how the paper and software could be improved.

So my previous post on peer review hit a nerve. Actually all of my posts on peer review hit a nerve and create massive traffic spikes and I’m still really unsure why. The strength of feeling around peer review seems out of all proportion to both its importance and indeed the extent to which people understand how it works in practice across different disciplines. Nonetheless it is an important and serious issue and one that deserves serious consideration, both blue skies thinking and applied as it were. And it is the latter I will try to do here.

I’ve been interested for some time in capturing information and the context in which that information is created in the lab. The question of how to build an efficient and useable laboratory recording system is fundamentally one of how much information is necessary to record and how much of that can be recorded while bothering the researcher themselves as little as possible. The problem with sophisticated systems that can catch everything is that they break. The problem with simple systems is that they don’t provide enough structure to be useful. But a little structure with a simple framework, like twitter, might provide a route to getting a lot of useful information easy recorded for a lab record.

I’ve been meaning for a while to write something about peer review, pre and post publication, and the somewhat bizarre attachment of research community to the traditional approaches. A news article in Nature tho, in which I am quoted seems to have really struck a nerve for many people. The context in which the quote is presented doesn’t really capture what I meant but I stand by the statement in isolation. I think there are two important things to tease out here, firstly a critical analysis of the problems and merits of peer review, and secondly a close look at how it could be improved, modified, or replaced. I think these merit separate posts so I’ll start here with the problems in our traditional approach.

I was honoured to talk at the symposium to celebrate Peter Murray-Rusts’ work. I didn’t want to give the usual kind of talk to this audience. I wanted to focus on what I think are the big risks and opportunities for the research community and why I believe that a focus on maximising research impact might be a way to bring the community together in a positive way. However I made my own point by inadvertently putting up a permissions slide that prohibited livestreaming, live blogging, and recording. By using a restrictive licence I very effectively reduced the potential impact of my talk about impact. The message is pretty clear. If you want to make a difference, use an open licence and give people permission to re-use your work. If you want to make no impact at all then restrictive licences are a great way to achieve that.

Nature Publishing Group yesterday announced a new venture, very closely modelled on the success of PLoS ONE, titled Scientific Reports. Others have started to cover the details and some implications so I won’t do that here. I think there are three big issues here. What does this tell us about the state of Open Access? What are the risks and possibilities for NPG? And why oh why does NPG keep insisting on a non-commercial licence? I think those merit separate posts so here I’m just going to deal with the big issue. And I think this is really big.

Image by Aníbal Pées Labory via Flickr

Long term readers of this blog will know that I occasionally write an incomprehensible post that no-one understands about the nature of time on the web. This is my latest attempt to sort my thinking out on the issue. I don’t hold out that much hope but it seemed appropriate for the New Year…
2010 was the year that real time came to the mainstream web. From the new Twitter interface to live updates, the flash crash, any number of other developments and stories focussed …

Last Friday I spoke at the STM Innovation Seminar in London, taking in general terms the theme I’ve been developing recently of focussing on enabling user discovery rather than providing central filtering, enabling people to act as their own gatekeeper rather than publishers taking that role on themselves. The example I used, the JACS hydride oxidation paper, wasn’t such a good example because, as a retraction, there is more detailed information available in the published retraction. Had it come a few days earlier the arsenic microbes paper, and subsequent detailed critique might well have made a better example. But both of these examples seem to be pointing towards a world in which post publication peer review is not just happening, but expected. How can publishers work to make the best of this new information and treat papers as the beginning of a story rather than its end?

I recently made the most difficult decision I’ve had to take thus far as a journal editor. That decision was ultimately to accept the paper; that probably doesn’t sound like a difficult decision until I explain that I made this decision despite a referee saying I should reject the paper with no opportunity for resubmission not once, but twice. One of the real problems I have with traditional pre-publication peer review is the way it takes a very nuanced problem around a work which has many different parts and demands that you take a hard yes/no decision.

David Crotty, over at Scholarly Kitchen as an interesting piece on metrics, arguing that many of these have not been thought through because they don’t provide concrete motivation to researchers to care about them. Really he’s focused mainly on exchange mechanisms, means of persuading people that doing high quality review is worth their while by giving them something in exchange, but the argument extends to all sorts of metrics. Why would you care about any given measure if achieving on it doesn’t translate into more resources, time, or glory? You might expect me to disagree with a lot of this but for the most part I don’t.