Home » Blog, Featured

A collaborative proposal on research metrics

24 September 2010 6 Comments
Measuring time
Image by aussiegall via Flickr

tldr: Proposed project to connect metrics builders with those who can most effectively use them to change practice. Interested? Get involved! Proposal doc is here and free to edit.

When we talk about open research practice, more efficient research communication, wider diversity of publication we always come up against the same problem. What’s in it for the jobbing scientist? This is so prevalent that it has been reformulated as “Singh’s Law” (by analogy with Godwin’s law) that any discussion of research practice will inevitably end when someone brings up career advancement or tenure. The question is what do we actually do about this?

The obvious answer is to make these things matter. Research funders have the most power here in that they have the power to influence behaviour through how they distribute resources. If the funder says something is important then the research community will jump to it. The problem of course it that in practice funders have to take their community with them. Radical and rapid change is not usually possible. A step in the right direction would be to provide funders and researchers with effective means of measuring and comparing themselves and their outputs. In particular means of measuring performance in previously funded activities.

There are many current policy initiatives on trying to make these kinds of judgements. There are many technical groups building and discussing different types of metrics. Recently there have also been calls to ensure that the data that underlies these metrics is made available. But there is relatively little connection between these activities. There is an opportunity to connect technical expertise and data with the needs of funders, researchers, and perhaps even the mainstream media and government.

An opportunity has arisen for some funding to support a project here. My proposal is to bring a relevant group of stakeholders together; funders, technologists, scientists, adminstrators, media, publishers, and aggregators, to identify needs and then to actually build some things. Essentially the idea is a BarCamp style day and a bit meeting followed by a two day hackfest. Following on from this the project would fund some full time effort to take the most promising ideas forward.

I’m looking for interested parties. This will be somewhat UK centric just because of logistics and funding but the suggestion has already been made that following up with a similar North American or European project could be interesting. The proposal is available to view and edit as a GoogleDoc. Feel free to add your name, contact me directly, or suggest the names of others (probably better to me directly). I have a long list of people to contact directly as well but feel free to save me the effort.

Ed. Note: This proposal started as a question on Friendfeed where I’ve already got a lot of help and ideas. Hopefully soon I will write another post about collaborative and crowdsourced grant writing and how it has changed since the last time I tried this some years back.

Enhanced by Zemanta


  • M J Weller said:

    Hi Cameron
    definitely count me in. I’m working on digital scholarship at the Open University and ‘making this stuff count’ is the biggest problem. I’m going to be looking at some form of metrics this year so would definitely be interested.

  • Anonymous said:

    Happy to help hack stuff together if i can…

  • Cameron Neylon said:

    Very happy to have you both on board. Starting to get some traction now which is positive. Of course. I still need to actually get the money as well…

  • Quick Links | A Blog Around The Clock said:

    […] A collaborative proposal on research metrics […]

  • Sergey Parinov said:

    It is interesitng that similar ideas cross one’s mind at the same time in different places. E.g. we in Russia are implementing, I think, similar approach. See an article in English on this topic “The electronic library: using technology to measure and support Open Science” at http://socionet.ru/publication.xml?h=repec:rus:mqijxk:25

    What do you think about our views on research metrics?

  • Science in the Open » Blog Archive » Metrics and Money said:

    […] connecting funders, technologists, data holders, and yes, even publishers, is at the core of the proposal I’m working with at the moment. We need funders to want to have access to data and to want to reward performance on those […]