Home » Blog, Headline

Beyond the Impact Factor: Building a community for more diverse measurement of research

29 November 2010 9 Comments
An old measuring tape
Image via Wikipedia

I know I’ve been a bit quiet for a few weeks. Mainly I’ve been away for work and having a brief holiday so it is good to be plunging back into things with some good news. I am very happy to report that the Open Society Institute has agreed to fund the proposal that was built up in response to my initial suggestion a month or so ago.

OSI, which many will know as one of the major players in bringing the Open Access movement to its current position, will fund a workshop that will identify both potential areas where the measurement and aggregation of research outputs can be improved as well as barriers to achieving these improvements. This will be immediately followed by a concentrated development workshop (or hackfest) that will aim to deliver prototype examples that show what is possible. The funding also includes further development effort to take one or two of these prototypes and develop them to proof of principle stage, ideally with the aim of deploying these into real working environments where they might be useful.

The workshop structure will be developed by the participants over the 6 weeks leading up to the date itself. I aim to set that date in the next week or so, but the likelihood is early to mid-March. The workshop will be in southern England, with the venue to be again worked out over the next week or so.

There is a lot to pull together here and I will be aiming to contact everyone who has expressed an interest over the next few weeks to start talking about the details. In the meantime I’d like to thank everyone who has contributed to the effort thus far. In particular I’d like to thank Melissa Hagemann and Janet Haven at OSI and Gunner from Aspiration who have been a great help in focusing and optimizing the proposal. Too many people contributed to the proposal itself to name them all (and you can check out the GoogleDoc history if you want to pull apart their precise contributions) but I do want to thank Heather Piwowar and David Shotton in particular for their contributions.

Finally, the success of the proposal, and in particular the community response around it has made me much more confident that some of the dreams we have for using the web to support research are becoming a reality. The details I will leave for another post but what I found fascinating is how far the network of people spread who could be contacted, essentially through a single blog post. I’ve contacted a few people directly but most have become involved through the network of contacts that spread from the original post. The network, and the tools, are effective enough that a community can be built up rapidly around an idea from a much larger and more diffuse collection of people. The challenge of this workshop and the wider project is to see how we can make that aggregated community into a self sustaining conversation that produces useful outputs over the longer term.

It’s a complete co-incidence that Michael Nielsen posted a piece in the past few hours that forms a great document for framing the discussion. I’ll be aiming to write something in response soon but in the meantime follow the top link below.

Enhanced by Zemanta


  • Egon Willighagen said:

    Congrats! I think you got the right people together…

  • Mark Hahnel said:

    Congratulations, thoroughly deserved and another great example of how powerfull collaboration can be. As Egon says, the callibre of those listed on the google doc history says it all.

  • Mr. Gunn said:

    This is great news, Cameron! Let me know the details as the event gets closer.

  • Cactus Acide » L'observatoire du neuromancien » L’observatoire du neuromancien 11/29/2010 said:

    […] Science in the Open » Blog Archive » Beyond the Impact Factor: Building a community for more diver… […]

  • Nikki Rogers said:

    Hi, congratulations on your funding. I am managing the JISC-funded ResearchRevealed project (a 2-year project in its final phase). We are interested in tools to support researchers capture the impact they think their research is having – we have a prototype bookmarklet tool – see . We’d be keen in coming to your workshop and look forward to further details, thanks!

  • Nikki Rogers said:

    Formatting went a little wrong there – try http://researchrevealed.ilrt.bris.ac.uk/?p=69 !

  • Quick Links | A Blog Around The Clock said:

    […] Beyond the Impact Factor: Building a community for more diverse measurement of research […]

  • Cameron Neylon said:

    Hi Nikki and thanks for the comment (and sorry for the delay in replying). This is exactly the kind of thing we need. A bit of lateral thinking to help solve a problem.

  • Quora said:

    Which currently active projects focus on reputation metrics in science?…

    Cameron Neylon has instigated a project ”to connect metrics builders with those who can most effectively use them to change practice”. The title is Beyond the Impact Factor: Linking funder needs to the development of new research metrics http://c