Tag: michael-nielsen

Posted on

Designing for the phase change: Local communities and shared infrastructure

Pink crystal.
Image via Wikipedia

Michael Nielsen‘s talk at Science Online was a real eye opener for many of us who have been advocating for change in research practice. He framed the whole challenge of change as an example of a well known problem, that of collective action. How do societies manage big changes when those changes often represent a disadvantage to many individuals, at least in the short term. We can all see the global advantages of change but individually acting on them doesn’t make sense.

Michael placed this in the context of other changes, that of countries changing which side of the road they drive on, or the development of trade unions, that have been studied in some depth by political economists and similar academic disciplines. The message of these studies is that change usually occurs in two phases. First local communities adopt practice (or at least adopt a view that they want things changed in the case of which side of the road they drive on) and then these communities discover each other and “agglomerate”, or in the language of physical chemistry there are points of nucleation which grow to some critical level and then the whole system undergoes a phase change, crystallising into a new form.

These two phases are driven by different sets of motivations and incentives. At a small scale processes are community driven, people know each other, and those interactions can drive and support local actions, expectations, and peer pressure. At a large scale the incentives need to be different and global. Often top down policy changes (as in the side of the road) play a significant role here, but equally market effects and competition can also fall into place in a way that drives adoption of new tools or changes in behaviour. Think about the way new research techniques get adopted: first they are used by small communities, single labs, with perhaps a slow rate of spread to other groups. For a long time it’s hard for the new approach to get traction, but suddenly at some point either enough people are using it that its just the way things are done, or conversely those who are using it are moving head so fast that everyone else has to pile in just to keep up. It took nearly a decade for PCR for instance to gain widespread acceptance as a technique in molecular biology but when it did it went from being something people were a little unsure of to being the only way to get things done very rapidly.

So what does this tell us about advocating for, or designing for, change. Michael’s main point was that narrow scope is a feature, not a bug, when you are in that first phase. Working with small scale use cases, within communities is the way to get started. Build for those communities and they will become your best advocates, but don’t try to push the rate of growth, let it happen at the right rate (whatever that might be – and I don’t really know how to tell to be honest). But we also need to build in the grounding for the second phase.

The way these changes generally occur is through an accidental process of accretion and agglomeration. The phase change crystallises out around those pockets of new practice. But, to stretch the physical chemistry analogy, doesn’t necessarily crystallise in the form one would design for. But we have an advantage, if we design in advance to enable that crystallisation then we can prepare communities and prepare tooling for when it happens and we can design in the features that will get use closer to the optimum we are looking for.

What does this mean in practice? It means that when we develop tools and approaches it is more important for our community to have standards than it is for there to be an effort on any particular tool or approach. The language we use, that will be adopted by communities we are working with, should be consistent, so that when those communities meet they can communicate. The technical infrastructure we use should be shared, and we need interoperable standards to ensure that those connections can be made. Again, interchange and interoperability are more important than any single effort, any single project.

If we really believe in the value of change then we need to get these things together before we push them too hard into the diverse set of research communities where we want them to take root. We really need to get interoperability, standards, and language sorted out before the hammer of policy comes down and forces us into some sort of local minimum. In fact, it sounds rather like we have a collective action problem of our own. So what are we going to do about that?

Related articles
Enhanced by Zemanta
Posted on

Incentives: Definitely a case of rolling your own

Lakhovsky: The Convesation; oil on panel (Бесе...
Image via Wikipedia

Science Online London ran late last week and into the weekend and I was very pleased to be asked to run a panel, broadly speaking focused on evaluation and incentives. Now I had thought that the panel went pretty well but I’d be fibbing if I said I wasn’t a bit disappointed. Not disappointed with the panel members or what they said. Yes it was a bit more general than I had hoped and there were things that I wished we’d covered but the substance was good from my perspective. My disappointment was with the response from the audience, really on two slightly different points.

The first was the lack of response to what I thought were some of the most exciting things I’ve heard in a long time from major stakeholders. I’ll come back to that later. But a bigger disappointment was that people didn’t seem to connect the dots to their own needs and experiences.

Science Online, both in London and North Carolina forms, has for me always been a meeting where the conversation proceeds at a more sophisticated level than the usual. So I pitched the plan of the session at where I thought the level should be. Yes we needed to talk about the challenges and surface the usual problems, non-traditional research outputs and online outputs in particular don’t get the kind of credit that papers do, institutions struggle to give credit for work that doesn’t fit in a pigeonhole, funders seem to reward only the conventional and traditional, and people outside the ivory tower struggle to get either recognition or funding. These are known challenges, the question is how to tackle them.

The step beyond this is the hard one. It is easy to say that incentives need to change. But incentives don’t drop from heaven. Incentives are created within communities and they become meaningful when they are linked to the interests of stakeholders with resources. So the discussion wasn’t really about impact, or funding, or to show that nothing can be done by amateurs. The discussion was about the needs of institutions and funders and how they can be served by what is being generated by the online community. It was also about the constraints they face in acting. But fundamentally you had major players on the stage saying “this is the kind of thing we need to get the ball rolling”.

Make no mistake, this is tough. Everyone is constrained and resources are tight but at the centre of the discussion were the key pointers to how to cut through the knot. The head of strategy at a major research university stated that universities want to play a more diverse role, want to create more diverse scholarly outputs, and want to engage with the wider community in new ways. That smart institutions will be looking to diversify. The head of evaluation at a major UK funder said that funders really want to know about non-traditional outputs and how they were having a wider impact. That these outputs are amongst the best things they can talk about to government. That they will be crucial to make the case to sustain science funding.

Those statements are amongst the most direct and exciting I have heard in some years of advocacy in this space. The opportunity is there, if you’re willing to put the effort in to communicate and to shape what you are doing to match to match their needs. As Michael Nielsen said in his morning keynote this is a collective action problem. That means finding what unites the needs of those doing with the needs of those with resources. It means compromise, and it means focusing on the achievable, but the point of the discussion was to identify what might be achievable.

So mostly I was disappointed that the excitement I felt wasn’t mirrored in the audience. The discussion about incentives has to move on. Saying that “institutions should do X” or “funders should do Y” gets us nowhere. Understanding what we can do together with funders and institutions and other communities to take the online agenda forward and understanding what the constraints are is where we need to go. The discussion showed that both institutions and funders know that they need what the community of online scientists can do. They don’t know how to go about it, and they don’t even know very much what we are doing, but they want to know. And when they do know they can advise and help and they can bring resources to bear. Maybe not all the resources you would like, and maybe not for all the things you would like, but resources nonetheless.

With a lot of things it is easy to get too immersed in the detail of these issues and to forget that people are looking in from the outside without the same context. I guess the fact that I pulled out what might have seemed to the audience to be just asides as the main message is indicative of that. But I really want to get that message out because I think it  is critical if the community of online scientists wants to be the mainstream. And I think it should be.

The bottom line is that smart funders and smart institutions value what is going on online. They want to support it, they want to be seen to support it, but they’re not always sure how to go about it and how to judge its quality. But they want to know more. That’s where you come in and that’s why the session was relevant. Lars Fischer had it absolutely right: “I think the biggest and most consequential incentive for scientists is (informal) recognition by peers.” You know, we know, who is doing the good stuff and what is valuable. Take that conversation to the funders and the institutions, explain to them what’s good and why, and tell the story of what the value is. Put it in your CV, demand that promotion panels take account of it, whichever side of the table you are on. Show that you make an impact in language that they understand. They want to know. They may not always be able to act – funding is an issue – but they want to and they need your help. In many ways they need our help more than we need theirs. And if that isn’t an incentive then I don’t know what is.

Related articles
Enhanced by Zemanta
Posted on

Michael Nielsen, the credit economy, and open science

No credit cards please.......

Michael Nielsen is a good friend as well as being an inspiration to many of us in the Open Science community. I’ve been privileged to watch and in a small way to contribute to the development of his arguments over the years and I found the distillation of these years of effort into the talk that he recently gave at TEDxWaterloo entirely successful. Here is a widely accesible and entertaining talk that really pins down the arguments, the history, the successes and the failures of recent efforts to open up science practice.

Professional scientific credit is the central issue

I’ve been involved in many discussions around why the potential of opening up research practice hasn’t lead to wider adoption of these approaches. The answer is simple, and as Michael says very clearly in the opening section of the talk, the problem is that innovative approaches to doing science are not going to be adopted while those that use them don’t get conventional scientific credit. I therefore have to admit to being somewhat nonplussed by GrrlScientist’s assessment of the talk that “Dr Nielsen has missed — he certainly has not emphasised — the most obvious reason why the Open Science movement will not work: credit.”

For me, the entire talk is about credit. He frames the discussion of why the Qwiki wasn’t a huge success, compared to the Polymath project, in terms of the production of conventional papers, he discusses the transition from Galileo’s anagrams to the development of the scientific journal in terms of ensuring priority and credit. Finally he explicitly asks the non-scientist members of the audience to do something that even more closely speaks to the issue of credit, to ask their scientist friends and family what they are doing to make their results more widely available. Remember this talk is aimed at a wider audience, the TEDxWaterloo attendees and the larger audience for the video online (nearly 6,000 when I wrote this post). What happens when taxpayers start asking their friends, their family, and their legislative representatives how scientific results are being made available? You’d better believe that this has an affect on the credit economy.

Do we just need the celebrities to back us?

Grrl suggests that the answer to pushing the agenda forward is to enlist Nobelists to drive projects in the same way that Tim Gowers pushed the Polymath project. While I can see the logic and there is certainly value in moral support from successful scientists we already have a lot of this. Sulston, Varmus, Michael and Jon Eisen, and indeed Michael himself just to name a few are already pushing this agenda. But moral support and single projects are not enough. What we need to do is hack the underlying credit economy, provide proper citations for data and software, exploit the obsession with impact factors.

The key to success in my view is a pincer movement. First, showing that more (if not always completely) open approaches can outcompete closed approaches on traditional assessment measures, something demonstrated successfully by Galaxy Zoo, the Alzeimers Disease Neuroimaging Initiative, and the Polymath Projects. Secondly changing assessment policy and culture itself, both explicitly by changing the measures by which researchers are ranked, and implicitly by raising the public expectation that research should be open.

The pendulum is swinging and we’re pushing it just about every which-way we can

I guess what really gets my back up is that Grrl sets off with the statement that “Open Science will never work” but then does on to put her finger on exactly the point where we can push to make it work. Professional and public credit is absolutely at the centre of the challenge. Michael’s talk is part of a concerted, even quite carefully coordinated, campaign to tackle this issue at a wide range of levels. Michael’s tour of his talk, funded by the Open Society Institute seeks to raise awareness. My recent focus on research assessment (and a project also funded by OSI) is tackling the same problem from another angle. It is not entirely a coincidence that I’m writing this in a hotel room in Washington DC and it is not at all accidental that I’m very interested in progress towards widely accepted researcher identifiers. The development of Open Research Computation is a deliberate attempt to build a journal that exploits the nature of journal rankings to make software development more highly valued. 

All of these are part of a push to hack, reconfigure, and re-assess the outputs and outcomes that researchers get credit for and the the outputs and outcomes that are valued by tenure committees and grant panels. And from where I stand we’re making enough progress that Grrl’s argument seems a bit tired and outdated. I’m seeing enough examples of people getting credit and reward for being open and simply doing and enabling better science as a result that I’m confident the pendulum is shifting. Would I advise a young scientist that being open will lead to certain glory? No, it’s far from certain, but you need to distinguish yourself from the crowd one way or another and this is one way to do it. It’s still high risk but show me something in a research career that is low risk and I’ll show something that isn’t worth doing.

What can you do?

If you believe that a move towards more open research practice is a good thing then what can you do to make this happen? Well follow what Michael says, give credit to those who share, explicitly acknowledge the support and ideas you get from others. Ask researchers how they go about ensuring that their research is widely available and above all used. The thing is, in the end changing the credit economy itself isn’t enough, we actually have to change the culture that underlies that economy. This is hard but it is done by embedding the issues and assumptions in the everyday discourse about research. “How useable are your research outputs really?” is the question that gets to the heart of the problem. “How easily can people access, re-use, and improve on your research? And how open are you to getting the benefit of other people’s contribution?” are the questions that I hope will become embedded in the assumptions around how we do research. You can make that happen by asking them.

 

Related articles
Enhanced by Zemanta