The problem of academic credit and the value of diversity in the research community
This is the second in a series of posts (first one here) in which I am trying to process and collect ideas that came out of Scifoo. This post arises out of a discussion I had with Michael Eisen (UC Berkely) and Sean Eddy (HHMI Janelia Farm) at lunch on the Saturday. We had drifted from a discussion of the problem of attribution stacking and citing datasets (and datasets made up of datasets) into the problem of academic credit. I had trotted out the usual spiel about the need for giving credit for data sets and for tool development.
Michael made two interesting points. The first was that he felt people got too much credit for datasets already and that making them more widely citeable would actually devalue the contribution. The example he cited was genome sequences. This is a case where, for historical reasons, the publication of a dataset as a paper in a high ranking journal is considered appropriate.
In a sense I agree with this case. The problem here is that for this specific case it is allowable to push a dataset sized peg into a paper sized hole. This has arguably led to an over valuing of the sequence data itself and an undervaluing of the science it enables. Small molecule crystallography is similar in some regards with the publication of crystal structures in paper form bulking out the publication lists of many scientists. There is a real sense in which having a publication stream for data, making the data itself directly citeable, would lead to a devaluation of these contributions. On the other hand it would lead to a situation where you would cite what you used, rather than the paper in which it was, perhaps peripherally described. I think more broadly that the publication of data will lead to greater efficiency in research generally and more diversity in the streams to which people can contribute.
Michael’s comment on tool development was more telling though. As people at the bottom of the research tree (and I count myself amongst this group) it is easy to say ‘if only I got credit for developing this tool’, or ‘I ought to get more credit for writing my blog’, or anyone of a thousand other things we feel ‘ought to count’. The problem is that there is no such thing as ‘credit’. Hiring decisions and promotion decisions are made on the basis of perceived need. And the primary needs of any academic department are income and prestige. If we believe that people who develop tools should be more highly valued then there is little point in giving them ‘credit’ unless that ‘credit’ will be taken seriously in hiring decisions. We have this almost precisely backwards. If a department wanted tool developers then it would say so, and would look at CVs for evidence of this kind of work. If we believe that tool developers should get more support then we should be saying that at a higher, strategic level, not just trying to get it added as a standard section in academic CVs.
More widely there is a question as to why we might think that blogs, or public lectures, or code development, or more open sharing of protocols are something for which people should be given credit. There is often a case to be made for the contribution of a specific person in a non-traditional medium, but that doesn’t mean that every blog written by a scientists is a valuable contribution. In my view it isn’t the medium that is important, but the diversity of media and the concomitant diversity of contributions that they enable. In arguing for these contributions being significant what we are actually arguing for is diversity in the academic community.
So is diversity a good thing? The tightening and concentration of funding has, in my view, led to a decrease in diversity, both geographical and social, in the academy. In particular there is a tendency to large groups clustered together in major institutions, generally led by very smart people. There is a strong argument that these groups can be more productive, more effective, and crucially offer better value for money. Scifoo is a place where those of us who are less successful come face to face with the fact that there are many people a lot smarter than us and that these people are probably more successful for a reason. And you have to question whether your own small contribution with a small research group is worth the taxpayer’s money. In my view this is something you should question anyway as an academic researcher – there is far too much comfortable complacency and sense of entitlement, but that’s a story for another post.
So the question is; do I make a valid contribution? And does that provide value for money? And again for me Scifoo provides something of an answer. I don’t think I spoke to any person over the weekend without at least giving them something new to think about, a slightly different view on a situation, or just an introduction to something that hadn’t heard of before. These contributions were in very narrow areas, ones small enough for me to be expert, but my background and experience provided a different view. What does this mean for me? Probably that I should focus more on what makes my background and experience unique – that I should build out from that in the directions most likely to provide a complementary view.
But what does it mean more generally? I think that it means that a diverse set of experiences, contributions, and abilities will improve the quality of the research effort. At one session of Scifoo, on how to support ground breaking science, I made the tongue in cheek comment that I thought we needed more incremental science, more filling in of tables, of laying the foundations properly. The more I think about this the more I think it is important. If we don’t have proper foundations, filled out with good data and thought through in detail, then there are real risks in building new skyscrapers. Diversity adds reinforcement by providing better tools, better datasets, and different views from which to examine the current state of opinion and knowledge. There is an obvious tension between delivering radical new technologies and knowledge and the incremental process of filling in, backing up, and checking over the details. But too often the discussion is purely about how to achieve the first, with no attention given to the importance of the second. This is about balance not absolutes.
So to come back around to the original point, the value of different forms of contribution is not due to the fact that they are non-traditional or because of the medium per se, it is because they are different. If we value diversity at hiring committees, and I think we should, then looking at a diverse set of contributions, and the contribution that a given person is likely to make in the future based on their CVs, we can assess more effectively how they will differ from the people we already have. The tendency of ‘the academy’ to hire people in its own image is well established. No monoculture can ever be healthy; certainly not in a rapidly changing environment. So diversity is something we should value for its own sake, something we should try to encourage, and something that we should search CVs for evidence of. Then the credit for these activities will flow of its own accord.
You are right that the concept of credit is meaningless unless accepted by the academic committees. At SciFoo there was a lot of discussion about microcredits and new ways of tracking contributions. The problem is that many of those metrics are not valued now anyway. There is nothing stopping me from putting every paper review and blog comment on my annual report but I don’t do it because it won’t matter. I do however list the experiments we recorded on our wiki.
These are just communication tools. They should be used to facilitate participation, not bean counting. People don’t feel the need to document every phone call or email they make as a scientific contribution. It should be the same with social software.
You are right that the concept of credit is meaningless unless accepted by the academic committees. At SciFoo there was a lot of discussion about microcredits and new ways of tracking contributions. The problem is that many of those metrics are not valued now anyway. There is nothing stopping me from putting every paper review and blog comment on my annual report but I don’t do it because it won’t matter. I do however list the experiments we recorded on our wiki.
These are just communication tools. They should be used to facilitate participation, not bean counting. People don’t feel the need to document every phone call or email they make as a scientific contribution. It should be the same with social software.
I agree with you, Always has been controversies about the credit given to researchers. in open or closed science
http://imechanica.org/node/3350
http://imechanica.org/node/3447
anyway, without having some sort of credit for open activities. achivement would be little.
Why are people reluctant to join in open review? Nature 447, 1052 (28 June 2007).
I agree with you, Always has been controversies about the credit given to researchers. in open or closed science
http://imechanica.org/node/3350
http://imechanica.org/node/3447
anyway, without having some sort of credit for open activities. achivement would be little.
Why are people reluctant to join in open review? Nature 447, 1052 (28 June 2007).
Great post. I am trying to wrap my head around how academia should work in a world where large datasets and especially people who support research by developing a variety of tools are a reality.
Part of the solution, at least for the tool developers is to make staff positions, that pay competitive salaries a reality. Take a good software engineer. Unless they absolutely love science, the salary differential between doing software development in academia and industry is too much, and in the former they don’t get appreciated that much.
Personally I find the “we need credit to blog” somewhat off base. Blogging, unless used as a direct publishing channel should be looked at in a larger context, same for other media forms. The key is getting the community to accept blogging, etc as legitimate forms of communication. The problem is that we have this broken system and everyone is obsessed on trying to figure out how they can do something that gives them a leg up in the same.
Great post. I am trying to wrap my head around how academia should work in a world where large datasets and especially people who support research by developing a variety of tools are a reality.
Part of the solution, at least for the tool developers is to make staff positions, that pay competitive salaries a reality. Take a good software engineer. Unless they absolutely love science, the salary differential between doing software development in academia and industry is too much, and in the former they don’t get appreciated that much.
Personally I find the “we need credit to blog” somewhat off base. Blogging, unless used as a direct publishing channel should be looked at in a larger context, same for other media forms. The key is getting the community to accept blogging, etc as legitimate forms of communication. The problem is that we have this broken system and everyone is obsessed on trying to figure out how they can do something that gives them a leg up in the same.
Deepak, I think your last paragraph is my point at core (as is Jean-Claude’s). The perception of what is success and how you get there is driving behaviour towards a very small set of stereotypes, and I think this is dangerous. I guess my point is that diversity of channels should be valued because it is a sign of diversity of viewpoints and this is valuable in its own right. How you actually convert that to a rational management and hiring strategy I am less sure :)
My gut feeling is that the central problem is less about salaries (although this is important) and more about permanence. People want security and the only people who get any security currently are tenured academics. Everyone else is on contracts. If there was a proper permanent status for a wider variety of people this would solve a lot of problems. Of course in the UK such a thing would lead to your department being downgraded in the RAE so no-one would do it.
Deepak, I think your last paragraph is my point at core (as is Jean-Claude’s). The perception of what is success and how you get there is driving behaviour towards a very small set of stereotypes, and I think this is dangerous. I guess my point is that diversity of channels should be valued because it is a sign of diversity of viewpoints and this is valuable in its own right. How you actually convert that to a rational management and hiring strategy I am less sure :)
My gut feeling is that the central problem is less about salaries (although this is important) and more about permanence. People want security and the only people who get any security currently are tenured academics. Everyone else is on contracts. If there was a proper permanent status for a wider variety of people this would solve a lot of problems. Of course in the UK such a thing would lead to your department being downgraded in the RAE so no-one would do it.
Cameron,
Completely agree. We should choose channels that we feel are the most effective.
I suppose having worked in industry in the US all these years, I subscribe to the “there is no such thing as security” mantra. Do tools developers want security? Most want challenges, since they are not in the race to publish as such. It’s just that they are generally treated as the periphery, rather than a core part of the process.
Cameron,
Completely agree. We should choose channels that we feel are the most effective.
I suppose having worked in industry in the US all these years, I subscribe to the “there is no such thing as security” mantra. Do tools developers want security? Most want challenges, since they are not in the race to publish as such. It’s just that they are generally treated as the periphery, rather than a core part of the process.
That might well be a cultural/geographical thing. Here people are more serious about the need for security. But then the pay is usually less in real terms. One of the things that continues to amaze me about the US is the number of people who clearly have the money to just get on and do something that they believe in (even when it doesn’t pay). That seems (to me at least) much rarer in the UK and Europe. Is tenure in the US about security or status?
But yes, challenge and being valued are also crucial. The executors can be just as important (or more important) than the person with the ideas. Or am I just saying that because I realise there are lots of people out there with better ideas than me? :) (Actually ideas was never my problem – it was getting them to work but that’s another story)
That might well be a cultural/geographical thing. Here people are more serious about the need for security. But then the pay is usually less in real terms. One of the things that continues to amaze me about the US is the number of people who clearly have the money to just get on and do something that they believe in (even when it doesn’t pay). That seems (to me at least) much rarer in the UK and Europe. Is tenure in the US about security or status?
But yes, challenge and being valued are also crucial. The executors can be just as important (or more important) than the person with the ideas. Or am I just saying that because I realise there are lots of people out there with better ideas than me? :) (Actually ideas was never my problem – it was getting them to work but that’s another story)
Definitely is. 3 years is about the max I have worked anywhere and no one has ever asked me why I change jobs so often. Software industry is more like that too.
What I like about the US is not that you have the money, often you don’t, but it’s alright to try and screw up. On the academic front, I am not sure that it’s quite the same culture though.
Re: Ideas, I am with you. Lots of ideas, but execution .. no comment :)
Definitely is. 3 years is about the max I have worked anywhere and no one has ever asked me why I change jobs so often. Software industry is more like that too.
What I like about the US is not that you have the money, often you don’t, but it’s alright to try and screw up. On the academic front, I am not sure that it’s quite the same culture though.
Re: Ideas, I am with you. Lots of ideas, but execution .. no comment :)
License
I am also found at...
Tags
Recent posts
Recent Posts
Most Commented