This post is both a follow up to last week’s post on the cost’s of peer review and a response to Duncan Hull‘s post of nine or so months ago proposing a game of “Fantasy Science Funding“. The game requires you to describe how you would distribute the funding of the BBSRC if you were a benign (or not so benign) dictator. The post and the discussion should be read bearing in mind my standard disclaimer.
Peer review is in crisis. Anyone who tells you otherwise either has their head in the sand or is trying to sell you something. Volumes are increasing, quality of review is decreasing. The willingness of scientists to take on refereeing is increasingly the major problem for those who commission it. This is a problem for peer reviewed publication but the problems for the reviewing of funding applications are far worse.
For grant review, the problems that are already evident in scholarly publishing, fundamentally the increasing volume, are exacerbated by the fact that success rates for grans are falling and that successful grants are increasingly in the hands of a smaller number of people in a smaller number of places. Regardless of whether you agree with this process of concentrating grant funding this creates a very significant perception problem. If the perception of your referees is that they have no chance of getting funding, why on earth should they referee
Is this really happening? Well in the UK chemistry community last year there was an outcry when two EPSRC grant rounds in a row had success rates of 10% or lower. Bear in mind this was the success rate of grants that made it to panel, i.e. it is an upper bound, assuming there weren’t any grants removed at an earlier stage. As you can imagine there was significant hand wringing and a lot of jumping up and down but what struck me was two statements I heard made. The first, was from someone who had sat on one of the panels, was that it “raises the question of whether it is worth our time to attend panel meetings”. The second was the suggestion that the chemistry community could threaten to unilaterally withdraw from EPSRC peer review. These sentiments are now being repeated on UK mailing lists in response to EPSRC’s most recent changes to grant submission guidelines. Whether serious or not, credible or not, this shows that the compact of community contribution to the review process is perilously close to breaking down.
The research council response to this is to attempt to reduce the number of grant proposals, generally by threatening to block those who have a record of serial rejection. This will fail. With success rates as low as they are, and with successful grants concentrated in the hands of the few, most academics are serial failures. The only way departments can increase income is by increasing the volume and quality of grant applications. With little effective control over quality the focus will necessarily be on increasing volume. The only way research councils will control this is either by making applications a direct cost to departments, or by reducing the need of academics to apply.
The cost of refereeing is enormous and largely hidden. But it pales into insignificance compared to the cost of applying for grants. Low success rates make the application process an immense waste of departmental resources. The approximate average cost of running a UK academic for a year is £100,000. If you assume that each academic writes one grant per year and that this takes around two weeks full time work that amounts to ~£4k per academic per year. If there are 100,000 academics in the UK this is £400M, which with a 20% success rate means that £320M is lost in the UK each year. Let’s say that £100M is a reasonable ballpark figure.
In more direct terms this means that academics who are selected for their ability to do research, are being taken away from what they are good at to play a game which they will on average lose four times out of five. It would be a much more effective use of government funding to have those people actually doing research.
So this is a game of Fantasy Funding, how would I spend the money? Well, rather than discuss my biases about what science is important, which are probably not very interesting, it is perhaps more useful to think about how the system might be changed to reduce these wastages. And there is a simple, if somewhat radical, way of doing this.
Cut the budget in two and distribute half of it directly to academics on a pro-rata basis.
By letting researcher’s focus on getting on with research you will reduce their need for funding and reduce the burden. By setting the bar naturally higher for funding research you still maintain the perception that everyone is in with a chance and reduce the risk of referee drop out due to dis-enchantment with the process. More importantly you enable innovative research by allowing it to keep ticking over and in particular you enable a new type of peer review.
If you look at the amounts of money involved, say a few hundred million pounds for BBSRC, and divide that up amongst all bioscience academics, you end up with figures of a £5-20K per academic per year. Not enough to hire a postdoc, just about enough to run a PhD student (at least at UK rates). But what if you put that together with the money from a few other academics? If you can convince your peers that you have an interesting and fun idea then they can pool funds together. Or perhaps share a technician between two groups so that you don’t lose the entire group memory every time a student leaves? Effective collaboration will lead to a win on all sides.
If these arguments sound familiar it is because they are not so different to the notion of 20% time, best known as a Google policy of having all staff spend some time on personal projects. By supporting low level innovation and enabling small scale judging of ideas and pooling of resources it is possible to enable bottom up innovation of precisely the kind that is stifled by top down peer review.
No doubt there would be many unintended consequences, and probably a lot of wastage, but in amongst that I wouldn’t bet against the occassional brilliant innovation which is virtually impossible in the current climate.
What is clear is that doing nothing is not an option. Look at that EPSRC statement again. People with a long term success rate below 25% will be blocked…I just checked my success rate over the past ten years (about 15% by numbers of grants, 70% by value but that is dominated by one large grant). The current success rate at chemistry panel is around 15%. And that is skewed towards a limited number of people and places.
The system of peer review relies absolutely on the communities agreement to contribute and some level of faith in the outcome. It relies absolutely on trust. That trust is perilously close to a breakdown.
