Home » Blog, Featured

Peer review: What is it good for?

5 February 2010 123 Comments
Peer Review Monster
Image by Gideon Burton via Flickr

It hasn’t been a real good week for peer review. In the same week that the Lancet fully retract the original Wakefield MMR article (while keeping the retraction behind a login screen – way to go there on public understanding of science), the main stream media went to town on the report of 14 stem cell scientists writing an open letter making the claim that peer review in that area was being dominated by a small group of people blocking the publication of innovative work. I don’t have the information to actually comment on the substance of either issue but I do want to reflect on what this tells us about the state of peer review.

There remains much reverence of the traditional process of peer review. I may be over interpreting the tenor of Andrew Morrison’s editorial in BioEssays but it seems to me that he is saying, as many others have over the years “if we could just have the rigour of traditional peer review with the ease of publication of the web then all our problems would be solved”.  Scientists worship at the altar of peer review, and I use that metaphor deliberately because it is rarely if ever questioned. Somehow the process of peer review is supposed to sprinkle some sort of magical dust over a text which makes it “scientific” or “worthy”, yet while we quibble over details of managing the process, or complain that we don’t get paid for it, rarely is the fundamental basis on which we decide whether science is formally published examined in detail.

There is a good reason for this. THE EMPEROR HAS NO CLOTHES! [sorry, had to get that off my chest]. The evidence that peer review as traditionally practiced is of any value at all is equivocal at best (Science 214, 881; 1981, J Clinical Epidemiology 50, 1189; 1998, Brain 123, 1954; 2000, Learned Publishing 22, 117; 2009). It’s not even really negative. That would at least be useful. There are a few studies that suggest peer review is somewhat better than throwing a dice and a bunch that say it is much the same. It is at its best at dealing with narrow technical questions, and at its worst at determining “importance” is perhaps the best we might say. Which for anyone who has tried to get published in a top journal or written a grant proposal ought to be deeply troubling. Professional editorial decisions may in fact be more reliable, something that Philip Campbell hints at in his response to questions about the open letter [BBC article]:

Our editors [...] have always used their own judgement in what we publish. We have not infrequently overruled two or even three sceptical referees and published a paper.

But there is perhaps an even more important procedural issue around peer review. Whatever value it might have we largely throw away. Few journals make referee’s reports available, virtually none track the changes made in response to referee’s comments enabling a reader to make their own judgement as to whether a paper was improved or made worse. Referees get no public credit for good work, and no public opprobrium for poor or even malicious work. And in most cases a paper rejected from one journal starts completely afresh when submitted to a new journal, the work of the previous referees simply thrown out of the window.

Much of the commentary around the open letter has suggested that the peer review process should be made public. But only for published papers. This goes nowhere near far enough. One of the key points where we lose value is in the transfer from one journal to another. The authors lose out because they’ve lost their priority date (in the worse case giving the malicious referees the chance to get their paper in first). The referees miss out because their work is rendered worthless. Even the journals are losing an opportunity to demonstrate the high standards they apply in terms of quality and rigor – and indeed the high expectations they have of their referees.

We never ask what the cost of not publishing a paper is or what the cost of delaying publication could be. Eric Weinstein provides the most sophisticated view of this that I have come across and I recommend watching his talk at Science in the 21st Century from a few years back. There is a direct cost to rejecting papers, both in the time of referees and the time of editors, as well as the time required for authors to reformat and resubmit. But the bigger problem is the opportunity cost – how much that might have been useful, or even important, is never published? And how much is research held back by delays in publication? How many follow up studies not done, how many leads not followed up, and perhaps most importantly how many projects not refunded, or only funded once the carefully built up expertise in the form of research workers is lost?

Rejecting a paper is like gambling in a game where you can only win. There are no real downside risks for either editors or referees in rejecting papers. There are downsides, as described above, and those carry real costs, but those are never borne by the people who make or contribute to the decision. Its as though it were a futures market where you can only lose if you go long, never if you go short on a stock. In Eric’s terminology those costs need to be carried, we need to require that referees and editors who “go short” on a paper or grant are required to unwind their position if they get it wrong. This is the only way we can price in the downside risks into the process. If we want open peer review, indeed if we want peer review in its traditional form, along with the caveats, costs and problems, then the most important advance would be to have it for unpublished papers.

Journals need to acknowledge the papers they’ve rejected, along with dates of submission. Ideally all referees reports should be made public, or at least re-usable by the authors. If full publication, of either the submitted form of the paper or the referees report is not acceptable then journals could publish a hash of the submitted document and reports against a local key enabling the authors to demonstrate submission date and the provenance of referees reports as they take them to another journal.

In my view referees need to be held accountable for the quality of their work. If we value this work we should also value and publicly laud good examples. And conversely poor work should be criticised. Any scientist has received reviews that are, if not malicious, then incompetent. And even if we struggle to admit it to others we can usually tell the difference between critical, but constructive (if sometimes brutal), and nonsense. Most of us would even admit that we don’t always do as good a job as we would like. After all, why should we work hard at it? No credit, no consequences, why would you bother? It might be argued that if you put poor work in you can’t expect good work back out when your own papers and grants get refereed. This again may be true, but only in the long run, and only if there are active and public pressures to raise quality. None of which I have seen.

Traditional peer review is hideously expensive. And currently there is little or no pressure on its contributors or managers to provide good value for money. It is also unsustainable at its current level. My solution to this is to radically cut the number of peer reviewed papers probably by 90-95% leaving the rest to be published as either pure data or pre-prints. But the whole industry is addicted to traditional peer reviewed publications, from the funders who can’t quite figure out how else to measure research outputs, to the researchers and their institutions who need them for promotion, to the publishers (both OA and toll access) and metrics providers who both feed the addiction and feed off it.

So that leaves those who hold the purse strings, the funders, with a responsibility to pursue a value for money agenda. A good place to start would be a serious critical analysis of the costs and benefits of peer review.

Addition after the fact: Pointed out in the comments that there are other posts/papers I should have referred to where people have raised similar ideas and issues. In particular Martin Fenner’s post at Nature Network. The comments are particularly good as an expert analysis of the usefulness of the kind of “value for money” critique I have made. Also a paper in the Arxiv from Stefano Allesina. Feel free to mention others and I will add them here.

Reblog this post [with Zemanta]

  • http://friendfeed.com/danielmietchen Daniel Mietchen

    @ Cameron, authors cope by less frequently submitting low-quality manuscripts, referees by providing careful (and often signed) reviews. To get a feeling on how it works, just take a look at any recently published discussion or final paper. Mine are at http://www.biogeosciences.net/title_and_author_search.html?x=0&y=0&author=mietchen .

    This comment was originally posted on FriendFeed

  • http://friendfeed.com/danielmietchen Daniel Mietchen

    After they have switched from CC-BY-NC-SA to CC-BY some years ago, I see only one problem with their current system: They do not have a journal with a scope extending significantly beyond the geosciences (fair enough, given that they are a geoscientific society), which is a pity for work that does not fit that scope, since there is no real alternative out there which does the review in public (even PLoS ONE does not generally publish the review correspondence). Rejections on scope grounds, by the way, come before the discussion stage of a paper and are thus not recorded in public – I hope that this is going to change too.

    This comment was originally posted on FriendFeed

  • http://friendfeed.com/danielmietchen Daniel Mietchen

    After they have switched from CC-BY-NC-SA to CC-BY some years ago, I see only one problem with their current system: They do not have a journal with a scope extending significantly beyond the geosciences (fair enough, given that they are a geoscientific society), which is a pity for work that does not fit that scope, since there is no real alternative out there (even PLoS ONE does not generally publish the review correspondence). Rejections on scope grounds, by the way, come before the discussion stage of a paper and are thus not recorded in public – I hope that this is going to change too.

    This comment was originally posted on FriendFeed

  • http://friendfeed.com/danielmietchen Daniel Mietchen

    Another interesting aspect of their publishing – not related to peer review – is that they use http://sref.org/ instead of http://doi.org/ .

    This comment was originally posted on FriendFeed

  • http://friendfeed.com/cameronneylon Cameron Neylon

    Interesting. Ulrisch Proschl has left some more detailed descriptions about the EGU approach at the blog post as well.

    This comment was originally posted on FriendFeed

  • http://friendfeed.com/danielmietchen Daniel Mietchen

    Another one on the "peer review week" theme: http://dx.doi.org/10.1007/s00114-010-0652-4 .

    This comment was originally posted on FriendFeed

  • http://www.mpch-mainz.mpg.de/~poeschl/ Uli Pöschl

    Dear Cameron and All:

    following up on a suggestion of Daniel Mietchen I encountered your ongoing discussion, which I find very interesting.

    I agree with many of the arguments put forward, and I would like to draw your attention to a relatively new form of scientific publishing and quality assurance that solves or reduces most of the problems you addressed: interactive open access publishing and peer review as practiced by the journal Atmospheric Chemistry and Physics (ACP, http://www.atmos-chem-phys.net) and a rapidly growing number of sister journals of the European Geosciences Union (EGU, http://www.egu.eu).

    Please find attached the abstract of a recent article explaining the concept, achievements and perspectives of interactive publishing, which effectively resolves the dilemma between free speech, rapid communication and thorough quality assurance as required in the scientific discourse. For more information, please visit the web pages of ACP and EGU (all freely available through open access and creative commons licensing):




    With best regards,
    Uli Pöschl

    Interactive Open Access Publishing and Peer Review: The Effectiveness and Perspectives of Transparency and Self-Regulation in Scientific Communication and Evaluation

    Ulrich Pöschl
    Max Planck Institute for Chemistry, Mainz, Germany, u.poschl@mpic.de
    Manuscript version of 26 October 2009, Submitted to LIBER Quarterly


    The traditional forms of scientific publishing and peer review do not live up to the demands of efficient communication and quality assurance in today’s highly diverse and rapidly evolving world of science. They need to be advanced and complemented by interactive and transparent forms of review, publication, and discussion that are open to the scientific community and to the public.

    The advantages of open access, public peer review and interactive discussion can be efficiently and flexibly combined with the strengths of traditional publishing and peer review. Since 2001 the benefits and viability of this approach are demonstrated by the highly successful interactive open access journal Atmospheric Chemistry and Physics (ACP, http://www.atmos-chem-phys.net) and a growing number of sister journals of the European Geosciences Union (EGU, http://www.egu.eu) and Copernicus Publications (http://www.copernicus.org).

    These journals are practicing a two-stage process of publication and peer review combined with interactive public discussion, which effectively resolves the dilemma between rapid scientific exchange and thorough quality assurance. The same or similar concepts have also been adopted in other disciplines, including the life sciences and economics. Note, however, that alternative approaches where interactive commenting and public discussion are not fully integrated with formal peer review by designated referees tend to be less successful. So far, the interactive open access peer review of ACP is arguably the most successful alternative to the closed peer review of traditional scientific journals.

    The principles, key features and results of interactive open access publishing and peer review are presented and discussed in this manuscript. The achievements and statistics of ACP and its sister journals clearly prove both the scientific benefits and the financial sustainability of open access. Future perspectives and a vision of improved communication and evaluation in the global information commons are outlined with regard to the principles of critical rationalism and open societies.

  • Ulrich Poschl

    P.S.: ACP and its EGU interactive open access sister journals are currently publishing about 2000 papers with a turnover of 2 MEUR per year, which the authors or their institutions are ready to cover. Moreover, they are top ranked in the citation statistics of their field (see ISI-SCIE, SCOPUS, Google Scholar, etc.). In other words, interactive open access publishing and peer review are already well established and continue to spread throughout the geosciences and beyond (see links of preceding post).

  • Ulrich Poschl

    Cameron, your proposal is very close to what the interactive open access journal Atmospheric Chemistry and Physics (ACP, http://www.atmos-chem-phys.net) and a growing number of sister journals of the European Geosciences Union (EGU, http://www.egu.eu) are practicing since 2001 with great success and at fairly large scale. The results are high and steeply increasing rates of submission and publication (currently 1000 papers per year for ACP), top quality and visibility (impact factors) at low rejection rates (only 10% as opposed to 50% in traditional journals with lower impact factors), and financial sustainability at low cost (approx. 1000 EUR per paper). I am confident that interactive open access publishing is suitable for most if not all scientific disciplines, and I can only recommend this approach to all scientific publishers.

  • Randall

    Great lesson Jay. verse 6 form above says: 6 “Therefore, say to the Israelites: ‘I am the LORD, and I will bring you out from under the yoke of the Egyptians. I will free you from being slaves to them, and I will redeem you with an outstretched arm and with mighty acts of judgment. 7 I will take you as my own people, and I will be your God. Then you will know that I am the LORD your God, who brought you out from under the yoke of the Egyptians. 8 And I will bring you to the land I swore with uplifted hand to give to Abraham, to Isaac and to Jacob. I will give it to you as a possession. I am the LORD.’”
    You put God’s statements that “I will bring you out… I will redeem you… I will take you as my own people, and I will be your God in bold print. May I suggest that those that emphasize the sovereignty of God do exactly the same thing. They place the emphasis on what God does and regard it as primary and regard what man does as secondary and the result of what God does. Someone has said that even Pelagians like Calvinism until someone points it out and then they back off of it. Thus they take a little of the credit and glory that belongs to God and appropriate it to themselves for making the right choice and deciding to give their lives to Jesus. Granted, they don’t see it that way, but it is logical to see it like that when they say the difference between them and a lost person is their free will choice. Calvinists do not deny man making a choice and being responsible for the choice he makes. They just believe the credit for their choice belongs to God rather than themselves.

    A second point has to do with Jesus drinking the cup of wrath. NT Wright seems to be the darling of many progressives in the CofC as well as the emerging church movement. I can’t get my head wrapped around his teaching that Jesus is my substitute but NOT a penal substitute. I wish someone could explain it to me better than what I’ve heard. In your post above Jesus certainly drinks the cup of wrath that is intended for, and deserved by sinners. This seems altogether consistent with the Calvinistic doctrine of penal substitution atonement. That is, he paid the penalty for my sin and his righteousness is imputed to me through faith. Did you intend for us to understand it that way?

    This comment was originally posted on One In Jesus.info

  • heavenbound

    NO NO NO! to quote Paul in this instance leaves us with an empty hope.
    Paul was given progressive revelation. The offering of the kingdom was still in his sight. That is why is went to the synagogue first and then to the teaching of a new message that salvation was being offered to the Gentiles. Remember what he said about another gospel? That was mixing law and grace together. The old testament is used to study from a point of where have we come from. Not to be used on where are we going. I really don’t do communion any more because of its Jewish nature you just explained.

    This comment was originally posted on One In Jesus.info

  • rey

    Considering that this whole thing is just a tradition of the elders and that Jesus and the apostles had a tendency of disregarding those, I don’t think any of this is accurate.

    Fascinating article, I never knew that. Um, according to your article, “one-cuppers” would be totally un-biblical in their practice.

    Anyone who is literate can clearly see that he only said “This is the New Covenant ratified by my blood over one cup” regardless however many other cups they had at the Passoever. For example, with the first cup mentioned in Luke he doesn’t ascribe this meaning to it. Only to the second. Just sayin.

    “This seems altogether consistent with the Calvinistic doctrine of penal substitution atonement.” Leave it to a Cavinist to make every discussion about bashing God’s grace.

    “I really don’t do communion any more because of its Jewish nature you just explained.” What does that mean ‘really don’t do communion’?

    This comment was originally posted on One In Jesus.info

  • http://news.ycombinator.com/user?id=patio11 patio11

    I’m just referring to points and badges as an incentive mechanism for directing user interaction of a site in ways which provide business value.

    This comment was originally posted on Hacker News

  • http://www.oneinjesus.info/ Jay Guin


    If I insisted on Passover as a condition to salvation, that might be another gospel. Understanding the Jewish history behind communion is not remotely another gospel.

    It’s unfathomable to me that you reject communion because of its Jewish roots. Faith has a Jewish nature. So does the concept of a Messiah/Christ. So does Jesus. Christianity has a Jewish nature.

    (Rom 11:24) After all, if you were cut out of an olive tree that is wild by nature, and contrary to nature were grafted into a cultivated olive tree, how much more readily will these, the natural branches, be grafted into their own olive tree!

    We Gentiles were grafted into a Jewish tree. We therefore have and should honor our Jewish roots.

    This comment was originally posted on One In Jesus.info

  • http://news.ycombinator.com/user?id=prole prole

    Exploring the linked site a little deeper, there’s this gem on making applications more game-like: http://lostgarden.com/Mixing_Games_and_Applications.pdfFrom what the author explains in the presentation, vim violates the tried-and-true video game mechanics of starting the user with only the most basic functionality. To teach vim using the author’s method, one could start by removing all but the most basic commands. Then, these missing commands could be introduced to the user one at a time, in a controlled environment where there is a clear task (eg. jump the cursor to a particular point in the text) that can be measured as success or failure.

    In the linked presentation, the author draws a comparison to the game Metroid. In the game, the player falls into a deep pit and has to find a way to climb out before being able to continue on. Failure to perform the new skill (accurately timing the character’s wall-jumping) is immediately clear because the player will fall back down into the pit. When the user finally times it correctly, he’s free from the pit and the brain rewards him with a sense of accomplishment.

    Back to vim, if you wanted to create such an environment for learning a new command to move the cursor around, you wouldn’t want the user to fall back on basic navigation with h, j, k, and l. You could disable these keys temporarily, or leave them but only reward the user if they accomplish the goal using the fewest key-presses possible.

    Once they "win" this "level" you’ve designed, those new navigation keys should be considered part of their arsenal of skills for solving future problems. Each skill mastery could be further rewarded by filling in parts of a cheat-sheet (like this one: http://www.viemu.com/vi-vim-cheat-sheet.gif). This can be seen in the section where the author talks about Link to the Past and the picture of the player’s item inventory. The vim player’s goal could be to "unlock" and master these keyboard skills and ultimately fill in the complete chart.

    This comment was originally posted on Hacker News

  • http://news.ycombinator.com/user?id=ZeroGravitas ZeroGravitas

    I don’t know if I’d call it a game, but there is vimtutor, which is a document that tells you how to edit it:http://www2.geog.ucl.ac.uk/~mdisney/teaching/unix/vimtutor

    This comment was originally posted on Hacker News

  • http://news.ycombinator.com/user?id=ZeroGravitas ZeroGravitas

    Having read the comments I found it interesting that one of the features is to show you which of the features of Office other people actually use. I’ve had the same experience with blog posts or hacker news submissions about Vim or Unix tools in general. Simply being informed of their existence is a real benefit, otherwise they’d just sit there in my machine unused. (The hefty guidebook Vi IMproved by Steve Oualline was good too)

    This comment was originally posted on Hacker News

  • http://backreaction.blogspot.com/ Bee

    Cameron: You know my take on the issue, but let me briefly summarize it. The problem isn't peer review. The problem is a) that the reviewers have little incentives to do a good job and/or b) are not aware what is required of them for their review to be beneficial for progress in science.

    Concretely I mean that reviewers have little time, have in the vast majority very insecure jobs and future options, so they'll fight for their own opinions in any possible way, even if they know it's unscientific. They know that, unfortunately, their colleagues' appreciation as well as their funding depends on how many people work on their own field (where there's flies, there must be shit). They rarely misunderstand the necessity of taking risk. But maybe worst of all is that the time it takes to offer thoughtful comments and constructive criticism is, the way it looks now, completely wasted. We simply have no culture in which criticism is sufficiently appreciated.

    These are social problems, caused by insufficient education and external pressures (time, financial, peer pressure). The problem with peer review are symptoms, not the disease.

  • stan


    You stated, “I really don’t do communion any more because of its Jewish nature you just explained.”

    Is that the reason you give Jesus in the assembly when you decide not to take communion? Do you express that thought to Jesus when communion is served? Are you telling me that . . . had you been in the upper room when Jesus was serving the communion meal . . . that you would have said, “No thanks.”

    I don’t understand your reasoning. Anyway, didn’t Jesus say, “Do this in memory of me.” Isn’t the bread and the wine something to embrace? Something To cherish? I would think that someone who was heavenbound would cherish this gift Jesus gave us.

    Some other questions for you:

    My god is the god of Abraham, Isaac, and Jacob. Is not my god your god? You wouldn’t reject my god would you . . . seeing that he was their god first?

    Wasn’t the “prophet like me” prophesied by the prophet speaking in Deuteronomy 18 . . . Moses the lawgiver? You wouldn’t reject The Prophet Greater Than Moses would you . . . seeing that the very god at work in Moses brought to completion that which Moses prophesied?

    Didn’t Paul tell us to study the scriptures . . . the Jewish scriptures? You wouldn’t reject the Psalms would you?

    This comment was originally posted on One In Jesus.info

  • stan

    Bishop Jay,

    Back to the lesson on the 5 cups. I really found myself nourished by this information, as I do in all of the summaries you are giving us on the faith lessons by Ray Vander Laan.

    After reading the lesson above, it seems to me that the bread that represented his body and the wine that represented his blood were incorporated into a passover meal that was already in practice. So instead of these two things taking the place ot the passover meal, or being instituted as something separate, they were made a part of the passover meal. The passover meal was “upgraded” or reshaped into the communion meal Christ initiated. Am I reading correctly?

    We see this practice in other aspects of what Jesus gave us. Christian baptism is an example that comes to mind. Baptism wasn’t new in Acts 2. There was already a baptism for forgivenss of sins. Jesus took the baptism that John the Baptizer was administering to the people and “upgraded” it. Not only was there foregiveness of sins as John taught . . . but also the gift of the Spirit was promised. A pretty good upgrade!

    So God’s practice of taking existing expressions of faith . . . and incorporating them into new expressions of faith . . . is well established. Did I get this right?

    Is it a stretch to conclude that it was Jesus’ intent to incorporate the symbolism from the original passover meal into the new meal that Christians would partake? This seems reasonable because, as the lesson illustrates, it would help us “understand our own Christianity in much greater depth.” and help us “understand the Lord’s Supper in greater depth.” By dropping the original components of the meal, we have lost the symbolism that Jesus intended to hand to us. Did I read this correctly? Is that what Ray Vander Laan is thinking? Is that where you are leading? I like it.

    This comment was originally posted on One In Jesus.info

  • Brian Whitworth

    Good article! See also our recent First Monday paper and the following month's suggested solution to achieve what Andrew Morrison suggests:
    It is about time this discussion was engaged.
    Brian Whitworth

  • http://cameronneylon.net Cameron Neylon

    Bee, I agree with everything you say here (as you know) and I think we're coming from the same perspective. How do we align incentives so that important things are done? The market is clearly broken but how do we fix it? But I still think there is a more fundamental problem.

    As far as I am aware this is no evidence that traditional peer review (prior to publication/decision, with a limited number of people) would be effective even if people had all the incentives in the world to do it properly. It doesn't matter how much the incentives are fixed if the wheels are still square.

  • Pingback: Examining Peer Review «

  • Pingback: The Third Bit » Blog Archive » Peer Review Is Broken

  • http://cameronneylon.net/ Cameron Neylon

    Interesting to think about this. The two obvious factors are first the “open” bit. One of the serious problems with traditional peer review is that it is necessarily limited by the small number of people involved – this is probably what maximises the random effects. The flip side, making it open, obviously has both the advantage and disadvantage of allowing anyone who is interested to comment. Central challenge is setting the procedural barriers to comment at a level which maximises important signal to noise.

    From a naive external perspective to the other advantage code has is that you can make “objective” assessments and test them against reality through explicit tests. Does this modification pass the unit tests? Is there evidence that speaks to whether this is real? Or just the imagination of us non-computational people. Comparison of code peer review in closed vs open systems?

    This comment was originally posted on The Third Bit

  • Dan

    Will O’Reilly start releasing the reviewer notes for its Conference proposals (rejected and accepted)?

    This comment was originally posted on O’Reilly Radar Insight, analysis, and research about emergin…

  • http://andyrussell.wordpress.com/ andyrussell

    I’d love to see all journals adopt the Nature policy of listing the contribution of each author at the end of the paper. I’ve published papers where I know that some of the co-authors haven’t even read the paper, let alone contributed text or figures to it. Ok, so they contributed a bit of data or something but I think that this should spelled out so those who really contribute can take the credit for it.

    [This is the example from the first Nature paper from 1999 that used this protocol: "R.R. conceived the experiment, and together with A.H. and L.L. carried it out; C.D.B. designed and carried out the data analysis; R.R. and C.D.B. co-wrote the paper."]

    This comment was originally posted on SomeBeans

  • http://www.blogger.com/profile/11076372969807940310 SomeBeans

    @andyrussell – it certainly is nice to know, it’s one of the things you tend to pick up at conferences but should really be recorded in the journal.

    This comment was originally posted on SomeBeans

  • Simon Higgins

    I like the way publishing organisations also make you responsible for layout, and make you use their own templates, often with tortuous styles for drawings, chemical reaction schemes etc. I sometimes wonder if they employ any sub-editors any more. That’s probably why their profit margins are so huge. We’re all so obsessed with high-impact publication, which means choice of particular journals, that they can dictate all this. Maybe we should have the courage to publish differently.

    This comment was originally posted on SomeBeans

  • http://www.blogger.com/profile/02896097973227483745 Nora Lumiere

    Very funny video, very interesting post for a non-scientist who’d always vaguely wondered what a "paper" was.
    Surprising that there’s so much unpaid work.

    This comment was originally posted on SomeBeans

  • http://www.blogger.com/profile/11076372969807940310 SomeBeans

    @SimonHiggins – I think the astrophysics and high energy physics communities have made some progress on doing their own thing at http://arxiv.org/ I think they have an advantage with being a relatively small community focused around a smaller number of facilities which makes reputation easy to develop personally.

    @Nora_lumiere – glad you enjoyed it! I had an inkling that for a lot of people the idea of an academic paper is pretty alien, which was why I wrote the post.

    This comment was originally posted on SomeBeans