I guess the point for me about the GWAS paper is that the criticisms in the 23andMe post as well as comments by dgmacarthur (I think it was) and others revealed fundamental technical issues that should have been exposed by technical review at any reputable journal, let alone Science where one presumes the standard is somehow higher than average (I can't speak to this myself as I've never reviewed for them nor had a paper get to review). It my be my reading of the post but it sounded as though there were fundamental technical checks that should be done for any GWAS paper that simply weren't.

In terms of the editor's problem, and I am an editor at PLoS ONE and for a new BMC journal soon to be launched, I couldn't agree more. The question is whether its worth the bother, is it more effective to simply put it out for anyone interested to review, and then allow the paper to be iteratively improved in response to those comments until it reaches some sort of level of agreement that it gets some sort of quality mark. I would argue that those papers that don't ever get a review should be released simply as data or blog posts or whatever. There will be some lost gems in there but that is a discovery problem - probably under the current system they wouldn't be published anyway.

Don't imagine that I'm saying peer review isn't useful. But the question, is which peers, and when, and how do you motivate them to contribute positively to the process of improving this piece of communication?