A couple of ideas have been rumbling along in the background for me for a while. Reproducibility and what it actually means or should mean has been the issue du jour for a while. As we revised the Excellence manuscript in response to comments and review reports, we also needed to dig a bit deeper into what it was that distinguishes the qualities of the concept of “soundness” from “excellence”. Are they both merely empty and local terms or is there something different about “proper scholarly practice” that we can use to help us.
At the same time I’ve been on a bit of a run reading some very different perspectives on the philosophy of knowledge (with an emphasis on science). I started with Fleck’s Genesis and Development of a Scientific Fact, followed up with Latour’s Politics of Nature and Shapin and Schaeffer’s Leviathan and the Air Pump, and currently am combining E O Wilson’s Consilience with Ravetz’s Scientific Knowledge and its Social Problems. Barbara Herrnstein Smith’s Contingencies of Value and Belief and Resistance are also in the mix. Books I haven’t read – at least not beyond skimming through - include key works by Merton, Kuhn, Foucault, Collins and others, but I feel like I’m getting a taste of the great divide of the 20th century.
I actually see more in common across these books than divides them. What every serious study of how science works agrees on is the importance of social and community processes in validating claims. While they disagree violently on what that validation actually means all of these differing strands of work show that it is explicit and implicit processes within communities, supported by explicit and implicit knowledge of how they operate and who is who, and who gets to say what, and when. The Mertonian norm of “organised scepticism” might be re-cast by some as “organised critique within an implicit frame of knowledge and power” but no-body is arguing that this process, which can be studied and critiqued, crucially can be compared, is not dependent on community processes. Whatever else it is, scholarship is social, occurring within institutions – that are the product of history – that influence the choices that individual scholars make.
In the Excellence pre-print we argued that “excellence” was an empty term, at best determined by a local opinion about what matters. But the obvious criticism of our suggesting “soundness” as an alternate is that soundness is equally locally determined and socially constructed: soundness in computational science is different to soundness in literature studies, or experimental science or theoretical physics. This is true, but misses the point. There is an argument to be made that soundness is a quality of the process by which an output is created, whereas “excellence” is a quality of the output itself. If that argument is accepted alongside the idea that the important part of the scholarly process is social then we have a potential way to audit the idea of soundness proposed by any given community.
If the key to scholarly work is the social process of community validation then it follows that “sound research” follows processes that make the outputs social. Or to be more precise, sound research processes create outputs that have social affordances that support the processes of the relevant communities. Sharing data, rather than keeping it hidden, means an existing object has new social affordances. Subjecting work to peer review is to engage in a process that creates social affordances of particular types. The quality of description that supports reproducibility (at all its levels) provides enhanced social affordances. What all of these things have in common is that better practice involves making outputs more social.
“More social” on its own is clearly not enough. There is a question here of more social for who? And the answer to that is going to be some variant of “the relevant scholarly community”. We can’t avoid the centrality of social construction, because scholarship is a social activity, undertaken by people, within networks of power and resource relationships. What forms of social affordance are considered necessary or sufficient is going to differ from one community to another, but this may be something that the all have in common. And that may be something we can work with.
Isn’t it ironic that more than a decade after social media arrived to us from outside scholarship, the scholars who were among the most early adopters of social media for scholarship are re-discovering just how social scholarship is? :-)
At least for me, these insights are somewhere between “well, duh” and “this should have been on everybody’s minds at least 15 years ago!” Thanks for sharing, I’m learning a lot and not the least to love reading books by proxy and getting the tl;dr :-)
Oh, and to provide a thought to your question in the title: I thought that the work to arrive at principles for a scholarly commons was valuable to define commonalities among fields and perhaps a much-needed reminder of how much we share and how specialized between-field differences actually are?