Science in the Open

Open Access week is a fitting time to be finalising a project on Open Data. About two years ago I started working with the Canadian development funder, the International Development Research Center, to look at the implementation of Open Data policy. This week the final report for that project is being published.

Everyone, it seems agrees that opening up research data is a good thing, at least in the abstract. While there are lots of good reasons for not making data open in specific cases, its hard to make a case against the idea that, in general, it would be good for more of the outputs of research to be more available. The traditional way to achieve this, at least for funded research projects is through funder policies requiring data management and data sharing.

But while there’s a steadily increasing number of funders requiring data management plans and data sharing there is less work asking whether they work. There’s a lot less work asking what might be an even harder question, what does it actually mean for them to work? For the past two years I’ve been working with the IDRC, a Canadian funder of development research to probe that question. In particular I’ve been looking at how policy, and support of policy can support the kind of cultural change that leads to good data management and data sharing being the default for researchers.

The reason the project focused this way was due to a striking early finding. My first task was to interview a set of international experts on research data management and sharing for their views on what worked and what did not. What emerged was a sharp divide between those who thought that Data Management Plans were the best way to get researchers thinking about RDM, and those who were worried that the requirement for DMPs was leading to a tick box “compliance culture”. There was a concern that this might actually be damaging the process of culture change. I wrote about this concern in the review written for the project, published in RIO Journal (where all the formal documentary outputs for the project can be found).

The main part of the project was focussed on seven case studies of IDRC-funded projects. We worked with the projects to examine the data that they were generating, and then to develop a Data Management Plan. Then we followed the challenges and issues that arose as the projects implemented those plans. IDRC funds development research done in developing countries. We had projects based in Vietnam, Niger, South Africa, Columbia Egypt and Brazil working on projects in areas from copyright to climate change, health issues to sexual harassment, ecology to economics.

When we looked at the projects we found any of the challenges you might expect for data sharing in places away traditional centers of research prestige. Lack of good network connectivity, limited capacity and resources. We saw serious issues arising out of the English-centric nature of not just research communication, but systems and infrastructures. And we saw that the funding of infrastructures was inconsistent, periodic and unstable.

But what we also saw were challenges that arose at the funder level. These were not restricted to IDRC, and not limited to development funding or even data sharing, but general issues of policy design and implementation for research funders. Funders work with many projects, and those projects have different contexts and issues. Often the relationship between funder and project is mediated through a single person, the Program Officer. This single person then has to be the conduit for all policy communication, implementation, and monitoring.

Its hardly news that the hard working people within funders have a lot on their plate. What I certainly hadn’t seen before was how this limited progress towards culture change amongst researchers. Within our pilot project we had a group of highly engaged program officers that were learning about data management as they went. New policies required them to become instant experts able to advise, and where necessary push, funded projects towards new practice. There are a lot of policy initiatives within funders at the moment, and the load means that not all of them get the attention they need.

This turns out to be a problem because of the message it can send. Culture change really involves change within both researchers, and the funder. Each on their own can change practice, but embedding that practice so it becomes a culture, of “that’s just how things are done” requires mutual reinforcement. It is not enough to say something is important, you have to show you care about it. Many policies say nice things, but without the backup, and the infrastructure to support it, the message that can come across is “we talk about this, but we don’t really care”.

I had a very engaged and sympathetic group of program officers and projects, but even for this group, data wasn’t always the highest priority. We had to plan to regularly reinforce, and provide support for, the message that data was important over the course of the pilot project. Across a whole funder the risk of balls being dropped is very high. This raises issues for policy design and implementation. In the final report, just published at RIO the following recommendations are made.

1. Policies serve at least two distinct functions. They serve as a) signals that a specific issue, in this case improvements in data management and sharing, are an issue that the policy maker takes seriously and b) as a means of creating interventions in the behaviour of those subject to the policy. These two aspects of policy intent can be in tension with each other. Policies in general have been designed with the intent of requiring and driving behaviour change. The aspect of signalling, and its role in supporting culture change has been taken less seriously. In the initial review it was noted that the aspirational signalling aspects of some policies were at odds with the content of their interventions.
2. The signalling function of a policy is important for culture change. Researchers and funder staff take note of the direction of travel. Articulation of aspirations, particularly when they align with existing sympathies and narratives is important. The details of policy design seem less important and can be antagonistic to cultural change where the aspirations are not matched by capacities provided by the policy maker.
3. Internal audiences are at least as important as external. Policy needs to be designed with a view as to how it can effect the desired cultural change within a funder as much as among researcher communities. How does the policy articulate the importance of its aspirational goals, does it give permission for individuals to act, how does it help create a strong culture that aligns with the policy goals? Perhaps most importantly does it provide the necessary internal levers to ensure that those charged with implementing policy can access the necessary resources, infrastructure and expertise?
4. The primary failure mode for policy is overreach. The most common issue with policy implementation is where demands are made that cannot be met or evaluated. Generally the concern is whether researchers have the capacities, resources or expertise to deliver on policy requirements. However the risks can be substantially higher where the shortfall is at the funder. The core of culture change is a sense that the aspirations underpinning the change are both important and shared amongst stakeholders. Where a grantee falls short, allowances can be made in a way that reinforces the importance of the direction of travel to the grantee. However when a funder fails to provide the support and infrastructures – including Program Officer time, skills and expertise – necessary for policy implementation, and most particularly when implementation is not evaluated, the message is sent that the issue is not in fact important. This in turn leads to instrumental behaviour and in turn a lack of, or even negative, cultural change.
5. The interventions required by policy must be properly resourced, continuous, and self-reinforcing. The second aspect of policy remains important. Interventions at the level of requirements and practice have a role to play, both as opportunities to engage – and therefore transmit culture – and as a means of driving best practice and therefore achieving successes for grantees from their behaviour change. For this goal to be achieved the interventions need to be embedded in an ongoing and reinforced narrative that engages both researchers and funder staff.

This being Open Access week it seems worth thinking about how we design policy, not just within funders but more generally so as to effect culture change, not just change behavior. What might we do to better design both policies and implementation to build a greater sense of a collective direction of travel, and how do we ensure that travel is monitored in a way that is sustainable and builds momentum rather than becoming a tick-box exercise?

I’m not quite sure exactly what was the reason but there was a recent flare-up of the good old “how much does it cost to publish a scholarly article” discussion recently. Partly driven by the Guardian article from last month on the history of 20th century scholarly publishing. But really this conversation just rumbles along with a regular flare up when someone does a new calculation that comes to a new number, or some assertion is made about “value” by some stakeholder.

Probably the most sensible thing said in the most recent flare up in my view was this from Euan Adie:

There isn't a "true cost" of publishing an article independent of context. More things with no true costs: bananas, newspapers, tweets

— Euan Adie (@Stew) July 7, 2017

There is a real problem with assigning a “true” cost. There are many reasons for this. One is the obvious, but usually most contentious, question of what to include. The second issue is the question of whether a single cost “per-article” even makes sense. I think it doesn’t because the costs of managing articles vary wildly even within a single journal so the mean is not a very useful guide. On top of this is the question of whether the marginal costs per article are actually the major component of rising costs. One upon a time that was probably true. I think its becoming less true with new platforms and systems. The costs of production, type-setting and format shifting for the web, should continue to fall with automation. The costs of running a platform tend to scale more with availability requirements and user-base than with the scale of content. And if you’re smart as a publisher you can (and should be) reducing the costs of managing peer review per article by building an internal infrastructure that automates as much as possible and manages the data resources as a whole. This shifts costs to the platform and away from each article, meaning that the benefits of scale are more fully realised. Dig beyond the surface outrage at Elsevier’s patent on waterfall peer review processes and you’ll see this kind of thinking at work.

Put those two things together and you reach an important realization. Driving per-article costs to platform costs makes good business sense, and that as you do this the long tail of articles that are expensive to handle will dominate the per-article costs. This leads to a further important insight. It will be an increasingly important part of a profitable journal’s strategy will be to prevent submission of high cost articles. Add back in the additional point that there is no agreement on a set of core services that publishers provide and you see another important part of the strategy. A good way to minimise both per-article and platform costs is to minimise the services provided, and seek to ensure those services handle the “right kind of articles” as efficiently as possible.

Pushing costs upstream

Author Selectivity

Nature, the journal has one of the best business models on the planet and, although figures are not publicly available, is wildly profitable. The real secret to its success is two-fold. Firstly, that a highly trained set of professional editors are efficient at spotting things that “don’t fit”. There is plenty of criticism as to whether they’re good at spotting articles that are “interesting” or “important” but there is no question that they are effective at selecting a subset of articles that are worth the publishing company investing in. The second contributor is author selectivity. Author’s do the work for the journal by selecting only their “best” stuff to send. Some people of course, think that all their work is better than everyone elses’s but they in turn are usually that arrogant because they have the kind of name that gets them past the triage process anyway. The Matthew effect works for citations as well.

It may seem counter intuitive but the early success of PLOS ONE was built on a similar kind of selectivity. PLOS ONE launched with a splash that meant many people were aware of it, but the idea of “refereeing for soundness” was untested, and perhaps more importantly not widely understood. And of course it was not indexed and had no Journal Impact Factor. The subset of authors submitting were therefore those who a) had funds to pay an APC b) could afford to take a punt on a new, unproven journal and c) had a paper that they were confident was “sound”. Systematically this group, choosing articles there were sure were “sound” would select good articles. When ONE was indexed it debuted with a quite high JIF (yes, I’m using a JIF, but for the purpose it is intended for, describing a journal) which immediately began to slide as the selectivity of submission decreased. By definition, the selectivity of the peer review process didn’t (or at least shouldn’t) change.

Anecdotally my own submissions follow exactly this path. I’ve never had a research paper sent to be refereed at Nature, every one was rejected without review. Each one of those articles went on to get a lot of citations – several are considered important in their field (although no-one ever really got the thermal equivalence thing, shame really). Similarly the first article my group sent to PLOS ONE (after rejection at several other journals) hasn’t done too badly either (as an aside, you can get to the articles via Scholar, I don’t want to artificially inflate altmetrics by linking directly). As an aside there’s an irony here. One of the ways to maximise the “quality” of articles a journal gets is to avoid being indexed, but have a profile amongs traditional academics in North America and Europe, and publish fast. That way you will systematically get submissions people think are good, but have been rejected by other journals. These are systematically likely to include important articles unrecognised by more conventional journals.

If you follow this logic through then having systems to rapidly reject papers that are likely to be problematic makes sense. Some journals have internal (but obviously secret) systems that reject everything from specific countries (one large publisher had an automatic rejection for everything from Iran at one point) or that fit specific patterns (GWAS studies that link single, or small numbers, of gene up or down regulation to a specific stimulus). Many editorial processes apply various forms of conscious or unconscious bias to “positive” effect here, rejecting articles from unknown institutions.

Technical measures

It isn’t just editorial processes that push costs upstream. It’s also possible to get authors to do extra work by technical means. The most obvious of these is journal submission systems. Once upon a time all the metadata for an article would have been extracted by the publisher or an indexer from the manuscript itself. Submissions involved some number of hard copies and a cover letter being posted. Today the many hours of filling in forms that are typical of submission systems places that work onto the authors’ shoulders.

Similarly the many, many efforts to get authors to use publisher-supported authoring tools seek the same goal. If successful, which they are generally not, they would push work out of the publisher workflow upstream to authors. One of the problems that has arisen with this is that authoring technologies keep moving faster than publishing ones. In some cases there are journals that successfully use the outputs of LaTex directly but these are the exceptions.

The best example remains that of the IUCr journals that essentially provide web based forms to upload standard data formats and small chunks of text into predefined sections. By obtaining structured data from the authors in a standardised format the publishing costs are reduced massively. This is a near ideal case where many reports that are structurally very similar can be supported by a platform. Doing this for articles in general is clearly much harder and there are no good examples of this that I’m aware of for articles in other fields (some fields have side-stepped this for particular data types but that’s different in subtle but important ways).

Obviously there are limits here. It’s not feasible to simply impose new technical requirements and expect all authors to simply acquiesce. And as noted previously in this quasi-series there are challenges relating to the control that submission system vendors hold over publisher workflows. Nonetheless this is an area where publisher costs can be substantially reduced at least in theory.

Pushing costs (and risks) downstream

If one solution to reducing costs is pushing the upstream, then another is clearly pushing them downstream. This can mean a range of things – costs to authors, funders, institutions, or “the system”. As these are often not obviously direct cash costs they can be diffuse. These are the costs of the publisher (or really of any actor in the publishing workflow) not doing something.

An extreme example of these is so-called “predatory” publishers that do not do peer review. To simplify the example we can put aside the question of whether publishers “add value” through peer review and treat this simply as a question of validation and certification. In a world where we assume that being published in “a journal” means an article has been peer reviewed, what (and where) are the costs of that not happening?

Eve and Priego (2017) have dissected the question of who is harmed in a recent article. In particular they note that the harm, largely in the form of risk, falls largely on those relying on “publication” in “a journal” as a quality indicator or a certification of peer review. There is an accumulating risk of being exploited by unscrupulous authors for systems and institutions that rely on such markers. The cost may materialise either as wastage when a poor decision is made (to hire someone or to fund a project) or more directly as the cost of replicating the validation process that was supposed to occur. Whatever you may have thought of Beall’s list, and I really didn’t think much of it, it was a product of donated labour. When it folded, many of the evaluation systems that had come to depend on it discovered those costs and had to find a way to cover them. What is worth noting is that, the more institutions rely on such markers (including eponymous lists) as reliable proxies, the greater the risk and the greater the cost when problems arise.

This is a toy example, albeit one that really did play out in practice. The risks also run the other way as Eve and Priego note, the loss of perfectly good work that is published in an otherwise unreliable venue. Similar kinds of cost and loss do play out across the publishing system where venue is relied upon as a proxy; at the extreme end the costs (financial and in terms of mortality and morbidity) of cases such as the Wakefield paper, assumed to be reliable because it was published in the lancet, and on the other of highly promising young researchers neglected (and the investment in their careers wasted) because they happen not to have the required paper in the right journals, or book with the right publisher.

There are many examples where publishers have to make a choice to internalise costs or push the risk downstream. Checking for image manipulation is a good example. This remains costly, and in most cases prohibitively so, to apply to every submitted image so spot checking is common, or restricting checks to accepted articles. But that passes the cost downstream, either to the next journal in the submission chain, or to the community of readers. Readers may even be unable to check images because of format shifting and resolution changes in the production process to reduce publisher costs. Unfortunately most of these decisions are done quietly without any publicity. It remains almost impossible to determine across journals exactly what checks are carried out, with what frequency, and at which stage.

What does this mean?

First and foremost, don’t take anyone who says “the cost of an article is [or should be] X” seriously unless they are very, very, clear on exactly what parts of the process of publishing they are talking about and the context in which those parts are taking place. Second, we need to get much more explicit about what services are being provided. There have been a range of efforts to try and improve on this, but they’ve not really taken off, at least in part because they’re not really in the interests of the big publishers, who benefit the most from playing the games I’ve described here. As I’ve argued elsewhere we need a much better understanding of how shifting the costs from the publisher balance sheet, to the academic worker, or institutional infrastructure, play out. How do cash costs of managing a strong quality assurance process within a publisher scale when they become the non-cash costs of an academic editor, or referee’s time? When they become the time wasted by researchers on following up on fatally flawed research?

All of the above relates to costs, and none of the above relates to price. We also need a better understanding of the political economics of pricing. Particularly we need to understand where we are creating a luxury goods market where price (either in work to get through the selection process or in money) drives the perception of brand, which reinforces the perceived value of the investment, and therefore leads to an increase in price. Be very wary of claims that publishing is a “value driven market”. This is both a way of avoiding a discussion of internal costs, which in some cases are outrageous, and of avoiding the discussion about precisely which services are being provided. It’s also a captive market. Another value-based market is plane flights out of Florida this week…

Above all, we need a much more sophisticated understanding and far better models of how costs are being distributed across the system in cash and non-cash costs, in labour and in capital. Overall we can point at the direction of travel, and the problems with existing pricing and cost structures. But its hard to make usable predictions or good design decisions without a lot more data. Getting that data out of the system as it exists at the moment will be hard, but the direction of travel is positive. There’s nothing wrong with publishers making a surplus, but any conversation about what kind of surplus is “appropriate” needs to based on a much deeper understanding of all the costs in the system.

I’ve just read Lave and Wenger’s (1991) book Situated Learning on the recommendation of Isla Gershon. Like many books I’ve been reading this was radical in its time but reads in some ways to me today as common sense. It’s actually quite hard for me to reconstruct the world view in which this was seen as a dangerously radical departure.

The core of Lave and Wenger’s argument is that to understand learning we have to see the learner as a whole person in a web of relationships. They propose that a particularly fruitful lens for analysing these relationships is to see learning as “legitimate peripheral participation” in a “community of practice”. That is, that productive learning occurs when a learner is licensed to be a participant in a community, and through that participation they become more fully engaged in its practices. They use case studies from models of apprenticeship as their material.

Throughout they emphasise the importance of seeing identity construction as a core part of learning. That is, the path to fully engaged membership in the community of practice is one of co-constructing an identity of being part of that community of practice. The parallels with the way I’ve been thinking about knowledge clubs should be obvious.

Lave and Wenger deliberately use apprenticeships as their case study to avoid getting bogged down in the critical debates around schooling. They are aiming to build a general model and are concerned to remove their model building from the circular political and analytical debates that swirl around the value of traditionally institutionalised schools. However at one key point they pose a central question. What is the community of practice that is being reproduced in a school, in their example in a physics class? It is not the community of (professional academic) physicists but perhaps a (subcomponent of) the community of schooled adults. In turn this raises the question of the goals of learners and teachers. They return repeatedly to tensions between the goals of apprentices and masters but leave the more political question of what is a school for to the side.

This struck me in turn as paralleling a passage in Ravetz’s (1971) Scientific Knowledge and its Social Problems. Ravetz is arguing that the most reliable scientific facts are ones that have been transferred and abstracted across multiple communities. In particular he discusses the process by which scientific claims that arise in specific communities are transmuted into the ahistorical stories that are used in standardised school curricula. The key point I took away was that teaching was a remarkably productive site to test candidate “facts” for their comprehensibility and salience to new community members. That is, the production of knowledge occurs at the boundaries of communities and is highly productive when it engages legitimate peripheral participants.

What is interesting is the way in which Lave and Wenger’s lens suggests a way of dissecting how a community can most productively structure interactions so as to maximise not just learning (and therefore community growth) but also teaching as a tool of knowledge production and validation. Lave and Wenger emphasise criteria for effective learning, access to resources, access to participation and transparency. They also note throughout the problematics of power relationships that underpin what it is to be a peripheral participant.

The question of why the academy is an apprenticeship system and whether this is good or bad has long troubled me. Ravetz, by focussing on craft knowledge (and in a similar way Collins on tacit knowledge) naturally leads to an assumption an apprentice-like system. Seeing learning through the lens of legitimate peripheral participation, implies several ways in which apprenticeship-like systems can be highly productive. It also provides criteria for the qualities of communities that support such participation effectively. All of these lead to seeing the importance of appropriately tensioning inclusion and exclusion. What I wonder is whether the qualities of communities of practice that support productive learning as they see it, naturally generate the values of scholarly communities both in theory (eg Merton) and in practice? What forms of practice support knowledge production that is effectively refined through teaching, so as to be most accessible to learning?

I had a twitter rant and a few people asked if I would convert it into a post. It also seemed worth preserving. This is a very lightly edited version of the thread that starts with this tweet.

I saw this thread last night about #openscience inclusion and diversity, and I'm not going looking for it again.

But here's my view. 1/n

— CⓐmeronNeylon (@CameronNeylon) August 10, 2017

The only thing that links all varying strands of open science (and open scholarship more generally) is inclusion and diversity as a first principle. The primary value proposition of open science is that diverse contributions allow better critique, refinement, and application. This is most obvious in the strands of citizen science, stakeholder engagement, and responsible research and innovation. But it is equally true in strands of open data, transparency, registered reports, and open notebook science.

The point of more complete release of record is to enable a diversity of critique and contribution via more diverse communication modes. The point of earlier release of results is to gain more diverse commentary and input on the direction of research. When we talk about need to support “machine reading” we’re also talking about a particular form of inclusion and diversity (see particularly Latour in Politics of Nature for the full fat version of this perspective). It’s just that way too many of us have privileged the diversity of machine readers over underrepresented minorities for way too long. Myself included.

The fundamental claim of open science is that diverse questions, diverse outputs, diverse critique, and diverse capacities combine to make better research. This also turns out to be the underlying position of many other folks from Boyle to Merton and beyond. Organized skepticism from Merton, full communication of the record, including failed experiments, from Boyle, esoteric and exoteric communities from Fleck. But most fundamentally diversity is the bedrock of empiricism. Testing knowledge claims across contexts for their generality, across diverse contexts to test them to breaking point. Falsification involves the serious attempt to disprove a theory. Where better to figure out how best to do that than from someone with a different experience and perspective?

Ultimately this is why I’ve moved from the sciences to the humanities. Science is good at answering carefully posed questions. Humanities has a great skill set and tool kits for asking whether those are the right questions to be posing, and why they are being posed that way. It often seems that the academic humanities aren’t so good at turning that apparatus on ourselves but the tool kits are at least there. The same has been true of open science We have talked implicitly about diversity (of outputs, of timing of release, of contributors). But because we have not focused on diversity explicitly we missed the general point. The point that diversity and inclusion are the core epistemological priors of the general argument.

So how do we fix that? Well to start with, we need to shut up and listen when underrepresented minorities raise issues with our scholarship. Not just because its polite or politic, but because these are some of the most powerful opportunities to test our claims in wider contexts. Fundamentally its just good scholarship. General knowledge claims need to find purchase or application in diverse contexts to have value (and that value is in turn contextual). The only way to test that is to work towards greater diversity and inclusion. Listen to those wider contexts. As we often say of open science, it’s just good research.

It took many people with sufficient patience to lead me to this. But you can find many who will challenge your perspectives productively on social media. If you can listen you can gain the insight of many different perspectives, many a long distance from your own experience. Unfortunately making a list is a great way of having those people get even more crap in their streams than they already do (hint: the verb in the previous sentence is listen). But they’re not hard to find if you put the effort in.

 

Following on from my post yesterday a couple of questions popped up about collective and collectivist models in scholarly communications. Richard Poynder is skeptical, which left me nonplussed because from where I sit what I described is happening all over the place. Funders are looking at investment strategies, collectives are forming and some of them are growing very rapidly. Which brings us to the second and more concrete question. How is it that things like Open Library of Humanities, a collective funding model for journal publishing, and similar models like Knowledge Unlatched* for books, work?

"classical econ of collectives is very old" @CameronNeylon do you have a classic econ analysis of @openlibhums etc? https://t.co/TIltjGQSPV

— @alexh.bsky.social Holcombe (@ceptional) July 6, 2017

The answer to this was, yes I do, but it keeps changing. But perhaps the time has come to lay that out and give a really concrete example. As I noted this is a weakness of our A Journal is a Club working paper, which has led some to think it only applies to a very narrow kind of journal.

The classical subscription journal club

In our “traditional” system we usually think of a journal as a club of subscribers. A set of actors band together to create a “club good”, one which is exclusive but broadly non-rivalrous. In the case of a subscription journal, this is access to content. “Access” here is meant broadly to include all the processes that lead to the creation of a journal, not just merely providing a key to existing content. However linking the funding of that content creation to access means that a certain class of free rider is avoided. The existence of that content is a collective (and in some ways at least partially public) good. The conventional analysis follows that it is only by making access to the collective benefits of that good exclusive that the classical collective action problems are avoided. That is, if the content were freely available then rational economic actors would not contribute, and the collective good would not be provided.

Buchanan, working in the 60s on club economics would have noted another important characteristic, particularly in the print world. That is that access is not purely non-rivalrous. With a fixed number of copies on the shelf, if there are too many members with access then there is a rising probability that when I go to the library I will not have access. Buchanan showed that for club goods, where access is usually non-rivalrous up to a point, but after that point there is increasing friction, which leads to a natural limit to the size of the club. Clearly as we move to the digital world, this friction drops radically (not to zero but certainly several orders of magnitude). This in principle allows that natural limit on club size to rise.

The observant reader will note that I’ve conflated two different clubs already. One is the club of subscribers to a journal. The second is the club of members of a particular library. I haven’t fully worked out the analysis for this, but it is possible to point to the way that big publishers have retained the link between pricing and historical print spend, and seeking to reduce the ability of libraries to combine holdings and expand their user base radically as an effort to prevent the natural club economics working its way through. Thus the serials crisis could be painted as a break down in the economic equilibrium of a series of close linked club-like groups that has been exacerbated by reducing market flexibility through contracts, pricing and big deals.

However, the main point I want to make here is that the link between the club that funds a journal (i.e. its subscribers, generally academic libraries) and those that actually use the content (and benefit from access as a club good) is not as obvious and direct as we often think.

Collective funding models through the lens of club economics

If we turn our attention to collective funding models that have recently developed, primarily Open Library of Humanities in the journal world and Knowledge Unlatched with books we can draw broadly two forms of analysis. The first is to reject the classical economic analysis that says these cannot work due to free-riders. In this form we note that libraries and universities are value-led organisations that are free to act in an economically irrational fashion in order to serve their mission. There’s an argument to be made for this analysis, and I think it might make a lot of sense in some cases at very large scale. For instance, it can make sense to see the community funding of Wikipedia as a case where at sufficiently large scale you can reach enough economic actors prepared to act in an irrational fashion to create a public good.

However, when it comes to budget negotiations and resource allocation in a cash strapped library or university setting an argument built around “we should be irrational” will only get so far. We can apply a club economics based analysis to projects like OLH and KU. That analysis simply leads to an obvious conclusion, there must be some club-like, and therefore exclusive, good benefit to the contributors. This benefit must have a value greater than the contribution. It is also likely that there will be friction in accessing that value that increases as the number of members increases. However, with more members the cost of contributions can go down, or alternatively more of the good can be provided. How friction, pricing of contributions, and goods provision scales with the number of contributors is crucial. So figuring out exactly what contributors are getting really matters.

Obviously exclusive access to content is not the club good. But neither is non-exclusive access. This is a collective, public-like good, not reserved to contributors. But what contributors get is a good feeling about making that public-like good available. This is a real benefit, particularly for brand driven organisations like universities, and for libraries within them that want to show internally that they support innovation in scholarly communications. This is an exclusive benefit, only available to those who contribute and have that contribution recognised. I doubt, for instance, that any universities are contributing anonymously to OLH. OLH assiduously recognizes each new member with dedicated postings and acknowledgement.

There is friction in access to this good. It’s great to be one of the first members of the cool kids club, it creates buzz, raises your profile, demonstrates your commitment to progressive change in scholarly communications. But once everyone is a member, the gloss wears off. When I first looked at the OLH model I thought this was what would cap growth. However it looks like I was wrong about that. Perhaps the momentum that has built up is its own driver, with institutions concerned about being left out? Classical economics is equilibrium economics and is generally poor at predicting the path towards equilibrium (such model systems are usually chaotic even in their approach to equilibrium). Overshoot is certainly possible, but equally possible is a shift in the perceived benefit.

A particular membership benefit for OLH is influence over its decision making. While an economic model would say that an investment in making future savings through subscription cuts is irrational (why not let others pay and reap the benefits anyway) investing in a level of control over the process by which that future saving is reached is certainly not. You might not be a first mover, but it is still worth engaging in having a say about the pace at which OLH grows, what gets included in the package, and why. Again, friction comes into play. The more members the less say any individual members have. In some cases membership differentiation can play a role here but having a system where certain libraries pay more to gain greater control seems antithetical to the values of the project and quite possibly detrimental to the good faith built up by contributors.

It would also be possible to explore other member benefits that relate to this control aspect. The ability to propose University Press journals for inclusion, first right of refusal to invest in particular projects that are of interest to specific libraries might be of interest. Other benefits might include access to expertise and support on publishing technology or financing. Access to these kinds of specialist resources was a big part of the success of the cases Mancur Olson describes in his book The Logic of Collective Action. In any case the major scaling challenge for OLH and similar efforts is exactly how to manage governance and strategy in ways that mean contributors have a stake that is meaningful to them and of value.

The contributions are small

One of the reasons this is working, and that the specific benefits are a little vague, is that the actual amounts contributed are relatively small. This is crucial, when the price of membership is low, the benefits can be a little diffuse. Feeling good about the contribution, a sense of being part of a movement for change, may well be enough. This works well because OLH has an efficient model for distributing costs, is growing membership, and thus keeping contribution pricing down even while it grows its journal portfolio. What happens though if OLH, or a similar system were to capture a larger portion of the market? Then contributions would rise as a proportion of library budgets. As the values rise the level of scrutiny rises and there will be a need for clear benefits.

It is the combination of the governance challenges with the rising proportional contributions that are likely to place upper limits on the scale of OLH and similar efforts. To be clear, flipping HSS journal publishing to a collective OA model does not need to be the end goal for OLH. It can declare victory as is in many ways, having demonstrated what is possible. But those of us who see such a system as a desirable model for funding the majority of scholarly communications will need to identify a route that navigates these twinned challenges of scale.

There are two questions I don’t have any answer for at all as yet. The first is what is the optimal ecosystem of such projects, one single system to rule them all with internal competition to constrain pricing? A small number of competing players or many smaller projects each building its own communities of interest? The second question, which is just as important, is what routes are feasible, and to what ecosystems? Just because a system is optimal, doesn’t mean we can get there. This in the end is to my mind the most important question. OLH and KU and similar efforts have already shown that non-access based subscription membership models can work. That means that membership benefits beyond access to content are feasible. But how we can institutionalise and grow them to the right scale to build a sustainable system of such efforts remains the big questions.

* Conflict of interest statement. I work on projects with Knowledge Unlatched Research as part of the work I do at Curtin University.

One of the things I find frustrating about discussions of economics in scholarly publishing is the way that discussions that are built around critique of capital models or neoliberalism are dismissed as impractical. Most recently Stuart Lawson’s interesting provocation, Against Capital, got a range of dismissive comments as being irrelevant because it required the overthrow of the capitalist system.

I find this, alongside another kind of response, most commonly from people in the business of scholarly publishing that such criticisms represent a failure to understand the financial realities of publishing, frustrating because they miss the point of critique. The point is to suspend judgement, move away from the default view of a system, and seek to look at it from a different angle. Change a few of the words around and those same people would describe such a text as “innovative” or “a refreshing change in perspective”, but because the discussion is political, or financial, the “realities” are thought  to be set in stone.

In the business setting this often leads incumbent publishers to a kind of spluttering defense of the value they create, while simultaneously complaining that the customer doesn’t appreciate their work. Flip the target slightly and we’d call this “missing the new market opportunity” or “failing to express the value offering clearly”. In the political world it privileges a particular set of economic models that happen to have the characteristic of reducing the power of communities to act collectively.

Lawson’s piece is exactly that, an invitation to consider what might be possible if we work collective. Nothing more, nothing less. Marx (and implicitly Piketty) tells us to seize the means of production not because violent revolution is desirable, but because the concentration of capital prevented communities coming together in new ways to create value for themselves. If you like, Marx was simply proposing disruptive innovation a century or so before Christenson, just he had a limited idea of the kinds of disruption that were feasible.

Today we live in much more interesting times. Capital is arguably more concentrated, particularly in the scholarly communications space, but the means of digital production are accessible at much lower cost than in the industrial past through open source tools and cheap platform technologies. Marx tells us that in the capitalist system labour benefits capital. Lawson’s invitation is to consider how our labour is contributing to capital, and how we might collectively leverage labour to bring more of that capital to serve the interests of our community. I doubt Marx would approve (and neither might Lawson) but this is just shrewd strategic business analysis, finding new opportunities in a crowded market.

What might that look like in practice? And where does the capital currently sit? One answer lies in digital collectives, collaborations and networks that create value for their members. And many of our existing scholarly communities fit this model. In our working paper, A Journal is a Club, we started sketching one form of this but the model is much more general.

The classical economics of clubs and collectives is very old, having been worked out by Buchannan, Olson and others in the 60s. They present a very hard nosed, market economics oriented view of what makes a club or collective sustainable. The return to members must greater than the members investment. The value need not be monetary of course, but it must have greater value to the members than what they put in.

Buchanan and Olson’s models both show the conditions under which this can be sustainable. The club must generate some “more than the sum of its parts” benefit from the resources that members contribute. For collectives this can be access to capacity, buying power, expertise, or the pooling of capital. For any collective this will generally be positive based on simply economic arguments. On the debit side there are costs of coordination and management and friction in access to the club good. Buchanan’s model shows that as the club size grows and competition for access to the collective good rises that the debits crowd out the credit and so there is a natural size limit.

A stable collective is therefore “right-sized” which is a contract to a capital concentration model where growth is always privileged. Collectives can also solve a whole class of problems for which both for-profit entities and pure (governmental) public provision fail as shown by Ostrom. This class of problems includes the management of resources that are hard to exclude people from calling on, but not able to be infinitely shared, including grant funding, expert attention, and (oddly enough) public good will. Many of the most important “goods” in the scholarly enterprise fall into the category of commons or of club goods.

If we look at collectives within the scholarly enterprise, they have largely seized the means of production. They retain control over the systems and infrastructures critical to their functioning, including their income. Many scholarly societies however are no longer functioning collectives, having ceded control of critical systems, including crucial choices over core revenue sources, to third party providers. Other societies have turned into de facto for profit companies as the “club” element of their activities is dominated financially by the revenue raised through internal publishing operations.

But this is all very well. Does it matter for the hard-nosed finance person who has to figure out how to pay for stuff? How does an anti-capitalist neo-liberal critique help the person operating within a capitalist neoliberal system? What I want to suggest is that it can help to focus attention on where value is being created, what assets actually matter, and what are the blocks to realizing value.

Two examples may help to illustrate this. Lingua, the journal owned by Elsevier is at this point more or a failure. It has gone from one of the most important journals in analytical linguistics to no longer being in the field, and seems well on its way to becoming irrelevant. How does a company as competent in its business strategy as Elsevier let this happen? I would argue, as I did at the time that the former editorial board of Lingua resigned to form Glossa that it was a failure to understand the assets.

The neoliberal analysis of Lingua showed an asset generating good revenues, with good analytics and a positive ROI. The capitalist analysis focussed on the fixed assets and trademarks. But it turns out these weren’t what was creating value. What was creating value was the community, built around an editorial board and the good will associated with that. Taking a critical perspective would have meant thinking hard about where the value was and what the real assets were. It would have made for a better business decision.

A more abstract example is the growth of interest in building collectives both for funding and mutual support in scholarly communication. These turn out to be difficult to fund. They should be sustainable, provided sufficiently low coordination costs and good management, but setting them up is hard. A related problem is the way in which we lose technical innovation to commercial players, not because we don’t have the resources, but because we don’t have collective funding mechanisms to support them.

Too often the objection to these is that we don’t have the spare funds to support them. But this is the wrong analysis. These collectives will fund themselves (and those that don’t should be allowed to fail) if they reach the right scale and maintain control over their coordination costs. The problem is not funding, it is capitalization.

And we, the community, do have the capital. It sits in strange places, in property, in university pension funds, in endowments, as well as in the share portfolio accounts of funders. But we always deploy the revenue from that capital, never thinking about how to deploy the capital itself. Why are funder share portfolios not leveraged to invest in reducing future costs of scholarly communications? Why are university endowment funds not more invested in instruments that deliver not just financial returns but returns on the mission of these charitable and public institutions?

That was the question that Lawson’s piece explicitly invited us to ask. Not (necessarily) how the world might be better if we overthrew the capitalist system, but to critically think about where the capital and the power in system really lies, and perhaps to realize just how much of it lies in the hands of our community. If we choose to use it. The anti-capitalist perspective reminds us of the power we potentially yield in a capitalist system if we choose both to think outside the system and then to act purposefully within it.

Acknowledgements: This post is in large part prompted by a discussion that I had with Sam Moore of Lawson’s piece and the responses to it.

This is my submitted paper to ElPub, a conference running in Cyprus over the next few days. I’m posting it here as a kind-of-preprint. Comments and thoughts are welcome. The version in the proceedings is available online as part of Chan and Loizides (eds) Expanding Perspectives on Open Science: Communities, Cultures and Diversity in Concepts and Practices.

Abstract

The debate over the meaning, and value, of open movements has intensified. The fear of co-option of various efforts from Open Access to Open Data is driving a reassessment and re-definition of what is intended by “open”. In this article I apply group level models from cultural studies and economics to argue that the tension between exclusionary group formation and identity and aspirations towards inclusion and openness are a natural part of knowledge-making. Situating the traditional Western Scientific Knowledge System as a culture-made group, I argue that the institutional forms that support the group act as economic underwriters for the process by which groups creating exclusive knowledge invest in the process of making it more accessible, less exclusive, and more public-good-like, in exchange for receiving excludable goods that sustain the group. A necessary consequence of this is that our institutions will be conservative in their assessment of what knowledge-goods are worth of consideration and who is allowed within those institutional systems. Nonetheless the inclusion of new perspectives and increasing diversity underpins the production of general knowledge. I suggest that instead of positioning openness as new, and in opposition to traditional closed systems, it may be more productive to adopt a narrative in which efforts to increase inclusion are seen as a very old, core value of the academy, albeit one that is a constant work in progress.

1. The many strands of “open”

“Open” is a contested and increasingly it seems polarized term. It is also highly contextual. A number of different efforts have been made to disentangle the various discourses that underpin the advocacy programs that operate under the banner of open, but there is, as yet, little consistency between them. Fecher and Friesike’s “Five Schools of Thought” (1) sit uneasily beside Pomerantz and Peek’s “50 Shades of Open” (2), and while they both refer to the Open Knowledge Definition, various Open Access declarations and the debate between Free and Open Source software there is no clarity of definition.

Arguably all of these roots and their more recent interrogations are strongly rooted in Anglo-American conceptions of scholarship and political economy. “Open” in scholarship borrows heavily from the movements for Free and Open Source Software (F/OSS) while sitting alongside the movements advocating Open Government and Open Data. All of these are rooted in Western and Anglo-American discourses, not infrequently with a techno-utopian and neo-liberal slant. Coleman notes how the distancing of F/OSS discourses from “[…]movements predicated on some political intentionality, direction, or reflexivity or a desire to transform wider social conditions” nonetheless serves those political programs (3).

These discourses connect “open” in scholarship to networked communications systems and usually the web. The connection to F/OSS as the supposed historical root of openness often makes this explicit. This in turn connects “open” to broader discussions of collaboration that are also seen as being supported by networked communications infrastructures. Opportunities to be gained through engagement, both in open sharing of resources and in collaboration are assumed to provide equitable gains. Openness in these discourses is presumed to be uniformly positive for all who engage with it. The presumption of equitable opportunities for the traditionally disenfranchised and disempowered is a driving motivation for many engaged in Open movements.

At the same time Nathaniel Tkacz (4) that “openness” is almost always situated as an oppositional movement, one that opposes “traditional” and “closed” processes whether they be in government, reporting, property, or scholarly communications. He draws a thread from Popper’s The Open Society via the neo-liberal discourses inspired by Hayek to the rhetorics of F/OSS and their successors.

Openness is conceived as a new mode of being, applicable to many areas of life and gathering significant momentum – ‘changing the game’ as it were. Once again, this ‘spirit of open’ is closely articulated with collaboration and participation – Tkacz (2012)

In a move that is challenging for many who see themselves as advocating “the opens” Tkcaz traces these discourses, and particularly openness as “freedom” to the political agendas of libertarian politicians like Douglas Carswell (the British Conservative MP, better known today for first defecting to the UK Independence Party, and then leaving after it successfully campaigned for the UK to leave the European Union) and the US Tea Party movement. He argues that the freedoms being pursued are largely negative in the sense discussed by Holbrook (5). Openness is generally the effort to be free from the restrictions of the status quo.

They are negative in two ways. First they are absolutist in nature, but secondly they frequently make little sense except in the context of the fight against an existing status quo. Open only exists as a contrast to closed and, as Tkcaz traces in many examples, and other critics have noted, the implementation of open leads to it becoming – or being co-opted by – the status quo. The old open becomes the new closed that a new generation will battle against.

Constructed this way, openness can never win. The old “open” is the new “closed”. We see this cycle in criticisms of “open-washing”, of the power of those groups who control the definitions of open in software, and in the development of open government and open scholarly communications agendas. From offices of open government, to the Open Source Institute and the Public Library of Science, once an advocate of open has achieved stability and a measure of power they become a target, not just for reactionary forces but for their erstwhile allies.

Tkcaz argues that this means that any open agenda always has enclosure as its endpoint, that the underpinning rhetoric, being negative inevitably sows the seeds of its own demise. In his words:

If we wish to understand the divergent political realities of things described as open, and to make visible their distributions of agency and organising forces, we cannot ‘go native’, as a young, anthropologically-minded Bruno Latour once wrote, meaning that we cannot adopt the language used in the practices we wish to study. To describe the political organisation of all things open requires leaving the rhetoric of open behind -  Tkacz (2012)

In this paper I want to argue that while Tkcaz’s challenge needs to be taken seriously, that it is not fatal. The key to this lies in understanding how meso-scale political organization, and the inevitable inclusion and exclusion that arises from group formation, interacts with individual (micro-scale) and macro-scale political economics. To do this I will draw on strands of economics, political economy, and cultural studies to seek to show how the oppositional stance and boundary work necessary to define groups can nonetheless be harnessed to aspirations for inclusion and interoperability.

In particular, I want to examine the political and epistemological challenges raised by the inclusion of knowledge-workers from traditionally “peripheral” positions with respect to power centres of traditional Western scholarship. Understanding how a wider range of knowledge-making groups can interact productively and equitably ultimately requires and understanding of how these groups are sustained and how their differing cultures affect their interactions. My aim is to sketch a route towards how three differing framings might be aligned to develop a philosophical underpinning for open agendas. In doing this my focus is on scholarship, but the argument can be developed for much broader application.

2. Cultural Science as a Model

Central to my argument is the need for an enhanced focus of scholarship on the formation, culture, and sustainability of groups. Many arguments founder on the way they move directly from individual micro-economic concerns to a global macro-level argument. The need for “meso-level” analysis in a range of different disciplines has emerged over the last decade. Here I draw on the model of “Cultural Science” developed by Hartley and Potts(6).

Cultural Science seeks to be an evolutionary model of groups and culture. The unit of analysis is a group or community that shares culture. Hartley and Potts name this culture-defined group a “deme” borrowing from both biological (an interbreeding community) and political (the “demos”) terminology. The key to the model is that demes do not merely “share” culture, they are made by culture. Culture makes the group and the group enacts and articulates the culture.

Culture is not, in this formulation, the aggregate product of the individual actions or behaviours of members of the group but the thing which draws in members of the group through creating common narrative and meaning. Demes can be seen as a parallel concept to Fleck’s “Knowledge Collectives” (7) and Ravetz’s (8) or Kuhn’s “communities” (9).The primary difference lies in the underlying concept of how demes are formed and sustained.

Any given person may be a member of multiple demes, and demes can be embedded within other demes. As an evolutionary model it poses serious challenges of complexity in analysis, although arguably no more than the emerging complexity of selection operating at many different levels in biological systems. The key question for survival of a deme is how effectively it competes with other demes in the environment it finds itself in.

In the book “Cultural Science” (6) Hartley and Potts emphasize conflict between demes. More recently this has been developed to acknowledge that conflict need not be violent or existential (although it frequently is). We argue that it is through productive conflict that knowledge (or more generally capacities to act) are created. Demes may build internal capacities that allows them to act on other demes, that is to do violence, but they may alternately build capacities that enable them to interact productively with other demes to create new capacities. Without seeking to provide a strict definition at this stage, we can consider shared capacities that span more than one deme to be shared knowledge.

With the Cultural Science model in hand we can make some assertions about demes that do this successfully. They will have aspects of culture that promote productive interactions – productive conflict – across demic boundaries. These demes will invoke narratives and norms, and enact and articulate those norms, where they come into contact with differing view points. Such a set of norms might be expected to include acceptable modes of disagreement, agreed approaches to seeking resolution, a commitment to considering – indeed seeking out – alternative perspectives, and approaches for agreeing to disagree where resolution cannot be achieved.

3. An Epistemological Framing of Western Science Knowledge Systems from Cultural Science

If we were to look for an example of such a deme we would likely rapidly arrive at the Western Scientific Knowledge System (WSKS) as an example of a culture that has achieved both continuity in time and dominance over many other systems. We might note the set of cultural elements sketched out above align quite closely to Merton’s Four Norms (10) and to other (claimed) normative aspects of Western scientific culture. It could be further noted that the WSKS has a form of fractal organization in which discipline and subject and topic boundaries create opportunities for conflict at many different scales.

Finally, and crucially, we might note that the cultural elements that define the WSKS describe narrative and cultural aspirations not necessarily practice. Obviously if there is “too much” of a gap between the claims a deme makes about its practice and actual practice then the internal consistencies will build up and lead to failure. However it is also the case that a perfect alignment is not necessary.

This idea that aspiration towards enacting norms and demic narrative can be of value, even when those aspirations cannot be completely achieved, is also developed by Collins and Evans in Why Democracies Need Science (11). They make a different kind of argument for the value of WSKS in democracies and this has tensions with my argument that will discussed below. What we can adopt directly is the flow of their argument that by recognizing that there is value in the group level aspirations we can reconcile the tools and knowledge developed by both “Wave One” and “Wave Two” Science and Technology Studies (STS).

The so-called Wave One of STS uncritically accepted the value of Western Science and sought to examine how this value was created. Merton in particular worked on showing how individual human frailties could be ameliorated by shared norms and strong institutions that supported the creation of scientific knowledge. The overall group dynamic was assumed to be positive and ultimately objective. Wave Two STS critiqued this position noting that group dynamics was clearly related to power, that expertise and stakeholders from outside the academy were often discounted, and that the social context could determine both the process and outcomes of knowledge creation.

To reduce it to slogan form Wave Two showed that groups and institutions could never approach the objectivity and perfection assigned to them by Wave One. In parallel development of philosophy and epistemology that consistently showed that claims of WSKS to generating “truth” could not be demonstrated to be provable. The strong version of these two strands of scholarship led some to the other extreme. Because knowledge and the WSKS institutions supposed to be safeguarding it could not be shown to be provably reliable it follows that we must reject all authority.

Cultural Science, in common with the “Wave Three” proposed by Collins and Evans (11), offers a middle route. First we observe that a recognizable culture and community of WSK creation has persisted over (at least) several centuries. This evolved community has continuity and therefore its supporting culture has continuity. Through analysis of historical and contemporary narratives we can identify some elements of this culture that appear to persist: a valuing of observation, critique of claims, and interestingly an aspiration to civility in resolving disputes. Robert Boyle (12) writes in the 17th century responding to a critic with whom he has had no previous correspondence:

“[I will answer Linus’ objections] partly, because the Learned Author, whoever he be (for ‘tis the Title-Page of his Book that first acquainted me with the name of Franciscus Linus) having forborne provoking Language in his Objections, allowes me in answering them to comply with my Inclinations & Custom of exercising Civility, even where I most dissent in point of Judgement.” Boyle (1662)

Many of the social points Boyle makes about practice in his works, including issues of reproducibility and effective communication are in fact much more comprehensible than his actual observations and theories. These are situated in a language and theoretical framework that is largely incomprehensible to us today. Arguably this shows that while the emerging culture of 17th century Natural Philosophy is recognizably the same as that of modern science, the actual knowledge is lost to us as the Thought Collectives, to use Fleck’s language (7), have changed too radically.

The details of this idea that there is a recognizable scientific culture that persists over time, and provides sustainability and continuity to a community of practitioners require much work and are beyond the scope of this paper. If the idea is provisionally accepted then we must immediately ask the crucial question, what is it that makes this culture sustainable? Clearly this will be a mix of historical contingency, social context, and power relationships. But the central claim is that elements of the culture have contributed to that sustainability.

I want to suggest that one element that has contributed is a form of openness. It manifests historically in different ways but the valuing of observation, and of critique, the importance of effective communication and more recently efforts towards inclusion both in access to the outputs of research and influence over its conduct, can all be read as a valuing the testing of claims by exposing them across the boundaries of the community. We can use the rich literature on the nature of research communities, and their disciplinary splits and divisions, from Fleck (7), through Kuhn and Ravetz (8), but also to Latour (13) and Wave Two STS and indeed on to the work of Collins and Evans (11) on expertise in Wave Three, to understand how the culture of WSKS creates a myriad of hierarchical boundaries across which claims can be tested, while also driving interoperability across those boundaries by articulating shared values.

The Cultural Science framing suggests that Western Scientific Culture is doing two different things. Firstly at a high level, it creates interoperability though shared values. Secondly it drives the creation of new disciplinary groups at all scale levels creating boundaries across which knowledge claims can be tested. We can suggest that this culture, and at least some of the groups it has created, has thrived over time because it is well situated to creating productive conflict where groups interact. From the process of peer review, a managed form of a conflict in which one research group’s claims are tested by another, through to the insights that arise when whole disciplines clash as they come into contact, what emerges, as Ravetz noted is more abstracted, more general, and more widely used than what was initially created within the group.

My suggestion is that it is the various forms of openness that act to maximize the productivity of those conflicts. This is not to say that these values are perfectly enacted. As Wave Two STS tells us, scholars are embedded in social contexts and power structures laced with bias, assumptions and exclusions. Indeed, the tension between the necessary boundary work that defines the group, and the productivity of interactions that arise from relaxing those boundaries, is the key to understanding what is being created, what value it has, and to who.

4. The Economic and Political Sustainability of Knowledge Clubs

While I have sketched out an argument for explaining the sustainability of Western Scientific Culture as a whole, to examine the question of how institutions and groups operate we need to examine the sustainability of the overlapping and hierarchical groups that make up the larger deme. We use the term “Knowledge Clubs” (14) to refer these groups that have a commitment to generating knowledge with value beyond their boundaries, which is underpinned by these elements of openness.

The use of “clubs” is deliberate and has two motivations. Firstly, it emphasizes the tension between the definition of boundaries and the need to operate across them. Secondly it draws on the strand of economic theory that examines how groups can sustain the production of collective goods. The narrative for Knowledge Clubs within the WSKS is that knowledge is being created for the good of all. But such goods, Public Goods in economic terms, cannot support the sustainability of the club itself. This implies that the culture-made group is also capable of generating value, or utility, for the group itself.

Buchannan’s (15) work on the economic sustainability of clubs is central here. Buchannan identifies a class of goods that are neither public or nor private, but are important in sustaining groups. In modern terminology these are goods that are non-rivalrous (they can be shared out without diminishing them) but are excludable (it is easy to prevent non-group members from benefiting from them).

Where a group generates private goods (such as money) that are passed to individuals then engagement is easy to explain. If a group only generates public goods then a classic collective action problem ensues. Such a group can only be sustained if it is non-rational from an economic perspective. While this is by no means impossible – it can be argued that Wikimedia solves the collective action problem for public good creation of a free encyclopedia by relying on donations from non (economically) rational actors – evidence suggests this can only operate at the extremely large scales where a sufficiently large number of such actors can be found.

Clubs in Buchannan’s terms are sustained by this intermediate class of goods, which are termed club goods. I have previously argued that we can see knowledge as such a club good. Knowledge is created by and within groups. It is non-rivalrous, in Jefferson’s memorable language “…he who lights his taper at mine, receives light without darkening me”, but on its creation it is exclusive and excludable. Firstly, because it is only available to the group, but later the choices of how, and where to communicate it, what language to use, restrictions to access all create forms of exclusion.

We intuitively understand that knowledge held exclusively by a group, whether the scholars who originated it, or the community that subscribes for access to a specific – closed – journal, will not create as much value as it might. This is also consistent with the epistemological model sketched out above, where it is the process of exchange and translation amongst groups, which makes knowledge both more general and more valuable. We therefore have systems, including our systems of scholarly communication, in place that support the process of making knowledge more like a public good, removing various forms of exclusion piece by piece.

This process of investment in making club-good knowledge more public-like, a process of “public-making”, however raises the same collective action problem. Why would a Knowledge Club voluntarily give up a good, indeed invest in reducing the exclusivity that allows them to maintain control? Part of the answer is that we are actually quite selective about the modes of control we give up. Traditionally communication through a journal or a book is directed at and accessible (for many different meanings of the word) to a very select, and identifiably demic, group. Part of the answer is one of culture – and as we shall return to, values – that guide our practice as scholars.

Neither of these answers however will suffice for our economic framing. An economic framing suggests that the club is involved in an exchange where it gains something in return giving up exclusivity. That something must be a club or private good and there are in fact a range of these that can be identified. Some of these are quite abstract goods; recognition, prestige, and membership within disciplinary knowledge clubs. Some are much more concrete; jobs, professional advancement, and funding both for further research and personally.

5. An economic framing: Institutions as the underwriter of the public-making exchange

An important aspect of this exchange process to note is that the immediate benefits of the exchange are the more abstract and nebulous ones, recognition and attention. The more concrete, and more widely exchangeable goods take longer. These are individual benefits such as positions and salaries, and for demic groups recognition as a discipline and strand of scholarship that should be a visible part of a research institution. The coupling between public-making and these longer term benefits is something that we believe in. It a part of our culture. But from an economic perspective there is a distinct risk that the investment in public-making may not in fact pay off.

In financial terms these kinds of risks can be managed if there is an underwriter available. In the research community this underwriting is managed by institutions acting as a – partial – guarantor that the knowledge club’s investment in public-making will be convertible in an understood and predictable way into these concrete club and private goods. Institutions, both in the sense of research performing organizations such as universities, but also in the broader sense used by Ostrom (16) of “…the prescriptions that humans use to organize all forms of repetitive and structured interactions”, provide the assurances that support the risks of investing in public-making for the knowledge club.

There is, therefore tension at the heart of our institutions. Their purpose is (in part) to promote public-making, but they do this through acting as a guarantor in a transaction which provides excludable goods. The university itself is an exclusive club and needs to be to support the realization of benefits that arise from prestige and authority. To be predictable and therefore effective as guarantor institutions must necessarily be conservative in both the forms of public-making they support and recognize and in the rewards they award as a result of those activities. But to realize the full benefits of public-making they may need to be adaptable and even radical in a rapidly changing world.

Ostrom (17) showed that the way to understand institutions that resolve collective action problems is to see them as developing through a process of evolution. And that coordination at large scale required the development of hierarchical layers of organization. In turn the development of these layers provides stability and resilience to the system as a whole. All of this emphasizes that our institutions (in the sense of research organizations) should be expected to be resistant to change – should in fact be designed to be stable.

This analysis has implications that spread far beyond scholarly communications. In its role as a guarantor for the provision of club goods, which have as a core characteristic exclusivity, the institution is continually policing boundaries. This means working to protect the identity of the existing clubs, including their historical lack of diversity, it means policing the boundary of what counts as “scholarly” in terms of both work and outputs, and it means a focus on protecting existing and historical markers of prestige and authority.

As scholars we also reinforce this backwards looking boundary work whenever we rely on our research organizations to act as the guarantor of benefits that we exchange for public-making. Our continuing engagement with “traditional” modes of public-making and scholarly communication are both driven by our acceptance of the social contract we have with our institutions and act to reinforce that system.

As is often the case with economic arguments, this one appears to arrive at a profoundly depressing conclusion. Not only must we expect, indeed rely on, our institutions to be conservative, but this appears to open up a gaping hole between the harsh economics and the value of an open culture that the epistemological argument implies. The Cultural Science framing implies diversity is key to generalizing knowledge, whereas the economic argument seems inevitably to point to institutions that will slow the increase in diversity, both of activities and participants.

Arguably framing the opportunities presented by developing technologies as “new” forms of scholarly communication that are “different”, aligning ourselves with the oppositional discourse that Tkacz (4) describes, is counter productive. This offers a potential solution, that is to situate and to design these “new” practices as simply a more effective expression of old values. Successes in innovation in scholarly communication and open practice are often associated with small changes, with far superior but more radical opportunities often failing. Can we avoid the problems of conservatism, or at least speed up the uptake of new tools and practices, and also the oppositional discourse of openness by describing openness as an old value?

6. Framing Openness as a Core Value of the Academy

The economic analysis above paints a very harsh and transactional picture, but the reality is of course more complex. The institutions that are taking the role of guarantor spread beyond our research organizations to those broader “institutions” that are part of our research culture. Indeed, we can tie the sustainability of Western Science culture in part to its role in sustaining the cultural institutions that underwrite this exchange of knowledge. That is, our reliance on this exchange as scholars is underpinned by our self-identification as scholars, our identification with the demic group. It is deeply tied to the values that we hold. In these final sections I will argue that it is through a framing of openness as a value core to Western Science culture that we can both work for change within our institutions as well as enhance the diversity of our communities and therefore the value of the knowledge we create.

Shapin and Schaeffer (18) in their dissection of the historical conflict between Robert Boyle and Thomas Hobbes and the founding of the UK’s Royal Society describe Boyle as deploying three technologies. The three technologies; the material technology of the experiment, the literary technology of printing and dissemination, and a social technology – the scientific culture and institutions in our terms – that defined the interactions of scholars. These various technologies underpinned claims made by Boyle and other natural philosophers to openness and similar claims are made to this day. Openness to criticism and critique, openness to contributors from any place or walk of life, and openness through the accessibility of printing and disseminating accurate descriptions of the experiments.

Shapin and Schaeffer’s important contribution is to critically examine these claims and to show that in practice Boyle and others involved in defining and creating the culture and institutions of science that continues to this day fell a substantial distance short of their aspirations. Boyle sharply circumscribed what he would accept as legitimate criticism, claims and evidence from those of more noble birth were to be preferred over that from commoners, and access to the halls and demonstrations of the Royal Society were certainly not open to all. Indeed, it is only in the past 25 years that a ban on women (at least those who are not Fellows) entering the headquarters of the Royal Society was lifted.

Here we see again exactly the tension that has played out through this discussion. A claim of openness, and a narrative that this openness sits at the core of the value system, that is not quite realized in practice. The building of institutions that seek to enhance openness – the Royal Society holding formalized meetings, open to members, in the place of private demonstrations – that are nonetheless exclusive. Membership of the club, whether the Royal Society or other National Academies, has always been a marker of prestige and authority, even as the actual criteria for membership have changed radically over the years. Yet what is passed down to us today, is less that exclusive gentleman’s club and more the core values that it sought to express.

Move forward 200 years from the 17th to the mid-19th century and a debate was raging in the United Kingdom about who could contribute to the conduct of science. Lightman (19) reveals what might appear to our 21st century eyes as a startlingly modern debate on the interest “that not alone scientific readers, but those of every class, […] to approach the source from whence this species of knowledge is derived”. Lightman describes the growth of popular science journals to meet this demand. It is perhaps a sign of the strength of the tension we are discussing that the most visible survivor of this growth is the journal Nature which has been so entirely co-opted by our scholarly culture as an institutional signal of internal club prestige, that it can stand symbolically for the entire system of journal hierarchies.

In an illustration that progress is clearly not linear Lightman also discusses the positioning of Darwin – whose beard today stands as a (not particularly inclusive) symbol of a professional scientist – as a demonstration that amateurs can contribute to science. Lightman quotes Grant Allen, a 19th century popularizer of science describing Darwin as “merely an amateur, a lover of truth, who was impelled by curiosity”. The professionalization of the academy through the 20th century alongside the celebration of Darwin as a key figure in the history of science seems to have necessitated an assumption of his place as a “real scientist”. If we are to aspire to be part of the club that included Darwin then we must necessarily place him in that club. Arguably this was a backwards step in a trajectory of gradually implementing greater openness. Lightman notes that the “…appropriation of […] Darwin as [an] iconic figure[…] served to undermine the participatory ideal of the 19th-century popularizers and reflected the increasing power of professionalization”. That is, the evolution of the professionalized institutions, that stabilize and allow the scaling of the culture of Western Science created exclusion, even in the way that we create and describe iconic figures.

It would be straightforward to follow the gradual opening up of aspects of our institutions and culture through the 20th and 21st century. Examples could be given from increasing access to tertiary education, the public funding of research, through open access, the shift from “public understanding of science” through “public engagement” to “responsible research”, to issues of data availability and citizen science. However my point is to establish the deep roots of this agenda. Despite, or even in some cases because of, the limitations in putting it into practice, the idea that critical contributions to scholarship will come from outside has persisted. Indeed a case for the inverse can be made, that the culture of Western Science has persisted precisely because a commitment to openness, to public-making, is one of its core values.

7. An Aspiration to Openness as a Conservative Position

I began by noting that openness refers to many different things, and that as many others have noted, that the narrative associated with this variety is frequently one of new-ness, of technological possibilities, and of opposition to a status quo. As Tkacz notes this can lead to a cyclic inevitability as openness eats itself and becomes the new status quo, the new establishment.

I want to flip this on its head. In Boyle’s writings we see the concern for completeness of description, for reproducibility and for a commitment to observations, wherever they come from as the final arbiters. In the 19th, and again in the 20th and 21st centuries we see movements arise in which contributions are sought from anyone. In Merton’s norms (10) of communalism and universalism, Popper’s conception of falsifiability (20), and Kuhn’s idea that scientific revolutions are precipitated by the build up of external information (9), even in Latour’s model for the gradual expansion of the collective (13) we see repeated attempts to articulate the importance of openness to claims and ideas from the outside as a core part of the social activity of science.

Clearly this value is quietly ignored at least as frequently as it is found in practice, but the aspiration is a common thread. Indeed the institutionalization of imperfection may be critical in solving the economic problem of sustaining knowledge-making clubs that choose to invest in public-making. The argument made here has only provided the barest sketch of how Knowledge Clubs interacting may be engaged in both economic exchanges and productive general-knowledge producing conflict. If the most significant insights come from across boundaries then the boundaries themselves are also of value. A deeper analysis may provide a route to identifying the ways in which this tension can be managed both to create value in the economic sense and to maximize the public-good nature of generalized knowledge.

It is therefore the aspiration to openness, and its adoption as element of the identity and core values of the researcher, its centrality to our culture, that provokes us to attempt to move across boundaries and to create knowledge. That “full openness” or “total inclusion” can never be achieved is the consequence of an imperfect world. The aspiration to seek it still has value. In this sense the argument aligns with the claims for Elective Modernism made by Collins and Evans (11). However my conclusion is diametrically opposed.

Collins and Evans state that the scientific community must be protected so that its value system, its culture in our terms, can operate without disturbance. I argue here that disturbance is fundamental to its function, that the process of generalizing knowledge requires that new efforts are constantly made to break down barriers and reduce exclusion. Nonetheless the institutions that underwrite the exchanges fundamental to public-making do need protection. Understanding how they can change at an optimal pace remains a challenge.

Part of the answer may lie in the problem. It may be that an argument can be made that this tension is fundamental, that progress towards greater openness is a return to core values, that such progress must underpin any claim of real progress arising from Western Science. In that sense situating openness as a profoundly conservative position may be a viable political move. In the end the answer is not that openness is any one thing, it is that it is many different expressions of one underlying process. That it proceeds through cycles of change, institutionalization and reaction is then unsurprising. And if that is correct then we can start to pull the threads together that will allow us not merely to respond the institutions and culture that we have as they evolve around us, but to design them.

If this is true then we are perhaps living in a time of unprecedented opportunity for science and for scholarship. There are profound challenges to adapting our institutions to interact productively with differing knowledge systems, but we are perhaps for the first time well placed to do so. By understanding how tension between boundary work – and its exclusionary tendencies – and the value of diverse perspectives we may be able to improve, by design, on our institutions. If we can develop a narrative thread within our culture that this is merely the extension of an ongoing process that has served the academy well, then we arguably make this gradual and high imperfect progress a highly conservative position. This may offer us the best opportunity to accelerate the progress we are making on access, inclusion, and diversity and build a more generally valuable, and accessible knowledge system that truly includes the insights and perspective of those beyond the walls of the academy.

8. References

1. Fecher B, Friesike S. Open Science: One Term, Five Schools of Thought. In: Bartling S, Friesike S, editors. Opening Science [Internet]. Heidelburg: Springer; 2014 [cited 2016 Jan 19]. p. 17–47. Available from: http://papers.ssrn.com/abstract=2272036
2. Pomerantz J, Peek R. Fifty shades of open. First Monday [Internet]. 2016 Apr 12 [cited 2017 Jan 16];21(5). Available from: http://firstmonday.org/ojs/index.php/fm/article/view/6360
3. Coleman G. The Political Agnosticism of Free and Open Source Software and the Inadvertent Politics of Contrast. Anthropological Quarterly. 2004 Sep 3;77(3):507–19.
4. Tkacz N. From open source to open government: A critique of open politics | ephemera. ephemera: theory & politics in organization. 2012;12(4):386–405.
5. Holbrook JB. We Scholars: How Libraries Could Help Us with Scholarly Publishing, if Only We’d Let Them. In: Bonn M, Furlough M, editors. Getting the Word Out: Academic Libraries as Scholarly Publishers. ACRL; 2015. p. 43–54.
6. Hartley J, Potts J. Cultural Science. London ; New York: Bloomsbury 3PL; 2014. 266 p.
7. Fleck L. Genesis and Development of a Scientific Fact. New edition edition. Trenn TJ, Merton RK, editors. Chicago u.a: University of Chicago Press; 1981. 232 p.
8. Ravetz J. Scientific Knowledge and Its Social Problems. Reprint edition. Oxford: Oxford University Press; 1971. 449 p.
9. Kuhn TS. The Structure of Scientific Revolutions. 2nd Revised edition edition. Chicago, Ill: University of Chicago Press; 1970. 222 p.
10. Merton R. Science and technology in a democratic order. Journal of Legal and Political Sociology. 1942;1:115–26.
11. Collins H, Evans R. Why Democracies Need Science. Cambridge, UK ; Malden, MA: Polity Press; 2017. 200 p.
12. Boyle R. A defence of the doctrine touching the spring and weight of the air propos’d by Mr. R. Boyle in his new physico-mechanical experiments, against the objections of Franciscus Linus ; wherewith the objector’s funicular hypothesis is also examin’d, by the author of those experiments. Oxford: Thomas Robinson; 1662.
13. Latour B. Politics of Nature: How to Bring the Sciences into Democracy. Cambridge, Mass: Harvard University Press; 2004. 320 p.
14. Potts J, Hartley J, Montgomery L, Neylon C, Rennie E. A Journal is a Club: A New Economic Model for Scholarly Publishing [Internet]. Rochester, NY: Social Science Research Network; 2016 Apr [cited 2017 Apr 8]. Report No.: ID 2763975. Available from: https://papers.ssrn.com/abstract=2763975
15. Buchanan JM. An Economic Theory of Clubs. Economica. 1965;32(125):1–14.
16. Ostrom E. Understanding Institutional Diversity. Princeton: Princeton University Press; 2005. 376 p.
17. Ostrom E. Governing the Commons: The Evolution of Institutions for Collective Action. unknown edition. Cambridge ; New York: Cambridge University Press; 1991. 298 p.
18. Shapin S, Schaffer S. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. With a New introduction by the authors edition. Princeton, N.J: Princeton University Press; 1985. 448 p.
19. Lightman B. Popularizers, participation and the transformations of nineteenth-century publishing: From the 1860s to the 1880s. Notes Rec. 2016 Dec 20;70(4):343–59.
20. Popper KR. The logic of scientific discovery. London: Hutchinson and Co; 1959.

My normal practice is that nothing posted here has been seen or vetted by others. This post is a departure from that because I think it is important enough to justify whatever extra reach I can give it. This text, which has just been sent out to FORCE11 Members by email, was seen by the FORCE11 Board of Directors and the linked statement was approved by a board vote.

Dear Colleagues

Re: FORCE11 Board of Directors Statement on Restrictions to Immigration

FORCE11 works towards the goal of being a global platform that brings communities together to discuss challenging issues. These communities focus on the methods that underpin how scholars work, how we define and validate what we consider to be true, and how the credit and attribution for insight is assigned. Our business is seeking consensus across diverse communities with differing agendas and interests. At this time I believe this work to be more important than ever.

Like any organisation with its roots in the traditional North Atlantic centres of western scholarship our claim to global reach is an aspiration. I recognise this and seek to continually improve our work on the historical inequities of access to and inclusion in scholarship including exclusion on the basis of factors including, but not limited to, geography, social conditions, professional status, race, religion, sexual orientation, gender identity, citizenship and national origin.

It is still the case the most productive conversations happen face to face. This is especially true of efforts pursued within FORCE11. The history of the organisation is one of bringing together diverse groups to examine and solve problems. Exclusionary limitations on travel of groups of people specified on any basis is totally inimical to our goals and purpose.

It is for this reason that the Executive Board of FORCE11 has today released a statement that rejects policies that exclude immigrants on the basis of belonging to a group defined by race, religion, national origin or other criteria. This includes the recent White House Executive Order.

Objecting to policy is not enough however. I will be working with the Board and with our Working Groups and contributors to find ways in which we can limit the effect of exclusionary policies. Amongst other things I want to work across the full set of our activities to select venues and platforms that are, as a whole, inclusive of as wide as a group of global participants as possible. This will include considering restrictions to entry, cost of travel, financial risks, physical access and safety of participants. It is important to note that current policies both limit entry to some countries by some participants but also have the effect of making it impossible for some participants to leave their country of residence.

At the core of our work at FORCE11 is also exploring technology and social practice for scholarly communications. We will explore technology and design options for practical and effective multi-site meetings and workshops in the future. In all of this we welcome collaboration with like-minded organisations to work together, to reflect on our own practices, and to develop shared practice to support inclusive scholarly meetings and communications more generally.

Good scholarship has at its core, recognising and listening to valid criticism. I know that inclusion is something we need to continue to work on as an organisation, that any organization needs to continue to work on. I welcome comments, criticisms, and advice on how FORCE11 can work more effectively, and how we can work together, to pursue these goals.

Yours sincerely

Cameron