Category: Blog

Posted on

The Political Economics of Open Access Publishing – A series

Victory Press of Type used by SFPP
Victory Press of Type used by SFPP (Photo credit: Wikipedia)

One of the odd things about scholarly publishing is how little any particular group of stakeholders seems to understand the perspective of others. It is easy to start with researchers ourselves, who are for the most part embarrassingly ignorant of what publishing actually involves. But those who have spent a career in publishing are equally ignorant (and usually dismissive to boot) of researchers’ perspectives. Each in turn fail to understand what libraries are or how librarians think. Indeed the naive view that libraries and librarians are homogenous is a big part of the problem. Librarians in turn often fail to understand the pressures researchers are under, and are often equally ignorant of what happens in a professional publishing operation. And of course everyone hates the intermediaries.

That this is a political problem in a world of decreasing research resources is obvious. What is less obvious is the way that these silos have prevented key information and insights from travelling to the places where they might be used. Divisions that emerged a decade ago now prevent the very collaborations that are needed, not even to build new systems, but to bring together the right people to realise that they could be built.

I’m increasingly feeling that the old debates (what’s a reasonable cost, green vs gold, hybrid vs pure) are sterile and misleading. That we are missing fundamental economic and political issues in funding and managing a global scholarly communications ecosystem by looking at the wrong things. And that there are deep and damaging misunderstandings about what has happened, is happening, and what could happen in the future.

Of course, I live in my own silo. I can, I think, legitimately claim to have seen more silos than the average; in jobs, organisations and also disciplines. So it seems worth setting down that perspective. What I’ve realised, particularly over the past few months is that these views have crept up on me, and that there are quite a few things to be worked through, so this is not a post, it is a series, maybe eventually something bigger. Here I want to set out some headings, as a form of commitment to writing these things down. And to continuing to work through these things in public.

I won’t claim that this is all thought through, nor that I’ve got (even the majority of) it right. What I do hope is that in getting things down there will be enough here to be provocative and useful, and to help us collectively solve, and not just continue to paper over, the real challenges we face.

So herewith a set of ideas that I think are important to work through. More than happy to take requests on priorities, although the order seems roughly to make sense in my head.

  1. What is it publishers do anyway?
  2. What’s the technical problem in reforming scholarly publishing
  3. The marginal costs of article publishing: Critiquing the Standard Analytics Paper and follow up
  4. What are the assets of a journal?
  5. A journal is a club: New Working Paper
  6. Economies of scale
  7. The costs (and savings) of community (self) management
  8. Luxury brands, platform brands and emerging markets (or why Björn might be right about pricing)
  9. Constructing authority: Prestige, impact factors and why brand is not going away
  10. Submission shaping, not selection, is the key to a successful publishing operation
  11. Challenges to the APC model I: The myth of “the cost per article”
  12. Challenges to the APC model II: Fixed and variable costs in scholarly publishing
  13. Alternative funding models and the risks of a regulated market
  14. If this is a service industry why hasn’t it been unbundled already (or where is the Uber of scholarly publishing?)
  15. Shared infrastructure platforms supporting community validation: Quality at scale. How can it be delivered and what skills and services are needed?
  16. Breaking the deadlock: Where are the points where effective change can be started?
Posted on

Who’s in the Club? New frames for understanding knowledge sharing

English: Venn diagram (coloured)
Venn diagram (Photo credit: Wikipedia)

The following is a version of the text I spoke from at the STEPS 2015 Conference, Resource Politics, at a session on Open Science organised by Valleria Arza, where I spoke along with Ross Mounce and Cindy Regalado. This version is modified slightly in response to comments from the audience.

There aren’t too many privileged categories I don’t fall into. White, male, middle class, middle aged, home owner. Perhaps the only claim I could make in the UK context is not having a connection with Oxbridge. The only language I speak is English and I’ve never lived in a non-english speaking country, never lived outside of Australia or England in fact. What do I have to say about developing countries? Or transitional, or emerging or peripheral…all problematic terms rooted in a developed world, western, narrative.

I hope it is more than just a hand-wringing liberal response. I suspect all of us do, genuinely base our work on a conviction that we can make a difference for good. In this context we collectively  believe that the critical tradition of scholarship developed in Western Europe can bring benefits to disadvantaged groups. And after all the resources to actually take action are in our hands or within our gift to influence. Something must be done. We can do something. Therefore we must do it.

Obviously this is an old critique and one that has shaped  decisions about how to act. We seek to move beyond charity and paternalistic intervention to offering frameworks and taking consultative approaches. To requiring a deeper understanding of context. In my own work I’ve tried to focus on offering ideas about processes and implementation, not on what should be done. But my ideas are of course still trapped within the frameworks I work within.

Central to those frameworks for me is “Open”. I’m not going to seek to explain in detail what I mean by “open”. Nor am I going to provide a critical analysis of the issues it raises in a development context, or its dependence on western liberal democratic, or neoliberal, or even libertarian values. Others are better placed to do that. What I do want to propose is that “Open” in the sense that I mean it is a culture, and it is a culture deeply rooted in its particular (north) western historical context.

If you accept that Open is a culture then our traditional thinking would be that it is the product of a particular community. Or rather communities. Again the history is complex but we can identify a complex of groups, clubs if you like, that have grown up with their own motivations and agendas but have sufficient alignment that we can collect them together under the label of “Open”. “Open Source”, “Open Data”, “Open Access”, “Open Science”, but also perhaps “Open Government”, transparency and others. Often these groups are identified with a charismatic individual.

John Hartley and Jason Potts in their book Cultural Science, propose a shift in our usual way of thinking about these groups and their cultures, that is both subtle and to my mind radical. We would usually think of individuals coming together to form groups in a common interest (often framed as an political-economic analysis of the way the collective resources of the group combine to achieve action). The individuals in the group and their values combine to define the culture of the group.

Hartley and Potts invert this. Their claim is that it is culture that creates groups. This inversion, whether you take it at face value as a real description of causation, or simply as a useful way to reframe the analysis has an important consequence. It focuses the unit of analysis onto the group rather than the individual. Rather than asking how individual behaviour lead to the consequences of groups interacting, we ask how cultures do or do not align, reinforce or cancel out.

In the session at the Resource Politics Conference on Tuesday on Assessment of Assessments we heard how governance reflects the structural moves allowed within an institution and how the framing of a problem reflects (or creates) these structures. Martin Mahony spoke of certain framings as “colonising spaces” which I would in turn appropriate as an example of how adaptive cultural elements can be spread through their re-creation or co-creation by groups.

In any case take your pick. New model of how culture, groups, society and sociotechnical institutions  that they co-create actually work and evolve, or a different framing that lets us tackle interesting problems from a new perspective. Either way, is it helpful? And what does it have to do with “development” or “sustainability” or “open” for that matter?

I think its useful (and relevant) because it lets us take a new view on the status of knowledge created in different contexts and it provokes some new ways of asking what we should do with the resources that we have.

First it gives us a license to say that some forms of knowledge are simply incompatible, growing as they do out of different cultures. But crucially it requires us to accept that in both directions – forms of knowledge from other cultures that are inaccessible to us, but also that our knowledge is accessible to others. It also suggests that some forms of compatibility may be defined through absence, exclusion or antagonism.

An anecdote: A few years ago I was working on a program focussing on Scholarly Communication in Sub-Saharan Africa. One striking finding was the way the communities of scholars, in disciplines that traditionally don’t communicate across groups, were actively engaging on social media platforms. Researchers from Russia, Iran, Iraq, Pakistan, India, Madagascar, Zimbabwe, Brazil, and Chile were all discussing the details of the problems they were facing in synthetic chemistry, a discipline that in my world is almost legendary for its obsessive individualism and lack of sharing. They shared the language of chemistry, and of English as the lingua franca, but they shared that with the absent centre of this geographical circle. Their shared culture was one of exclusion from the North Western centre of their discipline.

And yet, our culture, that of western scholarship was still dominant. I was struck yesterday in the “Assessment of Assessment” session focussing as it did on questions of transparency, engagement, and above all framing, that the framing of the session itself was not interrogated. Why, in an area focussed on building an inclusive consensus, is the mode of communication one of individual experts at the centre (as I am here on the podium) with questions to be asked, when allowed, from the periphery (you in the audience)?

Worse than that the system these chemists were using, ResearchGate is a western commercial infrastructures built from the classic Silicon Valley mindset, seeking to monetise, to in many ways to colonise, the interactions of these scholars who define themselves precisely through opposition to much of that culture. Is it possible to build a system that would help this group communicate within the scope of their culture but that doesn’t impose assumptions of our western culture? What infrastructures might be built that would achieve this and how would they be designed?

For Hartley and Potts the group, this level of analysis is one defined by shared culture. And of which cultures support groups which support the dynamic co- and re-creation of that culture. So another way of approaching this is to view them through the lens of the economics of clubs. What makes a club viable and sustainable? What goods does it use to achieve this? This group economics focus is interesting to me because it challenges many of our assumptions about the politics and economics of “Open”.

Rather than adopt a language of nationalisation of private goods: you journal publisher must give up your private property (articles) and transfer them to the public, you researcher must share your data with the world; we ask a different question – what is the club giving up and what are they gaining in return? Knowledge in this model is not a public good, but rather a club good – there is always some exclusion – that we are seeking to make more public through sharing. The political/economic (or advocacy) challenge is how to create an environment that tips the balance for clubs towards knowledge sharing.

These two questions – how can we support peripheral communities to co-create their own cultures without imposing ours and how we might change the economics of knowledge systems to favour investment in sharing – lead for me to an interesting suggestion and a paradox. What enabling infrastructures can be built and how can we make them as neutral and inclusive as possible while simultaneously embracing that anything built with western resources will be framed by our own cultures?

My stance on this is a re-statement of the concept from Zittrain, Benkler, Shirky and others that networks at scale can deliver new kinds of value. That the infrastructures we seek to build can tip the balance towards club investment in sharing if they provide mechanisms for clubs to gain access to networks. This is an architectural principle, that we can take a step up (or down if you prefer) identifying the common aspects of functionality required. It is also not new.

The new step is to adopt a principle of cultural engagement in governance, a means of – in the language of this conference – aligning the institutions that provide infrastructures and their governance and structures with the maximum possible number (and not power, not centrality) of cultures. The the criteria we use is one of maximising the number of productive interactions between cultures through the platforms we provide.

And this is what brings us back to Open, to what for me is the core of the philosophy, value system, or culture of Open Practice. Not that sharing outwards to the public is the target in itself but that it is through sharing that we create new opportunities for interaction and it is being open to contributions, to productive interactions that in my old world view creates value, but in this new framing promotes the “clashes of culture” that create new knowledge.

Posted on

Researcher as victim. Researcher as predator.

English: Illustration of a leopard and cheetah
English: Illustration of a leopard and cheetah (Photo credit: Wikipedia)

Researchers for the most part are pretty smart people. At the very least they’ve managed to play the games required of undergraduate and post graduate students, and out-competed a substantial proportion of other vying for the same places. Senior academics have survived running the gauntlet of getting published, and getting funded, at least enough to stay in the race.

It has been observed that when smart people do dumb things it is worth looking closer. The dumb thing is usually being done for a smart reason. Indeed we might go one step further and suggest that where a system is populated largely by smart people the proportion of dumb things they are doing could be a good diagnostic of how good the system is at doing what it is meant to do, as opposed to what it is measured to do. On this basis we might wonder about the health of many universities.

We also know there are a few issues with our systems of scholarly communications. And some of these involve unscrupulous and deceitful players out to reap a commercial gain. The story of so called “predatory” journals is a familiar one, which usually focusses on “publishers” claiming to offer open access services, but more recently an article in the Guardian took a similar stance on traditional academic monograph publishers.

In both cases the researcher is presented as a hapless victim, “hoodwinked” as the headline states into parting with money (either directly in the form of APCs or indirectly through their libraries). But really? I’ve no intent to excuse the behaviour of these publishers, but they are simply serving a demand. A demand created by researchers under immense pressure to demonstrate their productivity. Researchers who know how to play the game.

What is a line on a CV worth? Does it make that grant a little more likely? Does it get you past the magic threshold to get on the applicant short list? Is there a shortcut? Researchers are experts at behaviour optimisation and seeing how systems work. I simply don’t buy the “hapless victim” stance and a lot of the hand wringing is disingenuous at best. On a harsh economic analysis this is perfectly rational behaviour. Smart people doing dumb things for smart reasons.

The expansion of journal lists, the increasing costs to libraries, and the ever expanding list of journals that would take just about anything were never perceived as a problem by researchers when they didn’t see the bills. Suddenly as the business model shifts and the researcher sees the costs the arms are going up. The ever dropping circulation (and ever rising prices) of monographs was never really seen as a problem until the library budgets for monographs started to disappear as the serials crisis started to bight.

The symptoms aren’t restricted to dodgy publishing practices of course. Peer review cartels and fake reviewers result from the same impulse, the need to get more stuff published. Paper mills, fake journals, secondary imprints that will take any book proposal, predatory OA and bottom feeding subscription journals are all expressions of the same set of problems. And the terrifying thing is that responsible publishers are doing a pretty good job of catching a lot of it. The scale of the problem is much, much greater than is obvious from the handful of scandals and a few tens of retractions.

At times it is tempting to suggest that it is not publishers that are predatory, but researchers. But of course the truth is that we are all complicit, from publishers and authors producing content that no-one reads, through to administrators counting things that they know don’t matter, and funders and governments pointing to productivity, not to mention secondary publishers increasing the scope of they indices knowing that this leads to ever increasing inflation of the metrics that makes the whole system go round.

We are all complicit. Everyone is playing the game, but that doesn’t mean that all the players have the same freedom to change it. Commercial suppliers are only responding to demand. Governments and funders can only respond to the quality assessments of the research community. It is only the research community itself that can change the rules. And only a subset of that.

Emerging researchers don’t have the power to buck the system. It is senior researchers, and in particular those who mediate the interface between the sources of funding and the community, the institutional leaders, Vice-Chancellors, Presidents, Deans and Heads of Department. If institutional leaders chose to change the game, the world would shift tomorrow.

Scott Edmunds perhaps summed it up best at the FORCE2015 meeting in Oxford:

It is no longer the case that people are gaming the system, the system has become a game. It’s time to say Game Over.

If we cast ourselves as mere victims we’ll never change the rules. The whole narrative is an excuse for doing nothing.

This post was prompted in large part by tweets from Martin Coward and Nancy Sims

Posted on

What exactly is infrastructure? Seeing the leopard’s spots

This is a photo of a black leopard from the Ou...
Black leopard from Out of Africa Wildlife Park in Arizona (Wikipedia)

Cite as: What exactly is infrastructure? Seeing the leopard’s spots. Geoffrey Bilder, Jennifer Lin, Cameron Neylon. figshare. http://dx.doi.org/10.6084/m9.figshare.1520432

We ducked a fundamental question raised by our proposal for infrastructure principles: “what exactly counts as infrastructure?” This question matters. If our claim is that infrastructures should operate according to a set of principles, we need to be able to identify the thing of which we speak. Call the leopard, a leopard. Of course this is not a straightforward question and part of the reason for leaving it in untouched in the introductory post. We believe that any definition must entail a much broader discussion from the community. But we wanted to kick this off with a discussion of an important part of the infrastructure puzzle that we think is often missed.

In our conversations with scholars and others in the research ecosystem, people frequently speak of “infrastructures” when what they mean are services on top of deeper infrastructures. Or indeed infrastructures that sit on deeper infrastructures. Most people think of the web as an essential piece of infrastructures and it is the platform that makes much of what we are talking about possible. But the web is built on deeper layers of infrastructure: the internet, MAC addresses, IP, and TCP/IP. Things that many readers will never even have heard of because they have disappeared from view. . Similarly in academia, a researcher will point to “CERN” or “Genbank” or “Pubmed” or “The Perseus Project” when asked about critical infrastructure. The Physicists at CERN have long since taken the plumbing, electricity, roads, tracks, airports etc. that make CERN possible for granted.

All these examples involve services operating a layer above ones we have been considering. Infrastructure is not commonly seen. That lower level infrastructure has become invisible just as the underlying network protocols, which make Genbank, PubMed and the Perseus Project possible have also long since become invisible and taken for granted. To put a prosaic point on it, what is not commonly seen is also not commonly noticed. And that makes it even more important for us to focus on getting these deeper layers of infrastructure right.

If doing research entails the search for new discoveries, those involved are more inclined to focus on what is different about their research. Every sub-community within academia tends to think at a level of abstraction that is typically one layer above the truly essential – and shared – infrastructure. We hear physicists, chemists, biologists, humanists at meetings and conferences assume that the problems that they are trying to solve in online scholarly communication are specific to their particular discipline. They say “we need to uniquely identify antibodies” or “we need storage for astronomy data” or “we need to know which journals are open access” or “how many times has this article been downloaded”. Then they build the thing that they (think they) need.

This then leads to another layer of invisibility – the infrastructures that we were concerned with in the Principles for Open Scholarly Infrastructure are about what is the same across disciplines, not what is different. It is precisely the fact that these common needs are boring that means they starts to disappear from view, in some cases before they even get built. For us it is almost a law: people tend to identify infrastructure one layer too high. We want to refocus attention on the layer below, the one that is disappearing from view. It turns out that a black leopard’s spots can be seen – once they’re viewed under infrared light.

So where does that leave us? What we hear in these conversations across disciplines are not the things that are different (ie., what is special about antibodies or astronomy data or journal classifications or usage counts) but what is in common across all of these problems. This class of common problems need shared solutions. “We need identifiers” and “we need storage” and ”we need to assign metadata” and “we need to record relationships”. These infrastructures are ones that will allow us to identify objects of interest (ex: identify new kinds of research objects), store resources where more specialised storage doesn’t already exists (ex: validate a data analysis pipeline), and record metadata and relationships between resources, objects and ideas (ex: describing the relationships between funders and datasets). For example, the Directory of Open Access Journals provides identifiers for Open Access journals, claims about such journals, and relationships with other resources and objects (such as article level metadata, like Crossref DOIs and Creative Commons license URLs).

But what has generally happened in the past  is that each group re-invents the wheel for its own particular niche. Specialist resources build a whole stack of tools rather than layering on the one specific piece that they need on an existing set of infrastructures. There is an important counter-example: the ability to easily cross-reference articles and datasets as well as connect these to the people who created them. This is made possible by Crossref and Datacite DOIs with ORCID IDs. ORCID is an infrastructure that provides identifiers for people as well as metadata and claims about relationships between people and other resources (e.g., articles, funders, and institutions) which are in turn described by identifiers from other infrastructures (Crossref, FundRef, ISNI). The need to identify objects is something that we have recognised as common across the research enterprise. And the common infrastructures that have been built are amongst the most powerful that we have at our disposal. But yet most of us don’t even notice that we are using them.

Infrastructures for identification, storage, metadata and relationships enable scholarship. We need to extend the base platform of identifiers into those new spaces, beyond identification to include storage and references. If we can harness the benefits on the same scale that have arisen from the provision of identifiers like Crossref DOIs then the building of new services that are specific to given disciplines will become so much easier.  In particular, we need to address the gap in providing a way to describe relationships between objects and resources in general. This base layer may be “boring” and it may be invisible to the view of most researchers. But that’s the way it should be. That’s what makes it infrastructure.

It isn’t what is immediately visible on the surface that makes a leopard a leopard, otherwise the black leopard wouldn’t be, it is what is buried beneath.

[pdf-lite]

Posted on

Next Steps…Joining the Centre for Culture & Technology at Curtin

English: Workhouse memorial, Banbridge (5 of 7...
Workhouse memorial, Banbridge (Photo credit: Wikipedia)

A question I have got quite frequently over the last few months is “…no but seriously, what are you actually doing…”. The answer has got more precise bit by bit but I truly didn’t leave PLOS with a specific plan for world domination. What I did want to do was find a way to combine a return to a hard core research focus with projects that really are very practical. I guess its the same problem with all research, how to combine theory with practice.

As a first step towards finding that perfect balance I’m very happy to be joining the Centre for Culture and Technology at Curtin University as Professor of Research Communications. This is really exciting for a number of reasons. First and most importantly the Centre is developing a research program around research communications that cuts right to the core of my current interests, how can communities more effectively leverage the tools that the internet and web provides, to communicate scholarship. Expect more on this in the next months.

Second there is the exceptional team at CCAT that links deep expertise in cultural studies, digital and new media, practical implementation of open access models beyond the journal, visualisation, as well as a whole range of experience in indigenous culture, a space in which challenges deeply any easy claim of objectivity coming from the sciences. Which links to the third reason this is exciting. The post is in a humanities department, perhaps on the surface a strange place for someone with a science background to land, but for me another logical step towards looking for new perspectives and new tools to help illuminate what are fundamentally problems of cultural change.

The post is 50% so I will continue to be working on other projects, both analysis and technology, and I will remain in the UK. I will certainly be in Australia more often but my role is in part to act as a connection from CCAT into the wider networks of scholarly communications communities globally. So that elusive balance remains to be fully struck, but now I have a great anchor to work from.

 

 

 

Posted on

Costly signalling in scholarly communications

Male Blue Peacock in Melbourne Zoo, Australia.
Male Blue Peacock in Melbourne Zoo, Australia. (Photo credit: Wikipedia)

For a long time it was difficult for evolutionary biology to make sense of a (male) peacock’s tail. Clearly it is involved in courtship but the investment in growing it, and the disdvantage of carrying it around, would seem to be a disadvantage over all. The burden of the tail might be worth it for a single male if female preferences are fixed

Fisher found a solution to this problem by noting that the genes for large tails in male peacocks would tend to be carried along with the genes for a preference for males with large tails expressed in females. In combination these two traits can cause a run away selection process which could explain the extravagant display in many animals.

Zahavi proposed another solution in which the display is a form of “costly signalling”. The ability to invest in the production of a large tail demonstrates the health or fitness of the animal. For this to work the signalling must be costly and it must be difficult to fake. Coloured plumage in the presence of stalking predators implies speed and agility, large horns (or simply size) a sufficient ability to obtain food.

Hartley and Potts in their book Cultural Science (chapter 3) apply the idea of costly signalling to question of cultural evolution. They suggest that cultures will adopt forms of costly signalling to create within-group trust and cohesion. In turn cultural norms of truth-telling and even traditions of narrative (the assumption of sympathy for the ‘white hat’, the presentation of compromises as ‘necessary’, that even bad acts reveal the underlying goodness of the hero) build community and in extremis send members of that community out to die for it in battle. This is not a facile claim about “group evolution” or how genetic evolution might drive culture but part of a program to understand how culture itself evolves.

One of the challenges of understanding peer review in the scientific community is why we do it at all. It is a part of our culture but it is very hard to demonstrate how and where it contributes value. The humanistic approach to the empirical challenge to value is to respond that it is a cultural norm that defines the scholarly community. Even if peer review achieved nothing it would have value as a means of defining a community, the community that has a cultural dedication to peer review. The “we”, the culture that valuesand engaged with peer review, is defined in terms of its different from the “they” who do not. This form of identification reinforces the analogy both with Fisher (we select those who share culture) and Zahavi (the costly signalling of engaging in peer review is part of the creation of our scholarly culture).

So perhaps another way to look at engaging with peer review is as costly signalling. The purpose of submitting work to peer review is to signal that the underlying content is “honest” in some sense. In the mating dance between researchers and funders or researchers and institutions the peer review process is intended to make the pure signalling of publication and to make it harder to fake. Taking Fisher’s view of mutual selection, authors on one side, funders and instiutions on the other, we can see, at least as analogy, a reason for the run away selection for publishing in prestigious journals. A runaway process where the signalling bares a tenous relationship with the underlying qualities being sought, in the same way as the size of the peacock’s tail has a tenous link with its health and fitness.

But as Martin Eve has argued (Open Access in the Humanities, Chapter 2), we need such signals. The labour of detailed assessment of all research for the full range of desirable qualities is unaffordable. Summaries and signals are needed. The question, perhaps, is whether this costly signalling is as honest as it could be. Is it creating a sustainable culture and community with a solid base? The apparent rise in fraud in retractions, particularly amongst those high prestige publications, suggests that this is a question that should be seriously addressed. To stretch the biological analogy, has a gene for faked tails emerged? Such fake display is not uncommon in biology.

Addressing that question means asking questions about what the underlying qualities we desire are. That’s an important question which I’ve raised elsewhere but I don’t want to go down that route here. I want to explore a different possibility. One that arises from asking whether a different form of signalling might be possible.

Communicating research in a reproducible (or replicable, or generalizable, the semantics are also an issue for another time) fashion is hard work. Many of us have argued that to enable greater reproducibility we need to provide better tools to reduce that cost. But what if the opposite were true? What if the value actually lies precise in the fact that communicating reproducibility is costly but is also potentially a more honest representation of what a community values than publication in a high profile journal.

If you buy that argument then we have a problem. The sexual selection run away is hard to break out of, at least in the case of biological evolution. At some point survivability prevents tails or horns growing so big they overbalance the animal, but by that stage a huge and unnecessary investment has been made. However in the case made by Potts and Hartley the thing that is evolving is more malleable. Perhaps, by creating a story of how the needs of funders and institutions are better served by focussing on a different form of signalling it will be possible to shift.

Of course this does happen in nature as well. When a sub-population develops a different form of display and co-selection kicks off then populations diverge, sometimes to occupy different niches, sometimes to compete, and ultimately displace the original population. It’s one way that new species form.

 

 

Posted on

Freedoms and responsibilities: Goffman, Hunt, Bohannan and Stapel

There has been much talk about both “academic freedom” as well as the responsibilities of scholars over the past few weeks. Both of these are troublesome concepts, not least because one person’s “freedom” is another’s irresponsible conduct. But particularly in the context of “academic freedom” the question of freedom to do or say what, and what responsibilities come with that is complex. And of course the freedom to speak is not the right to an expectation to be taken seriously. Any such right or authority is also tied to certain, usually unspecified responsibilities.

The question of academic freedom has been most visibly raised in the context of Tim Hunt’s reported comments at a Korean conference. As it happens I have my own story involving Hunt and misintepretation, once which might provide a useful way in to the issue.

At the closing panel of the Berlin11 meeting I spoke in a discussion panel about the progress towards open access and the future of scholarly communications. The meeting, in Berlin, was held in an old building that had been wonderfully refurbished as a conference space. In my remarks I drew an analogy with the building, the idea of taking the best of the past and repurposing it to support the future, noting that the building literally showed the scars of history, in this case the damage inflicted by allied bombing in World War II.

It was later related to me that Hunt had said that this was a very eloquent defence of journals like Nature and Science. Of course anyone who knows me will know that was absolutely not my intended meaning. What I meant, and what I said, were not congruent with what was heard. My intent was to provoke thought on what was worth keeping, not to defend the status quo. But who is responsible for the misunderstanding? What is my responsibility for greater clarity?

It may seem like a trivial misunderstanding, but it could have not been. We were in Berlin. The building might have a very dark history, certainly it is a near statistical certainty that some members of the audience had lost family members to allied bombing. My comments could have been misintepreted as saying that the building was more important than their relative’s suffering. That issue did not occur to me at the time, and looking back today I am ashamed by that. It may not have changed what I said, but it would certainly have changed the way I said it. As a sufficient authority to be asked to offer my views in the final session of an important meeting I had a responsibility to take care that my comments were authoritative but also that they were responsible.

Nobel Laureates travel a lot. They are in demand as speakers because they have authority. They have authority obviously in their area of research but also as senior members of the research community they bring a perspective as leaders who have been involved in the governance and strategic development of the research community. When that authority is assumed without sufficient care, or in areas where the person in question is not well informed, the result tends to rebound badly – Jim Watson’s comments on race, Pauling’s on vitamin C come to mind.

To those who are given great authority, whether in the form of Nobel prizes or large twitter followings, is also given great responsibility. Sometimes discharged well, sometimes not. Academic authority and academic freedom are not easy bedfellows. The right to speak one’s mind is freedom of speech. The ability to deploy one’s authority is not a right. Authority is the ability to be listened to not the ability to speak freely. And that ability comes with responsibility. Academic freedom is not the right to speak one’s mind. It is rather the responsibility to speak on issues, with the authority that arises from scholarly rigour. It is the tradition that employment should not be at risk when a scholar speaks in their area of expertise.

The most damning indictment therefore of the cries of “Academic Freedom” in the defense of Hunt is that his comments were bad science. They were spectacularly uninformed by the large quantity of literature that shows virtually the opposite of what he said (see Curt Rice’s blog for an up to date summary). Further the defence that “it was just a joke” can only be made by failing to engage with the literature that shows that not only do jokes surface real bias and real issues, but that asking a disdvantaged group to accept something as a joke normalises that disadvantage. Hilda Bastian has covered this in her excellent post.

The question of responsibility has also been raised in the furore of John Bohannan’s recent exposes, first on fly by night scholarly publishers seeking to fleece researchers, and more recently on the reporting, review and publicity around poorly run “medical” studies. In both cases questions are raised of methodology. In the Open Access sting many commentators, myself included, excoriated Bohannan for not running a proper control, in essence not running a proper scientific study. In the more recent chocolate study issues of ethical oversight and risk to participants were raised. If Bohannan was a scientist, speaking with the authority of a scholar then this would be a reasonable criticism. His own claim of the title “gonzo scientist” raises some interesting questions in this regard but fundamentally he is a journalist and writer, governed by different rules of authority and responsibility.

In the case of the OA sting those questions of authority were muddied by the publication of the piece in Science. Online the distinction between this journalistic piece and a research article is not immediately clear. To be fair, in the piece itself John does make the point that conclusions on the prevalence of poor peer review practices in subscription vs open access journals cannot be drawn from this work. Indeed his aim is a different kind of “proof”, in this case an existence proof of the problem – there are “journals” that do little to no peer review, and many of them are open access.

The problems I have with the piece, and they are many, arguably conflate my expectations of a piece of scholarly research and the responsibilities of a scholar – the need to tell us something new or useful – with the very different aims of a journalist, to expose an issue to a wider audience. Indeed the very establishment power structures moving into place to defend Hunt are the ones that I, arguably hypocritically, deployed to combat Bohannan. “The problem has been known for some time” “Quiet work was being done on it” “Leave our community to get on and sort out our problems”. But did we need an outsider to make it public enough and urgent enough to drive real action? Did Bohannan have responsibilities to the Open Access community to tell us more about the problem, to do a proper study, or as a journalist was his responsiblity to publicly expose the issue?

Alice Goffman is another researcher facing a different set of tensions over responsibility, freedom and authority. Her book On the Run gives a challenging account of inner city life amongst deprived black american youth. Published in 2014 it can be seen as a warning of the subsequent events in Ferguson and Baltimore and other places.

Goffman is an enthographer and her book started life as a scholarly monograph, but one that has gone on to have success as a mainstream non-fiction book. Ethnography involves working closely with, often living with research subjects, and the protection of the privacy of subjects is held as a very high principle. As described in this Slate article (which is my main source) this generally means obscuring locations, names, even the chronology of events to create a narrative which surfaces a deeper underlying truth about what is going on. Goffman took this responsibility particularly seriously given she observed events that could land people in jail, going so far as to destroy her notebooks so as to protect her research subjects.

But as this uncomfortable narrative became more public and transformed into a mainstream non-fiction book the responsibilities of the author (no longer a scholar?) seemed to change. General non-fiction is supposed to be “true” and Goffman’s rearrangement of facts, people and timelines breaks this expectation. What is interesting is that was in turn is used to raise charges of scholarly misconduct. The responsibility of the author to the reader is in direct conflict with the responsibility of the scholar to their subjects, yet the critic chooses to attack the scholarship. Indeed, given that the criticism and claims of misconduct are based on a forensic analysis of the text in some sense Goffman is under attack because she didn’t do a good enough job of hiding the process of discharging her scholarly responsibilities, leaving inconsistencies in the timelines and events.

Which responsibility trumps which? What does “integrity” mean in this context, or rather disparate and competing contexts, and how does a public scholar working on important and challenging problems navigate those competing issues? Where is Goffman’s academic freedom and where do her academic responsibilities lie? In restricting her space for communication to the academic world? In speaking (her) truth to power? Or is that space left for those licensed to speak through mainstream books? Is it left for Bohannan because only the outsider can make that transition?

The question of research integrity in Goffman’s case is challenging. Her destruction of notebooks certainly disturbs me as someone concerned primarily with the integrity of the research record. But I can respect the logic and to the extent that it is seen as reasonable within her disciplinary context accept that as appropriate scholarly practice.

The question of fraud in natural and social science research may seem much clearer. Diederik Stapel (I could have easily chosen Jan Hendrik Schön or many others) simply made up datasets. Here it seems there are clear lines of responsibility. The scholar is expected to add to the record, not muddy it. As we move towards digital records and data sharing these expectations are rising. Reproducible research is a target that seems plausible at least in some disciplines, although ironically we are perhaps merely returning to the level of record keeping recommended by Robert Boyle in 1660.

Does academic freedom mean the right to publish results based on made up data? Of course not. The scholar has a responsibility to report accurately when speaking in a scholarly context. It is not a crime to make up data, even in a research context. Ideas might be expressed though imagined or constructed datasets, they may even be an integral part of the research process as test sets, statistical tools or training sets. It is a “crime” to misrepresent or mis-use them. Even carelessness is treated a significant misdemeanour, leading as it does to retraction and consequent embarassment. Where does “carelesness” of the type that leads to retraction become “foolishness” that only requires mild rebuke?

But the idea of a “complete record” and “reproducibility” is a slippery one. In Goffman’s case reproducibility is impossible even in principle. Ethnographers would I imagine regard it as deeply problematic. The responsibility here is not even to report true facts, but the deeper truth – as the scholar sees it – that underlies the events they observe. Stapel may well have thought he was also telling “a truth”, just one for which the data wasn’t quite clean enough. A serious issue behind Bohannan’s chocolate expose is that p-value hacking, searching a weak dataset for “a truth” to tell, is endemic in many disciplines and that peer review as currently constructed is impotent in tackling it. Peer review assumes that authors have taken on board the responsibility to tell the truth (something Bohannan explicitly didn’t do for instance in the correspondence he had with PLOS One staff in the technical checks done before formal peer review).

Many of the technical discussions of reproducibility and data sharing founder on issues of reproducible for who? At what level? In what way? Bohannan shared his data, but you could not now reproduce his “experiment” precisely. His actions make that impossible. Goffman’s data does not exist but events in Ferguson, Baltimore and elsewhere arguably confirm her claims and narrative. Does Amgen’s failure to reproduce the vast majority of findings published on cancer biology in “top journals” mean we have a crisis?

Perhaps better to ask, what is the responsibility of authors publishing in cancer biology to their readers. To tell the truth as they see it? Obviously. To use all the tools at our disposal to prevent us fooling ourselves, to prevent us seeing what we want to see? Certainly. To provide the data? Increasingly so. To ensure the materials (cell lines, antibodies, reagents) are available to those who do want to do direct replication? Oh, that might be too much to expect. Today at least, but tomorrow? This is a discussion about responsibilies. Not technical details. Responsibilities to who, and for what, and how does that vary across disciplines. Perhaps focussing on “reproducibility” is the wrong approach.

As freedom of speech is merely right to a voice, not to a listener, academic freedom has its limits. The boundaries between scholarly speech, within a scholarly community, and wider public speech is blurring, as Goffman and Hunt have found, and as Bohannan has shown us. Context matters, whether the context of my previous writing on the merits and de-merits of Nature, the history of a building, or in the choice to make a joke of the wrong type in the wrong place. And the authority that comes from experience and responsibility in one space does not always travel well into a different context.

Does this mix of contexts and expectations mean we should simply give up? Just be quiet and retreat? That would be the easy answer. But the wrong one. Academic Freedom, or Academic Responsibility comes with the responsibility to speak. But it is a responsibility to be exercised with care. And with empathy for the different contexts that different audiences may find themselves in. Showing our working and showing our thinking. Showing the disciplinary traditions and expectations, the responsibilities that we have assumed, explicitly will help.

Above all, speaking from a position of authority (and I have chosen to use the word authority, rather than power deliberately) means assuming a higher level of reponsibility. This is perhaps best summed up in the direct advice “never punch down”. When speaking from a position of scholarly authority the limits of that authority, the limits of the experience, and the care expected in having mastery of the evidence are higher. And this is reasonable. And more and more important if scholarship is to be part of the wider world and not something that is done to it. If, after all, scholarship is about informed criticism and discussion, we all have a responsibility not just to speak, with care, but also to listen.

This piece has been very strongly shaped by a range of recent discussions, most strongly with Michael Nielsen (on John Bohannan’s work) and Michelle Brook (on diversity, power relations, integrity and the tensions between them), but also the ongoing discussion on twitter and more generally about Tim Hunt’s comments and Bohannan’s recent “sting”.

Posted on

Community Support for ORCID – Who’s next to the plate?

Geoff Bilder, Jennifer Lin, Cameron Neylon

The announcement of a $3M grant from the Helmsley Trust to ORCID is a cause for celebration. For many of us who have been involved with ORCID, whether at the centre or the edges, the road to sustainability has been a long one, but with this grant (alongside some other recent successes) the funding is in place to take the organization to where it needs to be as a viable membership organization providing critical community services.

When we wrote the Infrastructure Principles we published some weeks back, ORCID was at the centre of our thinking, both as one of the best examples of good governance practice and as an infrastructure that needs sustaining. To be frank it has been disappointing, if perhaps not surprising how long it has taken for key stakeholders to step up to the plate to support its development. Publishers get a lot of stick when it comes to demanding money, but when it comes to community initiatives it is generally publisher that put up the initial funding. This has definitely been the case with ORCID, with funders and institutions falling visibly behind, apparently assuming others will get things moving.

This is a common pattern, and not restricted to scholarly communications. Collective Action Problems are hard to solve, particularly when communities are diverse and have interests that are not entirely aligned. Core to solving collective action problems is creating trust. Our aim with the infrastructure principles was very much to raise the issue of trust and what makes a trustworthy organization to a greater prominence.

Developing trust amongst our diverse communities requires that we create trustworthy institutions. We have a choice. We can create those institutions in a way that embodies our values, the values that we attempted to articulate, or we can leave it to others. Those others will have other values, and other motives, and cannot be relied upon to align with our communities’ interests.

Google Scholar is a classic example. Everybody loves Google Scholar, and it’s a great set of tools. But it has not acted in a way that serve the communities’ broader needs. It does not have an API to allow the data to be re-used elsewhere. We cannot rely on it to continue in its current form.

Google Scholar exists fundamentally so that researchers will help Google organize the world’s research information for Google’s benefit. ORCID exists so that the research community can organize the world’s research information for our community’s benefit. One answers to its shareholders, the other to the community. And as a result needs the support of our community for its sustainability. As the saying goes, when you don’t pay for the product, you are the product.

ORCID is a pivotal case. Will our communities choose to work together to build sustainable infrastructures that serve our needs and answer to us? Or will we repeat the mistakes of the past and leave that to other players whose interests do not align with our own. If we can’t collectively bring ORCID to a place where it is sustainable, supported by the whole community then what hope is there for more specialist, but no less necessary, infrastructures?

The Helmsley Trust deserves enormous credit for stepping up to the plate with this grant funding, as do the funders (including publishers) and early joining members who have gone before. But as a community we need to do more than provide time limited grants. We need to take the collective responsibility to fund key infrastructure on an ongoing basis. And that means that other funders, institutions, alongside publishers need to play their part.

Posted on

.everyone or .science? Or both? Reflections on Martha Lane Fox’s Dimbleby Lecture

English: Martha Lane Fox
Martha Lane Fox (Photo: The Cabinet Office License: OGL v1.0)

On March 30 the BBC broadcast a 40 minute talk from Martha Lane Fox. The Richard Dimbleby Lecture is an odd beast, a peculiarly British, indeed a peculiarly BBC-ish institution. It is very much an establishment platform, celebrating a legendary broadcaster and ring marshaled by his sons, a family that as our speaker dryly noted are “an entrenched monopoly” in British broadcasting.

 

Indeed one might argue Baroness Lane Fox, adviser to two prime ministers, member of the House of Lords, is a part of that establishment. At the same time the lecture is a platform for provocation, for demanding thinking. And that platform was used very effectively to deliver a brilliant example of another very British thing, the politely impassioned call for radical (yet moderate) action.

The speech calls for the creation of a new public institution. Dubbed “Dot Everyone” such an institution would educate, engage and inform all citizens on the internet. It would act as a resource, it would show what might be possible, it would enhance diversity and it would explore and implement a more values based approach to how we operate on the web. I have quibbles, things that got skipped over or might merit more examination, but really these are more the product of the space available than the vision itself.

At the centre of that vision is a call for a new civics supported by new institutions. This chimes with me as it addresses many of the same issues that have motivated my recent thinking in the research space. The Principles for Open Infrastructures I wrote with Geoff Bilder and Jennifer Lin, could as easily have been called Principles for Institutions – we were motivated to work on them because we believe in a need for new institutions. For many years I have started talks on research assessment by posing the question “what are your values” – a question implicit in the speech as it probes the ethics of how the internet is built in practice.

I was excited by this speech. And inspired.

And yet.

One element did not sit easily with me. I emphasized the British dimension at the top of this piece. Martha Lane Fox’s pitch was to “make Britain brilliant at the internet” and was focused on the advantages for this country. By contrast the first of the Principles for Open Infrastructures is that these new institutions must transcend geography and have international reach. Is this a contradiction? Are we pushing in different directions? More particularly is there a tension between an institution “for everyone” and one having a national focus?

The speech answers this in part and I think the section is worth quoting in full:

We should be ambitious about this. We could be world leading in our thinking.

In this 800th year anniversary of Magna Carta, the document widely upheld as one of the first examples of the rule of law, why don’t we establish frameworks to help navigate the online world?

Frameworks that would become as respected and global as that rule of law, as widely adopted as the Westminster model of parliamentary democracy.

Clearly this new institution, “our new institution” as it is referred to throughout, has international ambitions. But I don’t imagine I am the only person to find something almost neo-colonial in these words. Britain has sought to export its values to the world many times, and been remarkably successful. But in the past this has also been paternalistic. The very best possible assessment of what was in many cases well intentioned imposition of British values is equivocal. Lane Fox sets up the “good” values of Britain against the lack of values that inhere in the big commercial players building the web. What is it today that make “our” values those that should inspire or lead any more than the, now questionable, values of the past?

To be clear I am absolutely not suggesting that these are issues that have escaped the speaker’s notice. Martha Lane Fox is an outstanding and effective campaigner for diversity and inclusion and the section of her talk that I have taken out of context above comes after a substantial section on the value of inclusion, focused largely on gender but with a recognition that the same issues limit the inclusion and contribution of many people on the basis of many types of difference. In truth, her views on the how and the why of what we need to change, on what those values are, are highly aligned with mine.

But that’s kind of the point.

If we are to have a new civics, enabled by the communications infrastructure that the web provides, then diversity will lie at the heart of this. Whether you take the utilitarian (not to say neo-liberal) view that inclusion and diversity drives the creation of greater value, or see it as simply a matter of justice, diversity and inclusion and acceptance of difference are central.

But at the same time the agile and flat governance models that Lane Fox advocates, to be fair in passing, for our new institution arise out the concept that “rough consensus and running code” are the way to get things done. But whose consensus matters? And how does the structural imbalance of the digital divide affect whose code gets to run first? This seems to me the central question to be resolved by this new civics. How do we use the power of web to connect communities of interest, and to provide infrastructures that allow them to act, to have agency, while at the same time ensuring inclusion.

At its best the web is an infrastructure for communities, a platform that allows people to come together. Yet communities define themselves by what they have in common, and by definition exclude those who do not share those characteristics. My implicit claim above that our institutional principles are somehow more inclusive or more general than Lane Fox’s is obviously bogus. Our focus is on the research community, and therefore just as exclusive as a focus on a single nation. There are no easy answers here.

The best answer I can give is that we need multiple competing centres. “Dot Everyone” is a call for a national institution, a national resurgence even. Alone it might be successful, but even better is for it to have competition. Martha Lane Fox’s call is ambitious, but I think it’s not enough. We need many of these institutions, all expressing their values, seeking common ground to build a conversation between communities, domains, geographies and nations.

The tension between facilitating community and diversity can be a productive one if two conditions are satisfied. First that all can find communities where they belong, and secondly that the conversation between communities is just and fair. This is a huge challenge, it will require nothing less than a new global infrastructure for an inclusive politics.

It is also probably a pipe dream, another well meaning but ultimately incomplete effort to improve the world. But if the lesson we learn from colonialism is that we should never try, then we should give up now. Better is to do our best, while constantly questioning our assumptions and testing them against other’s perspectives.

As it happens, we have some new systems that are pretty good for doing that. We just need to figure out how best to use them. And that, at core, was Martha Lane Fox’s point.

 

 

Posted on

End of Feed

This is icon for social networking website. Th...
Friendfeed (Photo credit: Wikipedia) Public Domain

Following on from (but unrelated to) my post last week about feed tools we have two posts, one from Deepak Singh, and one from Neil Saunders, both talking about ‘friend feeds’ or ‘lifestreams’. The idea here is of aggregating all the content you are generating (or is being generated about you?) into one place. There are a couple of these about but the main ones seem to be Friendfeed and Profiliac. See Deepaks’s post (or indeed his Friendfeed) for details of the conversations that can come out of these type of things.

A (small) feeding frenzy – Cameron Neylon, Science in the Open – 10 March 2008

Half the links in that quote are dead. I wrote the post above seven years ago today, and it very much marked a beginning. Friendfeed went on to become the coffee house for a broad community of people interested in Open Science and became the place where, for me at least, many of the key discussions took place. Friendfeed was one of a number of examples of “life feed” services. The original intent was as an aggregation point for your online activity but the feed itself rapidly became the focus. Facebook in particular owes a debt to the user experience of Friendfeed. Facebook bought Friendfeed for the team in 2009 and rapidly started incorporating its ideas.

Yesterday Facebook announced they were going to shutter the service that they have to be fair kept going for many years now with no revenue source and no doubt declining user numbers. Of course those communities that remained are precisely the ones that most loved what the service offered. The truly shocking thing is that although nothing has been done to the interface or services that Friendfeed offers for five years it still remains a best in class experience. Louis Gray had some thoughts on what was different about Friendfeed. It remains, in my view, the best technical solution and user experience for enabling the kind of sharing that researchers actually want to do.  I remember reading about Robert Scoble disliked the way that Friendfeed worked, and thinking “all those things are a plus for researchers…”. Twitter is ok, Facebook really not up to the job, Figshare doesn’t have the social features and all the other “facebooks for science” simply don’t have critical mass. Of course, neither did Friendfeed once everyone left either…but while there was a big community there we had a glimpse of what might be possible.

It’s also a reminder, as discussed in the Principles for Scholarly Infrastructures that Geoff Bilder, Jennifer Lin and myself released a week or so back, that relying on the largesse of third parties is not a reliable foundation to build on. If we want to take care of our assets as a community, we need to take responsibility for them as well. In my view there is some important history buried in the records of Friendfeed and I’m going to make some effort to build an archive. This script appears to do a good job of grabbing public feeds. It doesn’t pull discussions (ie the comments on other people’s posts) unless you have the “remote key” for that account. If anyone wants to send me their remote key (log in to friendfeed and navigate to http://friendfeed.com/remotekey) I’ll take a shot at grabbing their discussions as well. Otherwise I’ll just try and prioritize the most important accounts from my perspective to archive.

Is it recent history or is it ancient? We lost Jean-Claude Bradley last year, one of the original thinkers, and perhaps more importantly do-ers, of many strands in Open Research. Much of his thinking from 2008-2011 was on Friendfeed. For me, it was the space in which the foundations for a lot of my current thinking was laid. And where I met many of the people who helped me lay those foundations. And a lot of my insights into how technology does and does not help communities were formed by watching how much better Friendfeed was than many other services. Frankly a lot of the half-baked crap out there today could learn a lot by looking at how this nearly decade-old website works. And still works for those communities that have stayed in strength.

But that is the second lesson. It is the combination of functionality and the community that makes the experience so rich. My community, the Open Science group, left en masse after Facebook acquired Friendfeed. That community no longer trusted that the service would stay around (c.f. again those principles on trust). The librarian community stayed and had an additional five years of rich interactions. It’s hardly new to say that you need both community and technology working together to build a successful social media experience. But it still makes me sad to see it play out like this. And sad that the technology that demonstrably had the best user experience for research and scholarship in small(ish) communities never achieved the critical mass that it needed to succeed.