There has been much talk about both “academic freedom” as well as the responsibilities of scholars over the past few weeks. Both of these are troublesome concepts, not least because one person’s “freedom” is another’s irresponsible conduct. But particularly in the context of “academic freedom” the question of freedom to do or say what, and what responsibilities come with that is complex. And of course the freedom to speak is not the right to an expectation to be taken seriously. Any such right or authority is also tied to certain, usually unspecified responsibilities.
The question of academic freedom has been most visibly raised in the context of Tim Hunt’s reported comments at a Korean conference. As it happens I have my own story involving Hunt and misintepretation, once which might provide a useful way in to the issue.
At the closing panel of the Berlin11 meeting I spoke in a discussion panel about the progress towards open access and the future of scholarly communications. The meeting, in Berlin, was held in an old building that had been wonderfully refurbished as a conference space. In my remarks I drew an analogy with the building, the idea of taking the best of the past and repurposing it to support the future, noting that the building literally showed the scars of history, in this case the damage inflicted by allied bombing in World War II.
It was later related to me that Hunt had said that this was a very eloquent defence of journals like Nature and Science. Of course anyone who knows me will know that was absolutely not my intended meaning. What I meant, and what I said, were not congruent with what was heard. My intent was to provoke thought on what was worth keeping, not to defend the status quo. But who is responsible for the misunderstanding? What is my responsibility for greater clarity?
It may seem like a trivial misunderstanding, but it could have not been. We were in Berlin. The building might have a very dark history, certainly it is a near statistical certainty that some members of the audience had lost family members to allied bombing. My comments could have been misintepreted as saying that the building was more important than their relative’s suffering. That issue did not occur to me at the time, and looking back today I am ashamed by that. It may not have changed what I said, but it would certainly have changed the way I said it. As a sufficient authority to be asked to offer my views in the final session of an important meeting I had a responsibility to take care that my comments were authoritative but also that they were responsible.
Nobel Laureates travel a lot. They are in demand as speakers because they have authority. They have authority obviously in their area of research but also as senior members of the research community they bring a perspective as leaders who have been involved in the governance and strategic development of the research community. When that authority is assumed without sufficient care, or in areas where the person in question is not well informed, the result tends to rebound badly – Jim Watson’s comments on race, Pauling’s on vitamin C come to mind.
To those who are given great authority, whether in the form of Nobel prizes or large twitter followings, is also given great responsibility. Sometimes discharged well, sometimes not. Academic authority and academic freedom are not easy bedfellows. The right to speak one’s mind is freedom of speech. The ability to deploy one’s authority is not a right. Authority is the ability to be listened to not the ability to speak freely. And that ability comes with responsibility. Academic freedom is not the right to speak one’s mind. It is rather the responsibility to speak on issues, with the authority that arises from scholarly rigour. It is the tradition that employment should not be at risk when a scholar speaks in their area of expertise.
The most damning indictment therefore of the cries of “Academic Freedom” in the defense of Hunt is that his comments were bad science. They were spectacularly uninformed by the large quantity of literature that shows virtually the opposite of what he said (see Curt Rice’s blog for an up to date summary). Further the defence that “it was just a joke” can only be made by failing to engage with the literature that shows that not only do jokes surface real bias and real issues, but that asking a disdvantaged group to accept something as a joke normalises that disadvantage. Hilda Bastian has covered this in her excellent post.
The question of responsibility has also been raised in the furore of John Bohannan’s recent exposes, first on fly by night scholarly publishers seeking to fleece researchers, and more recently on the reporting, review and publicity around poorly run “medical” studies. In both cases questions are raised of methodology. In the Open Access sting many commentators, myself included, excoriated Bohannan for not running a proper control, in essence not running a proper scientific study. In the more recent chocolate study issues of ethical oversight and risk to participants were raised. If Bohannan was a scientist, speaking with the authority of a scholar then this would be a reasonable criticism. His own claim of the title “gonzo scientist” raises some interesting questions in this regard but fundamentally he is a journalist and writer, governed by different rules of authority and responsibility.
In the case of the OA sting those questions of authority were muddied by the publication of the piece in Science. Online the distinction between this journalistic piece and a research article is not immediately clear. To be fair, in the piece itself John does make the point that conclusions on the prevalence of poor peer review practices in subscription vs open access journals cannot be drawn from this work. Indeed his aim is a different kind of “proof”, in this case an existence proof of the problem – there are “journals” that do little to no peer review, and many of them are open access.
The problems I have with the piece, and they are many, arguably conflate my expectations of a piece of scholarly research and the responsibilities of a scholar – the need to tell us something new or useful – with the very different aims of a journalist, to expose an issue to a wider audience. Indeed the very establishment power structures moving into place to defend Hunt are the ones that I, arguably hypocritically, deployed to combat Bohannan. “The problem has been known for some time” “Quiet work was being done on it” “Leave our community to get on and sort out our problems”. But did we need an outsider to make it public enough and urgent enough to drive real action? Did Bohannan have responsibilities to the Open Access community to tell us more about the problem, to do a proper study, or as a journalist was his responsiblity to publicly expose the issue?
Alice Goffman is another researcher facing a different set of tensions over responsibility, freedom and authority. Her book On the Run gives a challenging account of inner city life amongst deprived black american youth. Published in 2014 it can be seen as a warning of the subsequent events in Ferguson and Baltimore and other places.
Goffman is an enthographer and her book started life as a scholarly monograph, but one that has gone on to have success as a mainstream non-fiction book. Ethnography involves working closely with, often living with research subjects, and the protection of the privacy of subjects is held as a very high principle. As described in this Slate article (which is my main source) this generally means obscuring locations, names, even the chronology of events to create a narrative which surfaces a deeper underlying truth about what is going on. Goffman took this responsibility particularly seriously given she observed events that could land people in jail, going so far as to destroy her notebooks so as to protect her research subjects.
But as this uncomfortable narrative became more public and transformed into a mainstream non-fiction book the responsibilities of the author (no longer a scholar?) seemed to change. General non-fiction is supposed to be “true” and Goffman’s rearrangement of facts, people and timelines breaks this expectation. What is interesting is that was in turn is used to raise charges of scholarly misconduct. The responsibility of the author to the reader is in direct conflict with the responsibility of the scholar to their subjects, yet the critic chooses to attack the scholarship. Indeed, given that the criticism and claims of misconduct are based on a forensic analysis of the text in some sense Goffman is under attack because she didn’t do a good enough job of hiding the process of discharging her scholarly responsibilities, leaving inconsistencies in the timelines and events.
Which responsibility trumps which? What does “integrity” mean in this context, or rather disparate and competing contexts, and how does a public scholar working on important and challenging problems navigate those competing issues? Where is Goffman’s academic freedom and where do her academic responsibilities lie? In restricting her space for communication to the academic world? In speaking (her) truth to power? Or is that space left for those licensed to speak through mainstream books? Is it left for Bohannan because only the outsider can make that transition?
The question of research integrity in Goffman’s case is challenging. Her destruction of notebooks certainly disturbs me as someone concerned primarily with the integrity of the research record. But I can respect the logic and to the extent that it is seen as reasonable within her disciplinary context accept that as appropriate scholarly practice.
The question of fraud in natural and social science research may seem much clearer. Diederik Stapel (I could have easily chosen Jan Hendrik Schön or many others) simply made up datasets. Here it seems there are clear lines of responsibility. The scholar is expected to add to the record, not muddy it. As we move towards digital records and data sharing these expectations are rising. Reproducible research is a target that seems plausible at least in some disciplines, although ironically we are perhaps merely returning to the level of record keeping recommended by Robert Boyle in 1660.
Does academic freedom mean the right to publish results based on made up data? Of course not. The scholar has a responsibility to report accurately when speaking in a scholarly context. It is not a crime to make up data, even in a research context. Ideas might be expressed though imagined or constructed datasets, they may even be an integral part of the research process as test sets, statistical tools or training sets. It is a “crime” to misrepresent or mis-use them. Even carelessness is treated a significant misdemeanour, leading as it does to retraction and consequent embarassment. Where does “carelesness” of the type that leads to retraction become “foolishness” that only requires mild rebuke?
But the idea of a “complete record” and “reproducibility” is a slippery one. In Goffman’s case reproducibility is impossible even in principle. Ethnographers would I imagine regard it as deeply problematic. The responsibility here is not even to report true facts, but the deeper truth – as the scholar sees it – that underlies the events they observe. Stapel may well have thought he was also telling “a truth”, just one for which the data wasn’t quite clean enough. A serious issue behind Bohannan’s chocolate expose is that p-value hacking, searching a weak dataset for “a truth” to tell, is endemic in many disciplines and that peer review as currently constructed is impotent in tackling it. Peer review assumes that authors have taken on board the responsibility to tell the truth (something Bohannan explicitly didn’t do for instance in the correspondence he had with PLOS One staff in the technical checks done before formal peer review).
Many of the technical discussions of reproducibility and data sharing founder on issues of reproducible for who? At what level? In what way? Bohannan shared his data, but you could not now reproduce his “experiment” precisely. His actions make that impossible. Goffman’s data does not exist but events in Ferguson, Baltimore and elsewhere arguably confirm her claims and narrative. Does Amgen’s failure to reproduce the vast majority of findings published on cancer biology in “top journals” mean we have a crisis?
Perhaps better to ask, what is the responsibility of authors publishing in cancer biology to their readers. To tell the truth as they see it? Obviously. To use all the tools at our disposal to prevent us fooling ourselves, to prevent us seeing what we want to see? Certainly. To provide the data? Increasingly so. To ensure the materials (cell lines, antibodies, reagents) are available to those who do want to do direct replication? Oh, that might be too much to expect. Today at least, but tomorrow? This is a discussion about responsibilies. Not technical details. Responsibilities to who, and for what, and how does that vary across disciplines. Perhaps focussing on “reproducibility” is the wrong approach.
As freedom of speech is merely right to a voice, not to a listener, academic freedom has its limits. The boundaries between scholarly speech, within a scholarly community, and wider public speech is blurring, as Goffman and Hunt have found, and as Bohannan has shown us. Context matters, whether the context of my previous writing on the merits and de-merits of Nature, the history of a building, or in the choice to make a joke of the wrong type in the wrong place. And the authority that comes from experience and responsibility in one space does not always travel well into a different context.
Does this mix of contexts and expectations mean we should simply give up? Just be quiet and retreat? That would be the easy answer. But the wrong one. Academic Freedom, or Academic Responsibility comes with the responsibility to speak. But it is a responsibility to be exercised with care. And with empathy for the different contexts that different audiences may find themselves in. Showing our working and showing our thinking. Showing the disciplinary traditions and expectations, the responsibilities that we have assumed, explicitly will help.
Above all, speaking from a position of authority (and I have chosen to use the word authority, rather than power deliberately) means assuming a higher level of reponsibility. This is perhaps best summed up in the direct advice “never punch down”. When speaking from a position of scholarly authority the limits of that authority, the limits of the experience, and the care expected in having mastery of the evidence are higher. And this is reasonable. And more and more important if scholarship is to be part of the wider world and not something that is done to it. If, after all, scholarship is about informed criticism and discussion, we all have a responsibility not just to speak, with care, but also to listen.
This piece has been very strongly shaped by a range of recent discussions, most strongly with Michael Nielsen (on John Bohannan’s work) and Michelle Brook (on diversity, power relations, integrity and the tensions between them), but also the ongoing discussion on twitter and more generally about Tim Hunt’s comments and Bohannan’s recent “sting”.
It would be a “crisis” if data destruction were to become acceptable.
Availability for whom? For anyone who wants to examine the original data for purposes of exploring a different interpretation of the work done, for affirming or dis-confirming an aspect of the account and so on.
Science is a communal activity.
More, IRBs (as far as I know)– as well as science journals — require data be available for some period (often a minimum of 4-5 years).
If there are problems with personal identity and adverse ramifications of identification via data content, redaction (extensive if need be) is the way to go (I do it often in one aspect of my work). There is, to my knowledge, no mandate that the original data (transcripts and the hard drive — now that is thorough) be totally obliterated.
I am sure Goffman can craft a rejoinder in her defense. But the bottom line = it smells of track-covering (since this is a book, not a scientific paper — perhaps data preservation is not relevant). However, if the latter is indeed the case, then the book has to be taken be taken as fiction and not science. That is too bad, but data destruction renders it necessarily so. Any debate about this is derailed in consequence of the author’s choices.
So I agree with the sentiment (from my laboratory research background) at some level. However such an absolute position is untenable.
First obvious point is that much (possibly the majority of data in sheer volume terms) that is collected is discarded (astronomy, LHC etc) before a human ever sees it. Equally the raw data from most DNA sequencing is discarded and only the derivative data is kept.
More generally the question is one of how “raw” do you keep. The actual voltage traces of instruments are almost never kept (record of charges on the CCD of a camera for instance?) so there is always a choice made about which derivates are kept. And that’s before you get to issues of confidentiality and ethics. Some medical/administrative records are destroyed as a matter of course for instance after a certain period.
The cut off is always arbitrary and in Goffman’s case you could argue that the original notebooks are confidential at a level where there is no safe means of making them available. Then the books is “the data” in an observational sense. It feels odd from a science perspective but I think that perspective would chime with many social scientists and humanists.
But more than that there is the question of trust. The issue raised by Goffman’s critics is not really whether she changed facts (that’s their “smoking gun”, but its not their real complaint) but whether she is “reliable”. In that sense the notebooks wouldn’t help. They could as easily be a partial record, or even an incorrect one. This is what I mean by responsibility, we expect certain conduct in certain circumstances and reporting honestly and completely is one of them. If you don’t believe Goffman, you wouldn’t believe her notebook either.
This is why I introduce Stapel as an example. All the data in the world won’t prevent actual fraud with intent.
You split hairs.
She destroyed ALL (not part; not via some discipline-sanctioned criterion for “which is and is not” useful data to keep). She obliterated all.
She did not simply “discard” (as you put it) following some “reasonable” period (as is required of scholarly research). She dumped it all and within a very brief period. Perhaps this kosher in ethnography — and perhaps even more so when the output is placed in a book rather than an academic outlet. But then fiction, of evidential (or lack thereof) necessity, becomes a live question.
As for the Stapel “argument” –of course anyone can fabricate. BUT, that is a separate issue and, at best, a “look over there” argument. The bottom line is that elimination of all data is (at least in scientific venues) is explicitly prohibited for at least several years. Maybe she could have fabricated ala Stapel, but that is a different concern and tangential to the question of or or justification for total removal (not just redaction) of all her “evidence”. Two “wrongs” typically do not join to make a “right”.
One can rationalize all s/he wants, but it cannot make one empirically demonstrable, unplesant fact go away — a fact that calls into question Goffman’s credibility.
I think you may have inadvertantly made my point. You say in the top paragraph “no discipline-sanctioned approach” and then in the second “this may be kosher in ethnography”. That’s my central point, that we need more clarity about the context that a person is working in and an ability to test and critique that.
I would point out that in most modern DNA sequencing, astronomy, particle physics all of the original data is thrown away. Indeed in almost any work that involves an instrument of any sort we throw away the original signal. Increasingly data on DNA sequences is not being kept in favour of keeping the samples.
Even further there are examples where data has been thrown away for privacy reasons in the sciences. Perhaps the most illustrative example is the fact that we no longer know who the samples for the human genome project were taken from. That connection was very carefully destroyed to protect privacy. That seems rather similar to the Goffman case in some ways.
But overall my point is that the question of trust is a bit of a red herring. The question is one of trustworthiness and that requires context and community views. As the context gets broken (monograph becomes a trade non-fiction book) the responsibility to (try to) carry that context becomes greater. And it is necessary to consider the risks differently as context changes.
So I think there probably was a responsibility in shifting from monograph to trade non-fiction to be clear about the nature of the narrative. I’m not comfortable with the destruction of the notebook but I can understand the argument. The question of “what is the data” is really for the community of enthnographers to decide tho, not a natural scientist who has only ever dealt with molecules.
Thanks Cameron, for a great post and Stan for some interesting comments to follow. I hope you’ll permit me an extended response.
As someone who works across the social sciences, arts, humanities and now collaborates with clinically oriented Psychologists this thread reflects some interesting differences in the ways in which knowledge is created, used, stored, disseminated and how it is valued and then, ultimately how data is archived, managed or indeed in some cases destroyed across disciplines. This, as Cameron points out, ultimately also relates to the ethical imperative that as researchers we share our research outcomes responsibly and consider the differences between academic freedom and freedom of speech.
Other threads woven into this discussion include disciplinary differences in ethical considerations and how knowledge is constructed. As your ‘disciplinary other’ I will try and unpick these a bit, or as we say in my discipline ‘problematize’.
Firstly, of course I should clarify that in the social sciences, arts and humanities books (ethnographies, monographs, academically researched biographies) are in fact academic outputs. This of course is not the case in many sciences.
There is also, of late, a new interdisciplinary cross- over between ethnography, creative life writing and investigative journalism which raises some interesting ethical and integrity issues: are researchers based at Universities legally and ethically subject to the same laws and rules as journalists? Journalists are often protected by certain press regulations whereas researchers must abide by ethical codes of conduct as set out by their institutions and national bodies such as, for example, the UK Research Integrity Office.
This does not sit comfortably where the topic of the ‘reporting’ is illicit activities and where the data collected, again, might cause much harm if shared freely and openly. At what point is the ‘good’ of sharing overshadowed ethically by the detriment and harm caused? These are the questions that are truly routed in ethical debate, and at times conflict with national legislation.
The above also of course impacts on our so prized academic freedom: are there times when researchers perhaps ought not to publish something and forgo the academic accolades in favor of preserving the safety and wellbeing of our research participants? A question that was regularly raised during my work with Indigenous Australians, where historically used by paternalistic governments to underpin oppressive legislation. Where does this leave academic freedom and how does this intersect with the uses to which our words and research outputs are put? A question with which government struggles daily and has encouraged researchers to engage with.
Secondly, Alice Goffman’s work is routed in sociological research. Data, in sociology, can consist of personal data, which as was pointed out earlier can do harm if it were made publicly available, to those who provided said data to the researcher. In terms of integrity therefore, at what point do we think the ‘do no harm’ clause trumps the ‘thou shalt keep and share thy data responsibly’ clause? This question regularly rears its complicated head at my research ethics and integrity committee.
You may then ask: Is it actually necessary though to destroy the data completely? Would not an embargo suffice? This is where time frames matter, as pointed out by Stan. When is ‘short’, ‘too short’? In the social sciences there can be (and here I must stress that ‘the social sciences’ are rather diverse and what holds true for one area of inquiry, may not do for another, as I am sure is the case in the sciences).
‘Data’ may include personal information about actual living people and as such is subject to the data protection act, which, as we know, will vary across countries. In the UK personal data can be kept up to 7 years, whereas other forms of research data can be kept for longer. Goffman is based in the USA. Not sure what the law requires there, but it might explain her ‘decision’ to destroy her data after what seems like a short period of time. In the USA they have notoriously strict independent review boards (IRBs) which may well have mandated this destruction. It would not be the first time that social scientists have been asked, against their better judgement, to destroy their data as a result of research integrity procedures inappropriately applied.
In Sociology and Anthropology this becomes a tricky issue because often your data is a mixture of personal information as well as material that can be anonymised, should anonymisation be appropriate. Those social scientists using quantitative methods such as surveying will usually take a lot of care to provide ways in which they can disaggregate the personal information from the anonymised data sets so that the anonymised results can be shared and archived for a longer period of time and the personal data destroyed after, in the UK, 7 years.
For those researchers undertaking ethnography, this issue is trickier. Anonymisation is hard to do, especially with the internet and in fact most good Universities will supply training in anonymisation techniques as part of their research methods and ethics training. This may well include data destruction. In other cases, particularly anthropology, anonymisation is truly inappropriate and data destruction advocated against. Particularly in anthropology there are many, many cases where anonymisation would be in direct opposition to what the research participants would want as it conflicts with their Indigenous rights to culture or copyright and IP legislation and so on.
Lastly, the issue of reproducibility. Always a fun subject because it highlights nicely the two camps in which often people find themselves: the quantitative researchers and the qualitative ones. Many qualitative researchers, including ethnographers, have dismissed the idea that objective research even exists in their field and freely acknowledge their outcomes are context-sensitive, not reproducible, but therefore no less valuable.
Much can be gleaned theoretically, methodologically and otherwise from context-specific case-studies. Equally, some philosophers and science and technology researchers such as Karl Popper and Latour argue that in many ways science is not as objective as it thinks it is. The construction of the scientific paradigm and facts is as much a product of cultural epistemological preferences, funding climates, resources, institutional/ national research cultures, human expertise, the state of technology etc. and personalities: in other words, context-sensitive.
I know that many scientist friends of mine happily concur that this is the case, but do not think of themselves as being less ethical or indeed less competent as researchers. What they tend to do however, if preface their work with the necessary caveats to ensure they are transparent about the limitations of their work. It is this transparency and to my mind, scientific rigor that lies at the heart of this, something pointed out by Cameron above as well.
Good ethnographers do the same: they do not pretend to be objective, or that their work can be reproduced and they let their audience know this. This, I argue though makes their work no less rigorous or less ethical and in some ways non-problematical (see here I must dissent from Cameron’s assumption that all ethnographers find lack of reproducibility a problem: many do not find this a challenging issue at all).
Obviously this reproducibility of crops up to a lesser extent with those social scientists working in quantitative areas (although there are still questionable practices in evidence which I shan’t address here) so again I should emphasize that as with science research, the ‘disciplinary other’ is rather diverse and in fact does not always unanimously agree on what counts as ‘appropriate ethical conduct/ sharing’ and how data and information should be responsibly shared in public contexts.
As researchers I think, it is important therefore that, no matter what discipline we work in we should use our words carefully; try and assess the difference been how they might be perceived by those unfamiliar with our areas of inquiry and how insiders would grasp them; to explain things clearly without misrepresenting the facts and to freely acknowledge any caveats there might be in relation to any results we produce and to share our data responsibly so as not to cause harm of any kind. This might mean that we cannot shout as loudly about our research successes as we’d like or might mean we have to destroy our research data earlier than might seem appropriate, but then research should be for the common good, not personal glory.
That said, these things are easier said (or written about) than done, and I consider them to be a daily challenge where the practicalities of being a researcher in today’s climate are concerned. Current situations in the academe mitigate against good behavior. However. it should not stop us from at least trying, me thinks.
I routinely do case studies. We are currently examining self-knowledge in the last American split brain patient capable of responding to our inquiries. So, I have no qualms with ethnography in principle nor do I feel otherwise with respect to your comment: “Much can be gleaned theoretically, methodologically and otherwise from context-specific case-studies.”
My point is extremely simple. Trappings and nuance can be discussed — they are all vital — but they are NOT germane to the focal issue (and thus, in the present case, tend to distract more than illuminate).
To wit: Total destruction of data (whether mandated for some odd reason by a particular IRB) renders the available “ethnography” indistinguishable from fiction.
It may not be. Interpretation of any text is inevitable as is observation of the behavior grounding the text.
BUT, absent ANY record that the interpretations were made, there is NO way to know the status of the book’s content beyond the author’s “trust me its real”.
Without ANY notes or ANY pieces thereof, there is NO way for anyone to (a) know the “adventure” took place, or (b) examine the “data” (even if subjectively “tainted” — as are all social science data; and, some would argue, more objective data of the “hard” sciences) and make their own interpretations based on the original transcripts (and not a book whose content, of contingent necessity, forever must be regarded as potentially fictional.