Registered Reports for Qualitative Research: A call for feedback from humanities and social science researchers

tl;dr – in 2017 we are expanding the Registered Reports publication model into qualitative research for the first time. Since we’ve never done this before, we’re very interested to hear from qualitative researchers about how we can make the format attractive, and what obstacles or drawbacks we might encounter with pre-registration of qualitative studies.

Registered Reports are developing some serious momentum. What began in 2013 as a radical (and somewhat controversial) idea to curb bias in published research is rapidly becoming a standard addition to the publishing landscape. Twenty-eight journals have now signed on to offer the format, with more in the pipeline, and across a wide range of social, life and physical sciences.

For the uninitiated, Registered Reports are form of empirical publication where the peer review happens, in part, before the results of studies are known (full details here). Proposals that are scientifically robust, and which receive positive peer reviews, are then provisionally accepted for publication. Once the research is complete, the authors resubmit their manuscript with the results and discussion; it is re-reviewed and ultimately published, provided the authors adhered to their protocol, interpreted the evidence reasonably, and met any pre-agreed standards for assuring data quality.

The great strength of Registered Reports is the way it tackles bias. By accepting studies in advance of results, it prevents publication bias – the tendency for journals to selectively publish results that are considered clear or more attractive. And by requiring authors to pre-specify their research hypotheses and analysis methods, it also greatly reduces the capacity for biased reporting by authors.

As the popularity of Registered Reports has increased, so has its scholarly reach: from the initial launch within psychology and neuroscience it has expanded into political science, biology, and even physical sciences of physics and chemistry. But within every area, so far, that Registered Reports have ventured, it has been under the banner of quantitative research.

To lay my cards on the table, I'm a firm believer in the value of qualitative research. I gained a deep respect for it after being involved in this study, where we used qualitative methods (specifically, grounded theory) to explore the way parliamentarians use science and evidence in decision-making. Qualitative research can generate remarkably rich information, and in some ways it is more transparent than quantitative research. For instance, the qualitative researcher explicitly acknowledges their own bias, and – in our work at least – we publicly archived all of our data to enable qualitative replication.

Publication bias also appears to be just as much a problem in qualitative research as in quantitative research. As the authors of this study conclude:

"This suggests a mechanism by which "qualitative publication bias" might work: qualitative studies that do not show clear, or striking, or easily described findings may simply disappear from view. One implication of this is that, as with quantitative research, systematic reviews of qualitative studies may be biased if they rely only on published papers."

If publication bias is as much a problem for qualitative research as quantitative research, then there is clearly room for Registered Reports to become a popular option among qualitative researchers. This opens the door for Registered Reports to be taken up by fields such as anthropology, cultural studies, criminology, sociology, and beyond. In 2017 we will be launching Registered Reports in at least one journal that publishes papers within these fields, among other social sciences and humanities.

All that said, I am far from being an expert on qualitative methods, so if you are a qualitative researcher I would greatly appreciate your input into the design of a qualitative Registered Reports model. We need your help to get this right and to make it useful.

I am interested in any critical feedback, however positive or negative, and in a completely open-ended way. You can email me with any thoughts or leave a comment below this post. Anonymous comments are welcome. I am also interested in the views of qualitative researchers on certain specific issues, particularly: 

-- Do you believe that qualitative research within your field suffers from publication bias?

-- Have you ever had a paper rejected because, independently of the methods, the qualitative outcomes were judged to be insufficiently interesting, novel or conclusive?

-- Would you consider using Registered Reports as a submission option if it were offered at your preferred journal?

-- Which journals would you like to see Registered Reports offered within?

-- What do you see as the main benefits or drawbacks of Registered Reports in qualitative research?