tl;dr – in 2017 we are
expanding the Registered Reports publication model into qualitative research
for the first time. Since we’ve never done this before, we’re very interested
to hear from qualitative researchers about how we can make the format
attractive, and what obstacles or drawbacks we might encounter with
pre-registration of qualitative studies.
Registered Reports are
developing some serious momentum. What began in 2013 as a radical (and somewhat controversial) idea to curb bias in published research is rapidly
becoming a standard addition to the publishing landscape. Twenty-eight journals
have now signed on to offer the format, with more in the pipeline, and across a
wide range of social, life and physical sciences.
For the uninitiated,
Registered Reports are form of empirical publication where the peer review
happens, in part, before the results of studies are known (full details here). Proposals that are
scientifically robust, and which receive positive peer reviews, are then
provisionally accepted for publication. Once the research is complete, the
authors resubmit their manuscript with the results and discussion; it is
re-reviewed and ultimately published, provided the authors adhered to their
protocol, interpreted the evidence reasonably, and met any pre-agreed standards
for assuring data quality.
The great strength of
Registered Reports is the way it tackles bias. By accepting studies in advance
of results, it prevents publication bias – the tendency for journals to
selectively publish results that are considered clear or more attractive. And by requiring authors to pre-specify their research hypotheses
and analysis methods, it also greatly reduces the capacity for biased reporting by authors.
As the popularity of
Registered Reports has increased, so has its scholarly reach: from the initial launch within psychology and
neuroscience it has expanded into political science, biology, and even physical
sciences of physics and chemistry. But within every area, so far, that
Registered Reports have ventured, it has been under the banner of quantitative
research.
To lay my cards on the
table, I'm a firm believer in the value of qualitative research. I gained a deep respect for it after being involved in this study, where we used
qualitative methods (specifically, grounded theory) to explore the way
parliamentarians use science and evidence in decision-making. Qualitative
research can generate remarkably rich information, and in some ways it is more
transparent than quantitative research. For instance, the qualitative researcher explicitly
acknowledges their own bias, and – in our work at least – we publicly archived all of our data to enable qualitative replication.
Publication bias also
appears to be just as much a problem in qualitative research as in quantitative
research. As the authors of this study conclude:
"This suggests a mechanism by which "qualitative publication bias" might work: qualitative studies that do not show clear, or striking, or easily described findings may simply disappear from view. One implication of this is that, as with quantitative research, systematic reviews of qualitative studies may be biased if they rely only on published papers."
If publication bias is as
much a problem for qualitative research as quantitative research, then there is
clearly room for Registered Reports to become a popular option among
qualitative researchers. This opens the door for Registered Reports to be taken
up by fields such as anthropology, cultural studies, criminology, sociology,
and beyond. In 2017 we will be launching Registered Reports in at least one
journal that publishes papers within these fields, among other social sciences
and humanities.
All that said, I am far from
being an expert on qualitative methods, so if you are a qualitative researcher
I would greatly appreciate your input into the design of a qualitative Registered
Reports model. We need your help to get this right and to make it useful.
I am interested in any
critical feedback, however positive or negative, and in a completely open-ended
way. You can email me with any thoughts or leave a comment below this post. Anonymous comments are welcome. I am also interested in the views of qualitative
researchers on certain specific issues, particularly:
-- Do you believe that qualitative research within your field suffers from publication bias?
-- Do you believe that qualitative research within your field suffers from publication bias?
-- Have you ever had a paper rejected because, independently of the methods, the qualitative outcomes were judged to be insufficiently interesting, novel or conclusive?
-- Would you consider using Registered Reports as a submission option if it were offered at your preferred journal?
-- Which journals would you like to see Registered Reports offered within?
-- What do you see as the main benefits or drawbacks of Registered Reports in qualitative research?