Thursday 7 April 2016

So you've been scooped


It’s the moment every junior researcher dreads – and more than a few senior ones too. You’re on the verge of submitting that amazing paper describing a new and exciting finding, or a hot new method, and someone beats you to the post

That sinking feeling when you read the abstract in a zeitgeist journal announcing that  “Here we show for the first time….” followed by something achingly similar to what you have done. The rug has been ripped out. You’ve been cruelly gazumped with nothing left but doubts and self-recriminations. They will get all the credit and nobody will care what you did. You’ll be seen as some lame copycat following in their illustrious tailwind, even though you conceived your idea long before they published theirs. If only you’d worked harder. Worked more Sundays instead of spending time with family or friends. Written faster. Spent less time on Twitter. And the worst part is you had no clue that you were about to be gazumped. You’ve been blindsided.

The chances are, if you work in a busy or popular area using techniques that are widely available, this is going to happen to you at some point. And I’m going to try to convince you that unless your research falls within a very narrow set of parameters, it doesn’t matter. Not one bit.

It really doesn’t. Despite all the feelings of frustration and disappointment it provokes, this is all in your head. It is your own ego screaming into the void. On the contrary there are several positive sides to being “scooped”. (Note I refer to “scooped” here to refer to the kind of inadvertent gazumping that can happen when multiple researchers work independently but in parallel – I am not referring to the deliberate theft of ideas, which is extremely rare if it happens at all).

Here are some tips for junior researchers on how to come to grips with being scooped and why you shouldn’t feel so bad.

1.    It means you are doing something other people care about. Getting scooped is a sign that your research is important and that you are probably asking the right questions. If someone finds something similar to you it also adds to the convergent validity of your methods and suggests you may be doing work that is reproducible. Note: the corollary of this is not the case – just because you never get scooped doesn’t mean your research is unimportant. You might have cornered the market in a particular technique, or the field might be small, or your approach might be unusual or specialised in some other way.

2.    Being first isn’t necessarily a sign of being a good scientist. Why? Because many initial discoveries are wrong or overclaimed. As a post-doc, I was the “first” to show that TMS of the right inferior frontal cortex can impair response inhibition in healthy people. So what? Does that make my methods or results more convincing, or any better than later convergent findings? Does it make me a better scientist? Nope, nope, nope. If anything, my paper is weaker because it overclaimed. When I and my co-authors wrote it we knew we were the first to report this particular effect, so we aimed “high” with journals and over-egged the cake. We initially submitted it to a bunch of zeitgeist journals where it was predictably rejected, one after another (after all, we were only repeating what had already been concluded on the basis of brain injury). The spin remained, though, until it found its way into a specialist journal, and on the basis of the results we claimed evidence for a selective role of the IFG in response inhibition. We were wrong, as we and others later discovered – the original results turned out to be repeatable but our explanation was trite and erroneous.

3.    Most senior scientists know this. Many PIs – me included – are sceptical of researchers who claim to be the first to show something. For one thing it is almost never the case; the vast majority of science is a process of derivative, incremental advance, despite whatever spin the authors cake their abstracts in. When I’m assessing fellowship applications or job applications by junior researchers, the type of questions I’m asking are: is this research important either to theory or applications? Is it robust, feasible and transparent? Is the applicant an excellent communicator? I am not asking whether they were the first at making previous claims. I couldn’t care less. Knowing what I do about statistics and research culture, I know that s/he who claims they are first most likely did a small study, did not take the time to replicate their findings, fell prey to research bias, benefited from publication bias, and probably exaggerated the implications. Are these attractive characteristics in a scientist?

4.    In the vast majority of cases you don’t show you are a brilliant scientist or intellectual force by being the first to claim something. You prove your mettle by shaping the theoretical landscape in which everyone works. You set the scene, one of two ways. One way is by accruing a coherent body of important and credible work that changes the way people think about a topic (and not just by publishing a long list of glamour publications, but through the transparent accumulation of knowledge). Or, you construct a robust and falsifiable theory that could explain something better than all the other theories out there, and then set about trying to disconfirm it. If it is brilliant, others will try doing the same, and if nobody can disconfirm it then you've probably discovered something for real.

5.    There are a few cases where being the first might matter and can have career benefits. If you’re the first to describe an amazing new technique, or the first to make a Nobel-level discovery then scooping might count. But how many of us fall into that category? 0.0001%? The rest of us are labouring away in the trenches. Our discoveries are small and, frankly, none of us individually matter a great deal. Our value lies in our collective contribution as scientists. A large part of getting over being scooped is getting over yourself and realising that you are a small cog in a very big machine.

6.    Remember that what matters in science is the discovery, not the discoverer. That’s why the public pays your salary or stipend. When someone scoops you, it provides an opportunity for you to reflect on their findings in preparing your own paper. What can you learn from what they found, or from the data itself? If you have access to their data, can you perform a meta-analysis to aggregate evidence usefully between their study and your own? Might they be someone you could collaborate with on a future study to do something even bigger and better than either of you could do alone? Remember that in the quest to make discoveries, competition is for climbers and egomaniacs. Cooperation beats competition every time.

7.    Finally, if you really feel you have an idea for a study that is unique and you want to declaw the Scoop Monster, consider submitting it as a Registered Report. This might seem counterintuitive – after all, aren’t Registered Reports only for incremental research or replications? Aren't you risking being scooped by sharing your amazing idea with reviewers? Actually, you're more protected than you think, and Registered Reports are not limited to replications; they are simply an avenue for robust, transparent, hypothesis-driven research, and they can (and often do) describe novel ideas or critical tests of theory. Aside from all those benefits, Registered Reports offer something very simple that asserts intellectual primacy: when they are published, the date that the initial Stage 1 protocol was first received is published in the margin, right above all the other received and accepted dates. This means that if anyone publishes anything similar in the meantime, you will always be able to prove – if it really matters – that you had your idea before they published theirs. Plus your study will probably be three times the size and relatively bias-free.
  

Now, get back to sciencing (or chilling out) and leave the worrying about scooping to scientists who don't really understand how science works or why they are doing it.

7 comments:

  1. Thanks for this awesome post. I feel a lot better after reading this.

    ReplyDelete
  2. Hahaha, I was joking about you scooping my post but our posts really are quite similar in many respects - even structurally. I think I'll survive this affront and instead go to bed... ;)

    ReplyDelete
    Replies
    1. Hahaha. A perfect object lesson in why being scooped doesn't matter.
      I scooped you but you took extra 3 hours and wrote a better one.

      For those who haven't seen it: http://neuroneurotic.net/2016/04/08/3-scoops-of-vanilla-science-in-a-low-impact-waffle-please/

      Delete
  3. Speaking of point 7 - do you know of any case where a preprint or a preregistation was used to correct a false claim of primacy in a published paper?

    ReplyDelete
    Replies
    1. Not that I've seen, so far. There may be examples in other fields, though, like some parts of physics, where preprints are the norm.

      Delete
  4. And there is a perfect place to put what my advisor told me when I was a wee little graduate student and got frustrated that all of my ideas have been done before. I had quite a bizarre idea of "done before", anything that remotely resembled the study I was planning included, and unlike most people I read neuropsychological literature from the 60s and 70s, which as you yourself refer to show a student that everything has been done before! Anyway, Elizabeth Bates she said to me, you can always do it better. Did they do it right? Sometimes it may seem not the best argument to make to publish in a Glamour journal that this is not the first time but it is the best time or something like that but science itself improves enough because you did it first but you did it better and better and better... I do understand the worry and I have been through the worst kinds of scooping, with competing review were on purpose killing your paper type of thing, but that doesn't happen very often and even that's survivable. Good points overall.

    ReplyDelete
    Replies
    1. And I had meant to post this under where Sam has been scooped in terms of blog posting but on my phone somehow it showed up in the mean comments. Also, excuse the dictation errors I hope what I mean is still somewhat comprehensible.

      Delete