The things you hate most about submitting manuscripts

A few days ago I asked the twittersphere what rubs people the wrong way when it comes to submitting manuscripts to peer reviewed academic journals. Oh let us count the ways. From the irritation of having to reformat references to fit some journal’s arbitrary style, to consigning figures and captions to the end of a submission as though it really is still 1988, to the pointlessness of cover letters where all you want to say is “Dear Editor, here is our paper” but feel the need to throw in some gumpf about how amazing your results are. (Hint: aside from when the cover letter has a specific purpose, such as summarising a response to reviewers or conveying vital information about a key issue, I can tell you that a lot of editors -- maybe most -- ignore this piece of puffery).

The tweet proved a lot more popular than I expected and for a good two days you could see a steam of delicious rage rising from my timeline. 

I had an ulterior motive in seeking out this information from your good selves. As most of you will know, one of my aims is to help improve the transparency and reproducibility of published research, and one of the journals I edit for is working through its (future) adoption of the new Transparency and Openness Promotion (TOP) guidelines. The TOP guidelines are a self-certification scheme in which journals voluntarily report their level of policy compliance with a series of transparency standards, such as data sharing, pre-registration, and so forth. TOP is currently endorsed by over 500 journals and promises to make the degree of transparency adopted by journals itself more transparent. I guess you could call this "meta-transparency".

Now, in putting together our TOP policy at this journal at which I serve, we realised that it involves the addition of some new submission bureaucracy for authors. There will be a page of TOP guidelines to read beforehand and a 5-minute checklist to complete when actually submitting. We realise extra forms and guidelines are annoying for authors, so at the same time as introducing TOP we are going to strive to cut as much of the other (far less important) shit as possible. 

Here are the things you hated the most, and your most popular recommendations. For fun, I calculated an extremely silly and invalid score of every interaction to this tweet, adding up RTs, favourites and the number of independent mentions of specific points:

1. Abolish trivial house style requirements, including stipulations on figure dimensions & image file types, especially for the initial submission, as well as arbitrary house referencing and in-text citations styles. This is by far the most popular response. (score 112)

2. Allow in-text figures and tables according to their natural position until the very final stage of submission. (score 61)

3. Abolish all unnecessary duplication of information about the manuscript (e.g. word count, keywords), main author details and (most especially) co-author contact details that is otherwise mentioned on the title page or could be calculated automatically; abolish any requirement to include postal addresses of co-authors at least until the final stage (affiliation and email address should be sufficient, and should be readable from title page without requiring additional form completion); eliminate fax numbers altogether because, seriously, WTF are those fossils doing there anyway. (score 50)

4. Abolish requirement for submissions to be in MS Word format only. (score 36)

5. Abolish endnotes and either replace with footnotes or cut both. (score 33)

6. Allow submission of LaTeX files. (score 29)

7. Allow submission of single integrated PDF until the final stage of acceptance. (score 27)

8. Abolish cover letters for initial submissions. (score 21)

9. Abolish the Highlights section altogether because 

* Highlights are Stupid 

* Everyone knows Highlights are Stupid

* I can't think of anything else to say here, so I'll just repeat the conclusion that Highlights are Stupid (score 18) 

10. Remove maximum limits on the number of cited references. (score 7) 

11. Abolish the requirement for authors to recommend reviewers. (score 7) 

12. Increase speed of user interface. (score 6)

Not all of these apply to our journal, but we’ll try and improve on the things that do, and which we can change. 

Oh, and lucky number 13, which actually scored the same as abolishing cover letters, goes to Sanjay Srivastava: "Getting rejected, can you do away with that?” Alas that is beyond my current lowly powers, although...cough....I am getting there.* 

---------------------------------------

* Shameless plug alert: At one journal I edit for (Cortex), submitting a pre-registered article called a Registered Report greatly increases your chances of being published.  The rejection rate for standard (unregistered) research reports? Just over 90%.  The rejection rate for the 50% of Registered Reports that pass editorial triage and proceed to in-depth Stage 1 peer review? About 10%.  

The reason the rejection rate is so low for Registered Reports isn’t because our standards are any less (if anything they are higher, in my opinion) but because this format attracts  particularly good submissions and also gives authors the opportunity to address reviewer criticisms of their experimental design before they do their research – a point made by Dorothy Bishop who recently published an excellent Registered Report with Hannah Hobson.

So you've been scooped

It’s the moment every junior researcher dreads – and more than a few senior ones too. You’re on the verge of submitting that amazing paper describing a new and exciting finding, or a hot new method, and someone beats you to the post. 

That sinking feeling when you read the abstract in a zeitgeist journal announcing that  “Here we show for the first time….” followed by something achingly similar to what you have done. The rug has been ripped out. You’ve been cruelly gazumped with nothing left but doubts and self-recriminations. They will get all the credit and nobody will care what you did. You’ll be seen as some lame copycat following in their illustrious tailwind, even though you conceived your idea long before they published theirs. If only you’d worked harder. Worked more Sundays instead of spending time with family or friends. Written faster. Spent less time on Twitter. And the worst part is you had no clue that you were about to be gazumped. You’ve been blindsided.

The chances are, if you work in a busy or popular area using techniques that are widely available, this is going to happen to you at some point. And I’m going to try to convince you that unless your research falls within a very narrow set of parameters, it doesn’t matter. Not one bit.

It really doesn’t. Despite all the feelings of frustration and disappointment it provokes, this is all in your head. It is your own ego screaming into the void. On the contrary there are several positive sides to being “scooped”. (Note I refer to “scooped” here to refer to the kind of inadvertent gazumping that can happen when multiple researchers work independently but in parallel – I am not referring to the deliberate theft of ideas, which is extremely rare if it happens at all).

Here are some tips for junior researchers on how to come to grips with being scooped and why you shouldn’t feel so bad.

1.    It means you are doing something other people care about. Getting scooped is a sign that your research is important and that you are probably asking the right questions. If someone finds something similar to you it also adds to the convergent validity of your methods and suggests you may be doing work that is reproducible. Note: the corollary of this is not the case – just because you never get scooped doesn’t mean your research is unimportant. You might have cornered the market in a particular technique, or the field might be small, or your approach might be unusual or specialised in some other way.

2.    Being first isn’t necessarily a sign of being a good scientist. Why? Because many initial discoveries are wrong or overclaimed. As a post-doc, I was the “first” to show that TMS of the right inferior frontal cortex can impair response inhibition in healthy people. So what? Does that make my methods or results more convincing, or any better than later convergent findings? Does it make me a better scientist? Nope, nope, nope. If anything, my paper is weaker because it overclaimed. When I and my co-authors wrote it we knew we were the first to report this particular effect, so we aimed “high” with journals and over-egged the cake. We initially submitted it to a bunch of zeitgeist journals where it was predictably rejected, one after another (after all, we were only repeating what had already been concluded on the basis of brain injury). The spin remained, though, until it found its way into a specialist journal, and on the basis of the results we claimed evidence for a selective role of the IFG in response inhibition. We were wrong, as we and others later discovered – the original results turned out to be repeatable but our explanation was trite and erroneous.

3.    Most senior scientists know this. Many PIs – me included – are sceptical of researchers who claim to be the first to show something. For one thing it is almost never the case; the vast majority of science is a process of derivative, incremental advance, despite whatever spin the authors cake their abstracts in. When I’m assessing fellowship applications or job applications by junior researchers, the type of questions I’m asking are: is this research important either to theory or applications? Is it robust, feasible and transparent? Is the applicant an excellent communicator? I am not asking whether they were the first at making previous claims. I couldn’t care less. Knowing what I do about statistics and research culture, I know that s/he who claims they are first most likely did a small study, did not take the time to replicate their findings, fell prey to research bias, benefited from publication bias, and probably exaggerated the implications. Are these attractive characteristics in a scientist?

4.    In the vast majority of cases you don’t show you are a brilliant scientist or intellectual force by being the first to claim something. You prove your mettle by shaping the theoretical landscape in which everyone works. You set the scene, one of two ways. One way is by accruing a coherent body of important and credible work that changes the way people think about a topic (and not just by publishing a long list of glamour publications, but through the transparent accumulation of knowledge). Or, you construct a robust and falsifiable theory that could explain something better than all the other theories out there, and then set about trying to disconfirm it. If it is brilliant, others will try doing the same, and if nobody can disconfirm it then you've probably discovered something for real.

5.    There are a few cases where being the first might matter and can have career benefits. If you’re the first to describe an amazing new technique, or the first to make a Nobel-level discovery then scooping might count. But how many of us fall into that category? 0.0001%? The rest of us are labouring away in the trenches. Our discoveries are small and, frankly, none of us individually matter a great deal. Our value lies in our collective contribution as scientists. A large part of getting over being scooped is getting over yourself and realising that you are a small cog in a very big machine.

6.    Remember that what matters in science is the discovery, not the discoverer. That’s why the public pays your salary or stipend. When someone scoops you, it provides an opportunity for you to reflect on their findings in preparing your own paper. What can you learn from what they found, or from the data itself? If you have access to their data, can you perform a meta-analysis to aggregate evidence usefully between their study and your own? Might they be someone you could collaborate with on a future study to do something even bigger and better than either of you could do alone? Remember that in the quest to make discoveries, competition is for climbers and egomaniacs. Cooperation beats competition every time.

7.    Finally, if you really feel you have an idea for a study that is unique and you want to declaw the Scoop Monster, consider submitting it as a Registered Report. This might seem counterintuitive – after all, aren’t Registered Reports only for incremental research or replications? Aren't you risking being scooped by sharing your amazing idea with reviewers? Actually, you're more protected than you think, and Registered Reports are not limited to replications; they are simply an avenue for robust, transparent, hypothesis-driven research, and they can (and often do) describe novel ideas or critical tests of theory. Aside from all those benefits, Registered Reports offer something very simple that asserts intellectual primacy: when they are published, the date that the initial Stage 1 protocol was first received is published in the margin, right above all the other received and accepted dates. This means that if anyone publishes anything similar in the meantime, you will always be able to prove – if it really matters – that you had your idea before they published theirs. Plus your study will probably be three times the size and relatively bias-free.

  

Now, get back to sciencing (or chilling out) and leave the worrying about scooping to scientists who don't really understand how science works or why they are doing it.