Registered Reports: Peer Review Before Results

Most peer review judges a study after the results are in. Registered reports ask reviewers to judge the question and methods first, before any data exist.

Explainer · 9 min read · Directive Publications Editorial Team

What changes when review comes first

In conventional peer review, reviewers see your question, methods, and results together, and the results can colour the judgement. A clear positive finding can make a modest design look acceptable, while a null result can make the same design look underpowered or badly chosen.

Registered reports change the order. You submit the introduction, hypotheses, methods, and analysis plan before collecting data, and reviewers judge whether the question is worth answering and the design can answer it. If they are satisfied, the journal issues an in-principle acceptance: a commitment to publish the finished study regardless of what it finds.

The Center for Open Science keeps the main list of journals offering registered reports, more than 300 including one-off special issues, and sets two minimum features. Review happens before the outcomes are observed, and in-principle acceptance cannot be revoked because of the results, only for failed quality assurance, failure to follow the protocol, or unresolvable problems with reporting clarity or style.

For a senior author, the practical effect is that the reward moves from producing a tidy story to running the study you promised. That matters most where null findings tend to go unpublished, a problem we looked at in why negative and null results matter.

How two-stage review works

Stage 1: the protocol

A Stage 1 registered report is a paper without results. It usually contains:

• an introduction that motivates the question and states each hypothesis precisely
• methods detailed enough for another team to repeat them without contacting you
• inclusion and exclusion criteria, including how and when data will be excluded or replaced
• a complete analysis plan, with covariates and any correction for multiple comparisons
• a sampling plan, such as a power analysis or a Bayesian equivalent
• outcome-neutral checks, such as positive controls, that show whether the study was capable of testing its hypotheses
• pilot data, where the journal asks for it

The cover letter matters too. Nature Human Behaviour's registered report guidelines, for example, ask authors to confirm that funding and facilities are in place, that data collection will not begin until in-principle acceptance, and to give a timeline for completing the study.

The decision is reject, revise, or accept in principle. Expect substantive revisions, because this is the only point at which criticism can still improve the design; the approach in responding to peer review comments applies directly.

After acceptance, the approved protocol is registered, publicly or under embargo until Stage 2. The OSF Registry has a dedicated "Registered Report Protocol Preregistration" option, and some journals register the protocol for you. Some journals, such as PLOS ONE, also publish the accepted protocol as an article in its own right.

Stage 2: the completed paper

When the study is finished, you add results and a discussion to the approved introduction and methods. The introduction and hypotheses stay as approved, apart from changing future tense to past and correcting factual or typographical errors. Literature published after acceptance belongs in the discussion. Every registered analysis is reported, and anything else goes in a separate section labelled as exploratory.

Stage 2 decisions are not based on whether the results seem important, novel, or conclusive. Journals encourage or require public data and code, and some ask for a laboratory log of data collection dates to confirm the data post-date acceptance.

What in-principle acceptance does and does not promise

In-principle acceptance is conditional. At Nature Human Behaviour, for example, publication is pending completion of the approved methods and analyses, passing of all pre-specified quality checks, and a defensible interpretation of the results. That leaves three ways to lose it, and none involves the direction of the result.

1. An unapproved deviation from the protocol. Changes you did not clear with the editor can lead to rejection at Stage 2, including ones that seem minor to you.
2. Failed outcome-neutral checks. If the checks that show the study could test its hypotheses fail, the paper can be rejected, although an editor may instead offer the chance to add a redesigned study.
3. Conclusions the data do not support. Nature Human Behaviour's guidelines, for instance, warn authors not to base conclusions entirely on significant post hoc analyses.

You are not locked in. You can withdraw at any stage, but many journals then publish a short Withdrawn Registration notice giving the Stage 1 abstract and a brief reason. Some also treat a study not completed by the agreed Stage 2 deadline as withdrawn, so ask for an extension early if recruitment slips.

Registered reports, preregistration, and results-blind review

Preregistration is a time-stamped plan you deposit yourself. Nobody reviews it, and nobody commits to publishing the outcome.

Clinical trial registration is still required. The ICMJE recommends that medical journals require registration in a public registry at or before the time of first patient consent for enrolment, and a registered report does not replace that. BMC Medicine, whose registered report guidelines explicitly include clinical trials, also requires a completed SPIRIT checklist and figure for them.

Results-blind review, sometimes called results-free review, judges a completed study with the results and discussion withheld. The outcome no longer drives the decision, but because the data already exist, reviewers cannot improve the design.

A conventional published protocol is peer reviewed, but unless the journal offers it as a registered report, it carries no commitment to publish the results.

Which studies suit the format

A registered report is worth considering when all of the following are true:

• You can state the question, and hypotheses where relevant, before seeing any data.
• You can specify the analysis, including contingencies written as rules: if the data show A, you will run analysis A1; if B, analysis B1.
• Data collection has not started, or, for existing data, you have not yet analysed or accessed them in ways that could bias the plan.
• Funding and facilities are already in place.
• Your timeline can absorb Stage 1 review before the study begins.

Beyond confirmatory research with new data, eligibility varies. Some journals accept secondary analyses of existing datasets if you can show no prior access; Nature Human Behaviour asks for evidence such as self-certification or a letter from an independent gatekeeper. Some accept qualitative studies, systematic reviews, or meta-analyses, and others explicitly do not.

There is also a journal-independent route: Peer Community In Registered Reports (PCI RR) reviews both stages as preprints, and journals listed as "PCI RR-friendly" have committed to publishing reports it recommends, subject to their own conditions.

The format is a poor fit for purely exploratory work and for any study already under way.

Preparing a strong Stage 1 submission

1. Read the target journal's registered report guidelines before finalising the design, because power thresholds, eligible study types, and data-sharing rules differ.
2. Justify the effect size behind your power analysis. Nature Human Behaviour, for instance, requires a priori power of 0.95 or higher for frequentist tests, based on the lowest available or meaningful effect size estimate.
3. Number each question and hypothesis, and link each one to its sampling plan, its analysis, and the result that would support or contradict it.
4. State what you will not do as well as what you will, such as how many times you will chase missing data.
5. Define outcome-neutral checks that are independent of the hypotheses.
6. Plan ethics approval around review. Some journals require it before in-principle acceptance, and design changes made at Stage 1 may need an amendment.
7. For a clinical trial, register prospectively and complete any reporting checklist the journal requires.
8. Agree a realistic Stage 2 date with co-authors, including recruitment time.

When the protocol meets reality

If something has to change between acceptance and Stage 2, the deciding question is whether the change could alter the kind or validity of the inferences you can draw.

Keep a time-stamped record of every change as it happens, not at write-up. Make sure trainees collecting data know that a small unreported change can cost the paper; it belongs with the other research integrity responsibilities of a lab lead.

Deciding whether to use it

Registered reports move work to the start of a project: the detailed plan, the power analysis, and the arguments with reviewers all happen before data collection begins. In return, publication no longer depends on the result.

Check whether your target journal offers the format and which study types qualify, then draft the analysis plan first. If you cannot write that plan honestly before seeing data, a conventional article is the better choice. Directive Publications describes its peer review process and author guidelines publicly, and you can contact the editorial team if you are unsure which article types a journal accepts.