Guides · Research credibility and open science

Preregistration in marketing research: what to register and when

Preregistration means filing your hypotheses, sampling plan and analysis plan in a time-stamped registry before you see the outcomes, so readers can tell tested predictions apart from ideas the data suggested. It fits confirmatory experiments best; for archival, exploratory or predictive marketing work, a lighter registration plus a record of deviations still adds credibility. Leading marketing journals mostly encourage it rather than require it, and the direct evidence that it improves reproducibility is still largely correlational.

What preregistration is and what it protects against

A preregistration is a commitment made before the outcomes are known. You write down the research question, hypotheses, design and analysis plan and deposit them in an independent registry, which keeps a dated record and can hold it private for an embargo period. Nosek and colleagues stress that the goal is not to rank prediction above exploration. It is to make clear, after the fact, which analyses tested a prediction and which were generated by looking at the data.^[1]

The distinction matters because a p-value only means what it claims when the tests and the path to them were fixed in advance. Real analyses involve choices about exclusions, transformations, covariates and which groups to compare, and those choices are easy to make after seeing the data. Hindsight then makes the data-driven route feel as if it had been the plan all along, which is why good intentions alone do not solve the problem.^[1]

Simmons, Nelson and Simonsohn showed how fast these choices add up. In simulations where no true effect existed, testing two correlated outcomes, adding ten participants per cell after a first look, optionally controlling for gender, and optionally dropping one of three conditions each pushed the false-positive rate above the nominal 5 percent. Used together, they produced a significant result in about 61 percent of simulated studies. Their own remedy was disclosure, but these are exactly the decisions a preregistration fixes in advance.^[2]

How strong the evidence for it is

The case for preregistration rests mainly on the logic of statistical inference plus correlational evidence. Nosek and colleagues state that experimental proof of better reproducibility did not yet exist when they wrote. They point instead to a sharp drop in positive results after clinical trials had to register primary outcomes, and to a sample of preregistered studies in which 40 percent of published papers left out at least one experimental condition and 70 percent left out at least one outcome.^[1]

Much of the marketing-specific advice below comes from Deer and coauthors, a 2024 preprint that its authors list as forthcoming in the International Journal of Research in Marketing. It is a practical review, not an empirical test. Its authors also say openly that they disagree among themselves about how far the field should go, with one view noting that causal evidence on the benefits of open science practices is still scarce.^[3]

What a marketing preregistration should contain

Deer and coauthors suggest a simple workflow: decide to register, pick a form, fill it in, file it, run the study, then report any deviations in a separate document. Short templates exist, such as the eight-question AsPredicted form, alongside fuller OSF templates and specialised ones for secondary data, qualitative research and replications. Nosek and colleagues add a caveat: AsPredicted forms can stay private forever, so post the completed form to a public registry if you want it to count as a permanent record.^[3][1]

• Objective and hypotheses: the research question, each hypothesis, and the unit of analysis.^[3]
• Constructs and variables: independent and dependent variables, plus any mediators, moderators and controls.^[3]
• Design and measurement: the experimental conditions and how each construct is measured.^[3]
• Sample: selection and exclusion criteria, the target sample size with its justification such as a power analysis, and the rule for when data collection stops.^[3][2]
• Data collection: the mode, whether lab, online or secondary data, and for secondary data the sources and acquisition dates.^[3]
• Data preparation: missing-data handling, weighting, reliability checks, transformations and any convergence thresholds.^[3]
• Analysis: the planned tests, any endogeneity tests or model baselines, heterogeneity or mechanism analyses, the exploratory analyses you already expect to run, and fit measures for prediction tasks.^[3]

Experiments versus archival, exploratory and predictive work

A consumer behaviour experiment usually follows a set procedure with a predictable data structure, so its registration can be detailed. Deer and coauthors warn that a full pre-analysis plan for an exploratory scanner-panel study or a predictive model built on web-shop data tends to generate many deviations. For such projects they suggest a free-format registration of the objective, sampling, design and available variables, with the main effort going into documenting data preparation and code.^[3]

Pre-existing data are the hardest case. Nosek and colleagues note that a genuine test is still possible if nobody on the team has seen the relevant variables, but blinding weakens once you have read summaries of the dataset or seen related variables that correlate with your outcomes. Their advice is to register anyway and state plainly what was already known about the data, so readers can judge the risk for themselves.^[1]

Golder and colleagues, writing in defence of empirics-first research, observe that behavioural studies lend themselves to registration more readily than compiled or purchased data, and that confidentiality agreements with data providers are a real constraint. They argue that journals should accept clearly labelled post hoc findings, while treating hypotheses invented after the results and reported as if planned as unacceptable.^[4]

Deviations and alternatives when a full plan is impossible

Deviations are normal: recruitment falls short, participants do something unforeseen, or a variable violates a test's assumptions. Nosek and colleagues argue that a registration with transparently reported deviations still supports far more confident inference than no registration at all. Deer and coauthors recommend listing every deviation in a separate document, including changes that reviewers requested.^[1][3]

• Staged registration: register the checks on distributions and exclusions first, then register the outcome model once those checks are done.^[1]
• Decision trees and standard operating procedures: specify in advance which test follows each assumption check, or adopt lab-wide rules for routine choices such as clustered standard errors or one- versus two-tailed tests.^[1][3]
• Split samples: explore freely on one part of the data and keep a holdout sealed until the resulting model has been registered.^[1][3]
• Blind analysis: hide treatment labels or scramble outcomes while cleaning the data, so decisions cannot track the results.^[1][3]
• Registered reports: the design is peer reviewed before data collection and an in-principle acceptance guarantees publication if the plan is followed, whatever the results. The International Journal of Research in Marketing and Psychology and Marketing offer this format.^[3][1]

What preregistration does not fix

Registration does not stop selective reporting across studies or selective emphasis in the write-up. A researcher who registers twenty studies and publishes the single hit, or who runs ten registered analyses and builds the story on two, still leverages chance. Nosek and colleagues note that only complete reporting of every registered outcome lets readers correct for this.^[1]

For replications, Brandt and colleagues treat registration as one ingredient among several: define the target effect precisely, follow the original methods closely, secure high statistical power, and register the methods, the differences from the original and the success criteria before collecting data so the criteria cannot move later.^[5]

Golder and colleagues add that both sides of the preregistration debate accept it may do little for generalizability or robustness. In their view, confidence that a finding generalizes comes from broad investigation across settings rather than from registration.^[4]

What marketing journals expect

In the journal policy table compiled by Deer and coauthors, current as of September 2024, the Journal of Consumer Research, the Journal of Consumer Psychology and the International Journal of Research in Marketing encouraged preregistration without requiring it. Marketing Letters is listed as encouraging it at first submission and requiring it on resubmission, while the other leading journals in the table had no preregistration policy. Data and code requirements were far more common, so check current author guidelines before you submit.^[3]

A practical rule follows from the sources: register every confirmatory experiment in full, register what you can for archival or predictive work and document the rest, and keep a running log of deviations from the first day of data collection.^[3][1]

The evidence behind this

• Marketing knowledge: how it is made and when to trust it
• Research methods pitfalls: ratio variables and censored data

References

1. Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor (2018). The Preregistration Revolution. Proceedings of the National Academy of Sciences.
2. Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn (2011). False-Positive Psychology. Psychological Science.
3. Lachlan Deer, Susanne Jana Adler, Hannes Datta, Natalie Mizik, Marko Sarstedt (2024). Toward Open Science in Marketing Research.
4. Peter N. Golder, Marnik G. Dekimpe, Jake T. An, Harald J. van Heerde, Darren S. U. Kim, Joseph W. Alba (2022). Learning from Data: An Empirics-First Approach to Relevant Knowledge Generation. Journal of Marketing.
5. Mark J. Brandt, Hans IJzerman, Ap Dijksterhuis, Frank J. Farach, Jason Geller, Roger Giner‐Sorolla, James A. Grange, Marco Perugini, Jeffrey R. Spies, Anna van 't Veer (2013). The Replication Recipe: What makes for a convincing replication?. Journal of Experimental Social Psychology.

Bibliographic details only. Follow the DOI to read each study at its publisher.