Guides · Research credibility and open science

Open science practices in marketing: what journals expect now

Open science in marketing covers planning (preregistration and registered reports), documentation (sharing materials, data and code) and dissemination (preprints and open access). As of September 2024 most leading marketing journals required data and code at some stage of review while only encouraging preregistration. Proprietary data rarely rule out transparency entirely: a data availability statement, documented code and controlled access can replace a public file.

Why marketing is adopting open science

Deer and coauthors trace the push to concerns about selective reporting, p-hacking, undisclosed flexibility in data collection and analysis, and unpublished null results, alongside low reproducibility reported in psychology. They group the responses into three stages of a project: planning before data collection, documentation of materials, data and code, and dissemination of the results. Their paper is a 2024 preprint that its authors list as forthcoming in the International Journal of Research in Marketing.^[1]

The best-known trigger was the Open Science Collaboration's replication of 100 psychology studies, in which 36 percent of replications were significant and 39 percent were subjectively rated as reproducing the original result. Deer and coauthors note that a later reanalysis disputed how pessimistic that conclusion should be, yet the project still shifted psychology toward open practices.^[2][1]

What leading marketing journals require

The following summarises the policy table in Deer and coauthors, which covers the International Journal of Research in Marketing and the marketing journals on the FT50 list, as of September 2024. Policies change, so treat it as orientation and check each journal's current guidelines.^[1]

• Data and code: the Journal of Marketing Research requires a replication package of materials, data and code after conditional acceptance, shared with the coeditor and review team. The Journal of Marketing, Journal of Consumer Research and Journal of Consumer Psychology require files after a revise-and-resubmit decision, and Marketing Letters at first submission.^[1]
• Public posting: Marketing Science and Management Science require files after acceptance, and Marketing Science makes them publicly available by default. The Journal of Marketing and Management Science also use data editors to check that the files reproduce the reported results.^[1]
• Preregistration: encouraged but not required at the Journal of Consumer Research, the Journal of Consumer Psychology and the International Journal of Research in Marketing; Marketing Letters is listed as requiring it on resubmission.^[1]
• Registered reports: offered by the International Journal of Research in Marketing and Psychology and Marketing, with review of the design before data collection and publication guaranteed if the plan is followed.^[1]
• No listed requirement: the Journal of the Academy of Marketing Science and the Journal of Retailing had no data, code or preregistration requirement in the table, and the International Journal of Research in Marketing listed no data or code requirement. Every journal listed offered an open access option.^[1]

Building a replication package as you go

Deer and coauthors recommend organising the project for reuse from the first day rather than assembling files at acceptance, and having coauthors run each other's code before submission.^[1]

• Never edit raw data files. Make every cleaning and transformation step in code, and keep confidential files clearly marked so they are not published by accident.^[1]
• Write a README that explains the folder structure, the purpose of each file, the software versions and how to reproduce the results, plus a codebook describing every variable, its wording, scale and summary statistics.^[1]
• Use relative file paths, one script that runs the full pipeline, and version control; then test whether following your own instructions reproduces the tables on another computer or in a colleague's hands.^[1]
• For experiments, share stimuli and questionnaires in a format readable without the survey software, record the exact models of any equipment, and photograph or film the procedure.^[1]
• Deposit the package in an open repository such as OSF or Zenodo with a DOI and a reuse licence.^[1]
• In the paper, report the stopping rule, list every variable and condition, and show results with and without exclusions and covariates, as Simmons and colleagues require of authors.^[3]

When the data are proprietary

Much marketing research uses licensed scanner data or company data under non-disclosure agreements, and Deer and coauthors acknowledge that such agreements and privacy rules often prevent sharing. They also note an incentive to keep data in house to extract several papers from one dataset. Golder and colleagues call the confidentiality concerns nontrivial but point out that several journals have procedures to balance the interests involved.^[1][4]

Where data cannot be posted, a data availability statement should tell an outside researcher which source holds the data, what terms apply to access, and roughly what obtaining it would cost in money and time. You can still publish all code, codebooks and processing steps, so a replicator with access can rerun the analysis.^[1]

Other options carry trade-offs. Anonymised data, such as scanner data with masked store identities, risk re-identification in large datasets. Synthetic data mimic the statistical patterns of the original but may not protect a company's sensitive figures because key summary statistics survive. Deer and coauthors also propose that data licensors host archived replication files and grant replicators controlled access under agreement.^[1]

Replication and big-team science

Marketing has favoured replications that extend rather than copy. Lynch and colleagues describe how the Replication Corner at the International Journal of Research in Marketing targeted important papers, preferred conceptual replications and replications with extensions, and published failures as well as successes; most of its first thirty replications supported the original findings, sometimes while identifying moderators.^[5]

Urminsky and Dietvorst found that only 5 percent of Journal of Consumer Research papers historically reported independent replication, against 36 percent reporting replication of their own results. They suggest making within-paper replication more credible through detailed preregistration and by posting all relevant studies, including non-replications, with documented data and code.^[6]

For a close replication, Brandt and colleagues offer a 36-question recipe: obtain the original materials from the authors, follow the methods as closely as possible, aim for high power since published effects are likely overestimates, register the plan, and share data and analysis code afterwards.^[7]

Larger collaborations are spreading. Deer and coauthors describe many-lab replications, megastudies that test many interventions at once, and many-analyst projects, including one in which teams estimate the price elasticity of meat substitutes from a common dataset while documenting their analytic choices.^[1]

Costs and open disagreements

The practices are not free, and their advocates do not all agree. The authors of the Deer paper state that they disagree among themselves on which practices the field should adopt. One view stresses that transparency is essential for credibility; the other points to high costs, unequal access to resources, and little causal evidence yet that some practices deliver their promised benefits.^[1]

Golder and colleagues worry that rigid preregistration could discourage exploratory, empirics-first research and note that it may do little for generalizability. Adler and colleagues show how far business journals lag: as of May 2023, no business, management or marketing journal scored above 15 out of 30 on the TOP Factor, a rating of journal transparency policies, while 29 psychology journals did.^[4][8]

What to adopt on your next paper

• Set up the project folder, README and code pipeline before collecting data, so the replication package is ready when a journal asks for it.^[1]
• Preregister confirmatory experiments in full; for archival or predictive work, register the objective and design and document the rest.^[1][9]
• Apply the disclosure rules of Simmons and colleagues in every methods section, whether or not the study was registered.^[3]
• Draft a data availability statement early if your data are licensed, and confirm with the provider what a replicator could access.^[1]
• Post a preprint and, where open access fees are out of reach, share the accepted author version and keep supplementary files in an open repository.^[1]
• Plan a high-powered or independent replication of your central effect rather than relying only on studies run by your own team.^[6][7]

The evidence behind this

• Marketing knowledge: how it is made and when to trust it
• Research methods pitfalls: ratio variables and censored data
• Empirical generalizations in marketing: what the research shows

References

1. Lachlan Deer, Susanne Jana Adler, Hannes Datta, Natalie Mizik, Marko Sarstedt (2024). Toward Open Science in Marketing Research.
2. Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science.
3. Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn (2011). False-Positive Psychology. Psychological Science.
4. Peter N. Golder, Marnik G. Dekimpe, Jake T. An, Harald J. van Heerde, Darren S. U. Kim, Joseph W. Alba (2022). Learning from Data: An Empirics-First Approach to Relevant Knowledge Generation. Journal of Marketing.
5. John Lynch, Eric T. Bradlow, Joel Huber, Donald R. Lehmann (2015). Reflections on the replication corner: In praise of conceptual replications. International Journal of Research in Marketing.
6. Oleg Urminsky, Berkeley J Dietvorst (2024). Taking the Full Measure: Integrating Replication into Research Practice to Assess Generalizability. Journal of Consumer Research.
7. Mark J. Brandt, Hans IJzerman, Ap Dijksterhuis, Frank J. Farach, Jason Geller, Roger Giner‐Sorolla, James A. Grange, Marco Perugini, Jeffrey R. Spies, Anna van 't Veer (2013). The Replication Recipe: What makes for a convincing replication?. Journal of Experimental Social Psychology.
8. Susanne Jana Adler, Lukas Röseler, Martina Katharina Schöniger (2023). A toolbox to evaluate the trustworthiness of published findings. Journal of Business Research.
9. Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor (2018). The Preregistration Revolution. Proceedings of the National Academy of Sciences.

Bibliographic details only. Follow the DOI to read each study at its publisher.