Guides · Replication and generalization

Direct vs conceptual replication: what each result can tell you

A direct, or close, replication repeats an earlier study's procedures as faithfully as possible and mainly tells you whether the result can be reproduced. A conceptual replication tests the same hypothesis with different operationalizations and tells you how far the idea generalizes. A failure of either kind is not conclusive on its own, and marketing editors have generally favored conceptual replications and replications with extensions.

Two kinds of replication on one continuum

A direct or exact replication tries to rerun an earlier study as closely as possible, with the same materials, manipulations, measures and kind of participants. A conceptual replication tests the same underlying hypothesis while letting the operationalizations, the study design and the participant population differ, sometimes substantially.^[1]

In practice the line is blurry. Participants, times and places change, and the meaning of a stimulus can drift, so no social science study can be repeated identically and many intended direct replications turn out to be partly conceptual. For that reason some authors prefer the term close replication, meaning a study whose only differences from the original are the unavoidable ones.^[1][2][3]

A statistical argument goes further. A truly identical repeat would be pointless even if it were possible, because it could only return the same result; the value of a replication lies in getting the same result despite known differences in conditions.^[4]

A finer vocabulary for replication types

A 2024 review in the Journal of Consumer Research replaces the two labels with four types, ordered by how far they depart from the original study.^[5]

• Exact replication: only the sample changes, as when a model is tested on a holdout sample. It asks whether the original result arose by chance.^[5]
• Study-design replication: a later rerun that keeps every element judged theoretically relevant and changes only what logistics force. This is what many researchers call a direct replication.^[5]
• Methodological replication: reuses an intervention, scale or other method in new conditions to check that it still works as intended.^[5]
• Theory-test replication: tests the same prediction while potentially changing everything else, which matches most uses of the term conceptual replication. A study that tests a different implication of the theory is not a replication in this scheme.^[5]
• A parallel pair from the statistics literature: close replications keep nearly all known conditions the same, while differentiated replications deliberately vary major conditions such as firms, industries or countries to extend a result or find where it stops.^[4]

What a successful replication tells you

A successful direct replication mainly raises confidence that the original method produces the effect and adds only modestly to confidence in the theory. A successful conceptual replication raises confidence in how far the theory reaches and in the original study's interpretation.^[1]

The difference comes from background factors. If a direct replication holds constant the same incidental details as the original, both results could reflect an interaction with one of those details. When an effect survives changes in operationalizations and settings, that explanation becomes much less plausible.^[3]

Direct replication can also reproduce a flaw. An early dissonance experiment was directly replicated many times, across decades and populations, while carrying a design problem that compromised random assignment. Research on group decisions kept reusing materials tilted toward risky choices, which hid the fact that groups can shift toward caution as well.^[1]

Guidelines written for close replications accept this limit: a close replication inherits any confounds in the original design, and removing them requires a conceptual replication or a replication with extension.^[2]

What a failed replication tells you

Every empirical test joins a hypothesis to auxiliary assumptions about how to manipulate and measure the constructs, so a failure never isolates the theory as the culprit. It may mean the idea is wrong, the operationalization failed, or both.^[1][6]

The closer the replication, the more a failure counts against the original, because there is no deliberate difference to blame and the problem most likely lies in the study's underlying concept, planning or execution. A failed differentiated or theory-test replication is harder to read, since any of the changed factors could be responsible.^[4][5]

Even a failed direct replication stays ambiguous. It may show that the original was a false positive, or that the effect depends on something that differed between the studies. A multi-lab project that followed one preregistered protocol still found significant unexplained differences between labs for 8 of the 16 effects it tested.^[3]

Failures still carry weight as they accumulate. In a Bayesian reading, each well-conducted failure should lower confidence in the original finding by an amount that depends on the quality of the replication's assumptions. No single replication is decisive, but a series of them is informative.^[6]

A proposed hidden moderator needs its own test. Explaining a failure by some unmeasured difference between studies is credible only when a follow-up study tests that difference directly, and this applies to direct and conceptual replications alike.^[1]

The case for starting close

Some methodologists argue that conceptual replication without direct replication cannot shift beliefs, because a conceptual failure can always be attributed to the new method. On this view a finding must first be shown to be reliable before the theory behind it is tested, and while little is known about which design features matter, replications should sit nearer the direct end of the continuum.^[6]

Close replication is also the inexpensive first step in a research program. It shows quickly whether a new result can be repeated at all, and only after some close repetitions succeed do differentiated replications become the better investment.^[4]

When a specific procedure will be used for policy, program evaluation or practice, repeating that exact procedure matters as much as the theory, and even advocates of conceptual replication consider exact replication necessary there.^[1]

How marketing and consumer research editors lean

Marketing's best-known replication section chose conceptual replications and replications with extensions, arguing that exact replication is impossible and that learning where a finding holds is more useful than re-auditing the original statistics. Of the 30 papers it accepted, 22 were purely conceptual and only 8 included conditions closely matching the original.^[3]

In consumer research the dominant form is self-replication within the original paper. Because the same researchers run every study, this cannot detect effects tied to a particular lab or team, and a 2024 review suggests adding an independent study-design replication to original papers.^[5]

Published marketing examples sit at different points on the continuum. A B2B study ran a close replication of a relationship-value framework, reusing the instrument in a comparable manufacturing context because the framework had never been replicated. A French product-placement study reused the original film clips but changed the load manipulation and the country, and only partly reproduced the result. An IKEA-effect replication used a new product and added a mediator.^[7][8][9]

Choosing a type for your question

• To check whether a result can be reproduced at all, especially one that has never been replicated, start with a close or study-design replication.^[4][5][7]
• To learn how far a theory generalizes, run a theory-test replication, ideally varying one relevant factor at a time so that a failure can be traced to a cause.^[5]
• If the original design has known confounds, change what is needed to remove them and say so. A replication of a music-in-advertising effect rebuilt the stimuli and setting for exactly this reason.^[10][2]
• To get both kinds of evidence, include some conditions that closely match the original alongside the extension, or run one replication with the original measure and another with an improved one.^[3][2]
• Whatever the type, record how each element compares with the original so readers can place the study on the continuum; the companion guide on designing a replication covers the details.^[2]

The evidence behind this

• Research methods pitfalls: ratio variables and censored data
• Marketing knowledge: how it is made and when to trust it
• Empirical generalizations in marketing: what the research shows

References

1. Christian S. Crandall, Jeffrey W. Sherman (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology.
2. Mark J. Brandt, Hans IJzerman, Ap Dijksterhuis, Frank J. Farach, Jason Geller, Roger Giner‐Sorolla, James A. Grange, Marco Perugini, Jeffrey R. Spies, Anna van 't Veer (2013). The Replication Recipe: What makes for a convincing replication?. Journal of Experimental Social Psychology.
3. John Lynch, Eric T. Bradlow, Joel Huber, Donald R. Lehmann (2015). Reflections on the replication corner: In praise of conceptual replications. International Journal of Research in Marketing.
4. R. Murray Lindsay, Andrew S. C. Ehrenberg (1993). The Design of Replicated Studies. The American Statistician.
5. Oleg Urminsky, Berkeley J Dietvorst (2024). Taking the Full Measure: Integrating Replication into Research Practice to Assess Generalizability. Journal of Consumer Research.
6. Brian D. Earp, David Trafimow (2015). Replication, falsification, and the crisis of confidence in social psychology. Frontiers in Psychology.
7. Nektarios Tzempelikos (2020). Relationship value in business-to-business markets: a replication and extension of Ulaga and Eggert’s (2006) study. Journal of Business and Industrial Marketing.
8. Felipe Pantoja, Patrícia Rossi, Adilson Borges (2015). How Product-Plot Integration and Cognitive Load Affect Brand Attitude: A Replication. Journal of Advertising.
9. Marko Sarstedt, Doreen Neubert, Kati Barth (2016). The IKEA Effect: A Conceptual Replication. Journal of Marketing Behavior.
10. Ivar Vermeulen, Camiel J. Beukeboom (2015). Effects of Music in Advertising: Three Experiments Replicating Single-Exposure Musical Conditioning of Consumer Choice (Gorn 1982) in an Individual Setting. Journal of Advertising.

Bibliographic details only. Follow the DOI to read each study at its publisher.