ScholarSignal
Menu
Sign inEvaluate an idea

Evidence · Research methods

Marketing knowledge: how it is made and when to trust it

A marketing finding deserves trust when it replicates across many data sets and when pooled estimates correct for publication bias; single studies often do not hold up. When 100 psychology studies were repeated, only 36% produced significant results again, and bias-corrected meta-analysis cuts the average advertising elasticity by more than five times. Newer threats include causal wording for correlational studies and survey data produced by AI rather than by people.

Definition

Marketing knowledge is the body of findings that researchers and practitioners can rely on about how markets, buyers and marketing actions behave. How much to trust a finding depends on how it was produced: whether it has been replicated under varied conditions, whether pooled estimates account for selective publication, whether the design supports the causal claim being made, and whether the data really come from the people they are meant to represent.

What the research finds

Laws come from many data sets, not one study

The empirical generalization tradition holds that durable knowledge comes from patterns that repeat across circumstances and that managers need quantified effect sizes, not only directions of effect. A related empirics-first approach argues that starting from systematic observation, rather than from a hypothesis, produces more relevant and durable marketing knowledge.[1][2][3]

Replication filters out fragile findings

When 100 studies from three psychology journals were repeated with high-powered designs, replication effects were about half the size of the originals, and 36% of replications were significant against 97% of the originals. Flexible choices in data collection and analysis make false positives easy to produce. In marketing, extensive replication has been used to give patterns such as double jeopardy a sound theoretical footing.[4][5][6]

Meta-analysis helps, if it corrects for bias

Meta-analyses can answer questions no single study can, such as how many ad exposures maximise response (attitude peaked at about ten exposures in experiments) or how well transaction cost theory predicts make-or-buy decisions. But results depend on correcting for publication selection: a meta-analysis of 538 advertising elasticities found corrected averages more than five times smaller than earlier estimates (0.0008 short term, 0.03 long term), in line with randomized trials.[7][8][9]

Correlational studies are often written up as causal

Among 194,631 cross-sectional social-science articles from 1980 to 2024, 46% used causal language in the title or abstract, and the yearly share rose from 20% to 60% after 2000. In an experiment, readers often took such abstracts as causal evidence; methodological labels and associational wording reduced this. Summaries written by large language models could strengthen the overstatement, though asking them to be cautious helped.[10]

Citations reflect social ties as well as merit

In a study of all academic economists based in the United Kingdom, personal proximity and similarity in professional and political attributes predicted who cited whom, alongside scientific factors. Citation counts are therefore an imperfect measure of a finding's quality.[11]

AI-generated data threatens where evidence comes from

An autonomous AI agent passed 99.8% of 6,000 attention checks designed to catch careless people and bots, and about one-third of online survey participants report using AI help. A review of studies comparing samples simulated by large language models with human samples found that results vary widely by domain, so such samples are best limited to pretests and pilots until validated.[12][13]

How strong is the evidence?

The best-known replication figures come from psychology, not marketing, and the replication rate of marketing studies has been measured less directly. The corrected advertising elasticities come from one recent meta-analysis, although they agree with randomized trials. The citation study covers economists in one country. The warning about AI agents in online surveys comes from a commentary whose peer-review status is not recorded, and the study of causal language used large language models to classify articles. Taken together, these sources describe risks to marketing knowledge more precisely than they measure how widespread each risk is within marketing.

References

  1. Frank M. Bass (1995). Empirical Generalizations and Marketing Science: A Personal View. Marketing Science.
  2. Dominique M. Hanssens (2018). The value of empirical generalizations in marketing. Journal of the Academy of Marketing Science.
  3. Peter N. Golder, Marnik G. Dekimpe, Jake T. An, Harald J. van Heerde, Darren S. U. Kim, Joseph W. Alba (2022). Learning from Data: An Empirics-First Approach to Relevant Knowledge Generation. Journal of Marketing.
  4. Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science.
  5. Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn (2011). False-Positive Psychology. Psychological Science.
  6. Cullen Habel, Larry Lockshin (2013). Realizing the value of extensive replication: A theoretically robust portrayal of double jeopardy. Journal of Business Research.
  7. Susanne Schmidt, Martin Eisend (2015). Advertising Repetition: A Meta-Analysis on Effective Frequency in Advertising. Journal of Advertising.
  8. Inge Geyskens, Jan-Benedict E. M. Steenkamp, Nirmalya Kumar (2006). Make, Buy, or Ally: A Transaction Cost Theory Meta-Analysis. Academy of Management Journal.
  9. Joseph Korkames, T. D. Stanley, Stefan Stremersch (2025). Meta-analysis of advertising effectiveness: New insights from improved bias corrections. International Journal of Research in Marketing.
  10. Calvin Isch, Timothy G. Dorr, Neil Fasching, Grace Jennings, Duncan J. Watts (2026). Quantifying the prevalence and impact of overreaching causal claims in social science. Nature Human Behaviour.
  11. Carlo D’Ippoliti, Lucio Gobbi, Christian Alexander Mongeau Ospina, Giulia Zacchia (2023). Social determinants of citations: An empirical analysis of UK economists. Kyklos.
  12. Zhicheng Lin (2025). Synthetic respondents and the illusion of human data.
  13. Marko Sarstedt, Susanne Adler, Lea Rau, Bernd H. Schmitt (2024). Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psychology and Marketing.

Bibliographic details only. Follow the DOI to read each study at its publisher.

Related topics

A synthesis of 13 studies, written by ScholarSignal. Last reviewed 2026-09-26.

Building on this evidence?

ScholarSignal checks a new research idea against the closest existing work and an editor rubric, and tells you whether the contribution is new enough for the journals you target.

Evaluate my idea for free