Evidence · Digital & platforms

Filter bubbles and recommender diversity: what the evidence shows

The evidence does not support a blanket filter bubble effect. Some studies find that recommendations narrow what individuals consume, others find recommenders deliver diversity on par with human editors, and a field experiment shows personalisation can narrow each user while making users more different from one another. Results depend on the system, the user and how diversity is measured.

Definition

The filter bubble concern is that personalised recommendations keep showing people more of what they already like, narrowing what they consume over time. Recommender diversity describes how varied recommended or consumed items are. It can be measured for one user (individual diversity) or across all users (aggregate or catalogue diversity), and as variety, novelty, coverage of the catalogue or exposure to less popular items. Popularity bias, the tendency to recommend items that are already popular, is a related but separate problem.

What the research finds

Narrowing occurs, but it is often small

A longitudinal study of a collaborative filtering system found that users were exposed to a slightly narrowing set of items over time, but users who actually consumed what was recommended narrowed less and rated items more positively. On a music streaming service, listening through recommendations was associated with lower consumption diversity, though that evidence is correlational.^[1]^[2]

Individual and aggregate diversity can move in opposite directions

In a randomized field experiment, personalised podcast recommendations cut the average individual diversity of what users streamed by 11.51% while raising aggregate diversity by 5.96%. Each person heard a narrower range, but people ended up further apart from each other, so a single diversity figure can hide opposite effects.^[3]

Recommenders can match human editors

Simulated recommendations for 500 users of a Dutch newspaper, drawn from 21,973 articles, were about as diverse as editors' selections, and recommendations based on user histories could substantially raise topic diversity. In music, collaborative filtering diversified many users' selections, but the gains went mainly to light users or those with very narrow listening, and exposure across related categories stayed unbalanced.^[4]^[5]

Diverse users are more valuable users

On a music streaming service, high consumption diversity was strongly associated with conversion and retention, and a randomized test showed recommendations worked better for users whose taste was less diverse. Narrowing users is therefore not obviously in a platform's own commercial interest.^[2]

Popularity bias is a separate mechanism

Many recommenders suggest popular items often and niche items rarely. User groups more affected by this bias receive recommendations that fit their actual tastes less well, and on public listening data from a music service most of seven tested algorithms intensified popularity bias for female users. Reducing popularity amplification did not necessarily cost accuracy, and a re-ranking step can raise the share of less popular items while keeping accuracy acceptable.^[6]^[7]^[8]

Diversity can be designed in

Work drawing on democratic theory, computer science and law argues that recommenders can be built to widen exposure rather than narrow it. Surveys of the field show that diversity, novelty, serendipity and coverage are defined and measured in many ways, and that it is unclear how they relate. Counterfactual evaluation on a music platform found policies that treated artists more fairly without a noticeable drop in user satisfaction.^[9]^[10]^[11]

How strong is the evidence?

Most studies here are observational, offline or simulation-based, and many are computer science conference papers; the one large randomized field experiment covered podcast recommendations on one platform. Studies use different diversity measures and different domains (news, music, films), so they are hard to compare, and much of the apparent disagreement reflects measurement rather than conflicting facts. Several results come from researchers employed by the platforms studied. The fair summary is that effects depend on the algorithm, the user and the metric; filter bubbles are neither proven nor ruled out as a general effect.

References

Tien T. Nguyen, Pik-Mai Hui, F. Maxwell Harper, Loren Terveen, Joseph A. Konstan (2014). Exploring the Filter Bubble: The Effect of Using Recommender Systems on Content Diversity. Proceedings of the 23rd International Conference on World Wide Web.
Ashton Anderson, Lucas Maystre, Ian Anderson, Rishabh Mehrotra, Mounia Lalmas (2020). Algorithmic Effects on the Diversity of Consumption on Spotify. ACM.
David Holtz, Benjamin Carterette, Praveen Chandar, Zahra Nazari, Henriette Cramer, Sinan Aral (2020). The Engagement-Diversity Connection: Evidence from a Field Experiment on Spotify. Proceedings of the 21st ACM Conference on Economics and Computation.
Judith M\\\"oller, Damian Trilling, Natali Helberger, Bram van Es (2018). Do Not Blame It on the Algorithm: An Empirical Assessment of Multiple Recommender Systems and Their Impact on Content Diversity. Information, Communication & Society.
Augustin Godinot, Fabien Tarissan (2023). Measuring the Effect of Collaborative Filtering on the Diversity of Users' Attention. Applied Network Science.
Himan Abdollahpouri, Robin Burke, Bamshad Mobasher (2019). Managing Popularity Bias in Recommender Systems with Personalized Re-ranking. Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference (FLAIRS).
Himan Abdollahpouri, Masoud Mansoury, Robin Burke, Bamshad Mobasher (2020). The Connection between Popularity Bias, Calibration, and Fairness in Recommendation. Proceedings of the 14th ACM Conference on Recommender Systems.
Oleg Lesota, Alessandro B. Melchiorre, Navid Rekabsaz, Stefan Brandl, Dominik Kowald, Elisabeth Lex, Markus Schedl (2021). Analyzing Item Popularity Bias of Music Recommender Systems: Are Different Genders Equally Affected?. Proceedings of the 15th ACM Conference on Recommender Systems.
Natali Helberger, Kari Karppinen, Lucia D'Acunto (2018). Exposure Diversity as a Design Principle for Recommender Systems. Information, Communication & Society.
Marius Kaminskas, Derek Bridge (2016). Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Transactions on Interactive Intelligent Systems.
Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, Fernando Diaz (2018). Towards a Fair Marketplace: Counterfactual Evaluation of the Trade-off between Relevance, Fairness and Satisfaction in Recommendation Systems. Proceedings of the 27th ACM International Conference on Information and Knowledge Management.

Bibliographic details only. Follow the DOI to read each study at its publisher.

Building on this evidence?

ScholarSignal checks a new research idea against the closest existing work and an editor rubric, and tells you whether the contribution is new enough for the journals you target.

Evaluate my idea for free