If it bleeds, it leads
Online news consumption & the surprising story behind our new analysis of 22,743 experiments; social identity & conspiracy theories; our work in the New York Times; & an app to explore data on COVID
It’s December 2020. PhD student Claire Robertson is at her computer, reading over the final paragraph of her manuscript one last time. She’s in her second year of grad school, and is about to submit her first first-author paper, a registered report analyzing negativity bias in online news consumption, to Nature Human Behaviour.
She clicks submit.
About six months later, she finally hears back. It’s not a response she was expecting from the editors. In fact, she has to read the email twice to make sure her eyes aren’t playing tricks on her. Another research group has submitted a similar paper at the same time, analyzing the same dataset, with more or less the same results.
But instead of pitting the two teams of researchers against each other as competing papers (likely resulting in one team getting scooped), the editors saw an opportunity to raise the confidence in these converging results. The editors proposed an interdisciplinary collaboration between our two teams—our team coming from a background in social & cognitive psychology, the other team from the field of computer science.
Both teams eagerly joined forces.
Nearly two years later, and armed with a new set of methods, had our joint paper conditionally accepted for publication at Nature Human Behaviour. This was a Stage 1 Registered Report investigating what drives our consumption of online news using data from the viral online news sensation Upworthy.com. Together, our interdisciplinary team (Claire and Nicolas Pröllochs from the University of Giessen sharing first authorship; our visiting scholar Philip Pärnamets and Kaoru Schwarzenegger from ETH Zurich; and Jay and Stefan Feuerriegel from ETH Zurich sharing last authorship) sought to understand this question:
How does the presence of 1) negative and positive words and 2) discrete emotional words affect the click rate for news headlines?
To test yourself, think about these two headlines below: which would you be more likely to click on?
Our preliminary results suggest that you may have chosen headline #2—for a variety of reasons, such as the fact that humans preferentially attend to negative stimuli across many domains, and that negative information may be more “sticky” to us (see our full paper for more details).
Testing this in the real world, however, is often tricky. Most studies of online behavior are correlational, and while laboratory studies can test the causal impact of language and headlines, they remove subjects from the natural online environment.
The unique structure of our dataset, the Upworthy Research Archive, provided an opportunity for us to investigate the causal impact of language and headlines on article consumption. For every article published, Upworthy experimentally tested multiple iterations of headlines (see examples below), creating randomized controlled trials for each news story. These experiments allowed us to analyze the direct effects of negative and positive words, as well as discrete emotional words, on people’s consumption of the story, measured as the click-through rate.
Our initial findings analyzed 4,873 randomized controlled experiments. The results supported our hypotheses: people were more likely to click on the stories with negative headlines. Critically, the stories themselves were identical so this was purely an effect of the headlines.
Since our paper is a registered report, this first stage of analyses is considered a pilot, and we were only allowed to sample a preliminary dataset of a total of trials (i.e. each news headline and its variations) out of the total dataset. We have now replicated the same pattern of results with the full sample of 22,743 experiments! This includes ∼105,000 different headline variations that received more than 530 million impressions and ∼8 million clicks.
Our analyses revealed that negative words had a positive effect on click-through rate, while positive words had a negative effect on click-through rate. This suggests that a larger proportion of negative words increases the tendency of online users to access a news story (and vice versa for positive words). This interacted with the topics of the news headlines—for example, headlines relating to “Government and Economy,” “LGBT,” “Parenting and School,” and “People” received more clicks when they contained a large share of negative words.
Next, we tested the effect of the presence of four discrete emotions (anger, fear, joy, sadness) on click-through rate, finding that an increase in sadness increased the odds of a user clicking the headline, but finding no other statistically significant results for the other emotions. We found this result to be particularly interesting, as it goes somewhat against other results that find discrete emotions to be important determinants for various forms of user interaction.
To read the rest of our preliminary results, you can check out our full manuscript here. The next stage of our report will include these results on 22,743 randomized controlled trials. Importantly, we have now found these same results three different times, with three different methods (the current pilot analysis, our initial submission, and the other team’s initial submission all leveraged a slightly different approach).
Given that 89% of Americans consume at least some of their news online (a number that is increasing), it is crucial to understand what is driving our interaction with these articles.
To hear about the process from one of the first authors herself, Claire Robertson:
…what could have been a horrible experience for me (getting scooped and having nothing to show for all my hard work) ended up being an incredibly positive experience and opportunity for growth. It was intimidating to be the only woman and the only junior scientist on the project, but it really made me rise to the challenge, and I feel exceptionally proud of both the manuscript and everything I've learned, technically and theoretically. Obviously this is unique (the dataset itself being used by both teams) but I could see a world where people team up rather than race to get out overlapping projects. I think it would overwhelmingly be a good thing for science.
New pre-print, paper, and interactive dataset
We have another new first-author piece from PhD student Claire Robertson (and with visiting scholar Clara Pretus, incoming postdoc Steve Rathje, PhD student Elizabeth Harris, and Jay)! In this pre-print, we make the case that motives to believe conspiracy theories are often related to social identity, as conspiracy theories are well-positioned to fulfill social identity needs such as belongingness goals, the need to think highly of one’s in-group, and the need to feel secure in one’s group status. You can read the pre-print here.
Former postdoc and now NYU professor Madalina Vlasceanu has a new paper with Alin Coman out in Applied Cognitive Psychology! The study examined the effect that different instructions regarding accuracy had on belief change during discussions about COVID-19. While there appeared to be no effect of this condition, they found that individuals were sensitive to their conversational partners and changed their beliefs according to their partners' conveyed beliefs. You can read the full paper here!
A paper with the full dataset (51,404 participants from 69 countries) from our international study of the social & morality psychology of COVID-19 is now available! And better yet, we created a tool for you to visually interact with the data. You can check out the paper here and the tool here:
In the news
Our research on the impact of social media on polarization was covered in the New York Times. The article featured research from lab members Jay, incoming postdoc Steve Rathje, PhD students Elizabeth Harris and Claire Robertson, and former PhD students Anni Sternisko and Billy Brady:
Public outreach and talks
Jay gave a plenary address at the Canadian Psychology Associate in Calgary, focusing on themes from his book, The Power of Us, for which he also did a special book signing after the talk.
PhD student Elizabeth Harris gave a talk at APS 2022 in Chicago, IL, showing a strong pattern of partisan bias in belief updating across several experiments. She plans to defend her dissertation later this month.
Former PhD student and visiting scholar Diego Reinero gave a talk about our political backfire project at the Center for the Science of Moral Understanding Conference in Durham, NC.
Our incoming postdoc Steve Rathje also gave a talk at the CSMU conference, presenting his work examining 1) how social media following, sharing, and favoriting behavior is correlated with vaccine hesitancy, and 2) how unfollowing/following different accounts on Twitter may reduce affective polarization and improve well-being.
And finally, Jay gave the final keynote of the The Aarhus University ’22 Conference on
Online Hostility and Bystanders, titled “Morality in the Anthropocene”
Lab member and alumni shoutouts
A huge congratulations to our most recent PhD, Anni Sternisko, who won the 2022 NYU Stuart Cook Award (and who is also about to get married)! The award is given annually to an advanced student who has made impressive contributions to research in Social Psychology while at NYU. You can check out the list of previous winners here (which should also include previous PhD student Diego Reinero from 2021). Go Anni!
And now, for a few exciting career updates:
Our former postdoc Peter Miende-Siedlecki recently received tenure at the University of Delaware!
Our former postdoc, Andrea Pereira, started a new position as Head of Data and Research at Child Helpline International!
And our former PhD student Julian Wills recently accepted a Quant UX Research role on Google's Search team!
This month in photos
We took a new lab photo this month that celebrates our hybrid, international lab environment! Welcome to our newest RAs Alejandro Calvo, Anees Goparaju, Christina Capozzoli (not pictured), Gabrielle Sylvester (not pictured), Kinda Shammas, Nalinda Wanikpun, and Yishu Lu.
The social psychology program had a joint picnic in Central Park with our former department neighbors, the Freeman lab (now at Columbia University)! The attendees from our lab snapped a quick pic together.
And finally, we had a send-off get-together for our former PhD student Billy Brady who is leaving NYC for Chicago to begin his new position as an Assistant Professor at Northwestern University Kellogg School of Business!
In case you missed last month’s newsletter…
As always, if you have any photos, news, or research you’d like to have included in this newsletter, please reach out to Katie (nyu.vanbavel.lab@gmail.com) who writes our monthly newsletter. We encourage former lab members and collaborators to share exciting career updates or job opportunities—we’d love to hear what you’re up to and help sustain a flourishing lab community. Please also drop comments below about anything you like about the newsletter or would like us to add.
That’s all, folks—thanks for reading and we’ll see you next month!