‘Chronically online’ Gen Z and millennials are most vulnerable to fake news - study

ChatGPT and other artificial intelligence systems have made it possible to turn famous people, including journalists, into avatars and have them present misinformation that viewers can easily believe are true. The danger they pose is huge. 

Now, University of Cambridge psychologists have developed the first validated “misinformation susceptibility test” – a quick, two-minute quiz that gives a solid indication of how vulnerable a person is to being duped by the kind of fabricated news that floods online spaces. 

The test, proven to work through a two-year series of experiments involving over 8,000 participants, has been deployed by polling organization YouGov to determine how susceptible Americans are to fake headlines. 

The first survey to use the new 20-point test called ‘MIST’ by researchers and developed using an early version of ChatGPT has found that – on average – adult US citizens correctly classified two-thirds (65%) of headlines they were shown as either real or fake. However, the polling found that younger adults are less able than older adults at identifying false headlines and that the more time they spent online recreationally, the less likely they were to be able to tell real news from misinformation. 

The results run counter to common opinion

This runs counter to prevailing public attitudes regarding online misinformation spread that older, less digitally-savvy “boomers” are more likely to be taken in by fake news. 

The British study presenting the validated MIST has just been published in the journal Behavior Research Methods under the title “The Misinformation Susceptibility Test (MIST): A psychometrically validated measure of news veracity discernment.” 

The researchers want the public to test themselves at https://yourmist.streamlit.app/. Selecting true or false against 20 headlines gives the user a set of scores and a “resilience” ranking that compares them to the wider US population. It takes under two minutes to complete.  

“Misinformation is one of the biggest challenges facing democracies in the digital age,” said Prof. Sander van der Linden, a senior author of the MIST study and head of the Cambridge Social Decision-Making Lab. “We are seeing how online falsehoods create polarized belief systems in major nations, and the consequences, such as the attempted Capitol Hill insurrection.” 

“To understand where and how best to fight misinformation, we need a unified way of measuring susceptibility to fake news. That is what our test provides,” continued van der Linden, who wrote the new book Foolproof. The Cambridge team developed assessment tools that made it possible for them to work out the right level and mix of fake and genuine headlines to produce the most reliable results. Examples of real news came from outlets such as the Pew Research Center and Reuters. To create false but confusingly credible headlines – similar to misinformation encountered “in the wild” – in an unbiased way, the researchers used ChatGPT version 2. 

“When we needed a set of convincing but false headlines, we turned to GPT technology. The AI generated thousands of fake headlines in a matter of seconds. As researchers dedicated to fighting misinformation, it was eye-opening and alarming,” said Dr. Rakoen Maertens, MIST’s lead author.

However, another recent study by the same team used GPT to produce useful questions for a variety of psychological surveys. “We encourage our fellow psychologists to embrace AI and help steer the technology in beneficial directions,” said MIST co-author Dr. Friedrich Götz.

For the MIST, an international committee of misinformation experts whittled down the true and false headline selections. Variations of the survey were then tested extensively in experiments involving thousands of UK and US participants.

The latest YouGov survey saw 1,516 adult US citizens take the MIST in April 2023, and also respond to questions covering demographics, politics and online behavior.  

When it came to age, only 11% of the 18 to 29-year-olds got a high score (over 16 headlines correct), while 36% got a low score (10 headlines or under correct). By contrast, 36% of those 65 or older got a high score, while just 9% of older adults got a low score. 

The longer someone spent online for fun each day, the greater their susceptibility to misinformation. Some 30% of those spending up to two hours online daily got a high score, compared to just 15% of those spending nine or more hours online. The survey also analyzed channels through which respondents receive their news. The “legacy media” came out top. For example, over 50% of those who got their news from the Associated Press, or NPR (National Public Radio) or newer outlets such as Axios achieved high scores.  

Social media had the news audiences most susceptible to misinformation. Some 53% of those who got news from Snapchat received low scores, with just 4% getting high scores. Truth Social was a close second, followed by WhatsApp, TikTok and Instagram. 

Democrats performed better than Republicans on the MIST, with 33% of Democrats achieving high scores, compared to just 14% of Republicans. However, almost a quarter of both parties’ followers were in the low-scoring bracket. Perhaps alarmingly, half of all Americans now say they see what they believe to be misinformation online every day.

Maertens added that “younger people increasingly turn to social media to find out about the world, but these channels are awash with misinformation. Approaches to media literacy, as well as algorithms and platform design, require an urgent rethink.”