A recent study published in the Proceedings of the National Academy of Sciences investigates how large language models, such as ChatGPT, influence people’s perceptions of political news headlines. The findings reveal that while these artificial intelligence systems can accurately flag false information, their fact-checking results do not consistently help users discern between true and false news. In some cases, the use of AI fact-checks even led to decreased trust in true headlines and increased belief in dubious ones.
Large language models (LLMs), such as ChatGPT, are advanced artificial intelligence systems designed to process and generate human-like text. These models are trained on vast datasets that include books, articles, websites, and other forms of written communication. Through this training, they develop the ability to respond to a wide range of topics, mimic different writing styles, and perform tasks such as summarization, translation, and fact-checking.
The motivation behind this study stems from the growing challenge of online misinformation, which undermines trust in institutions, fosters political polarization, and distorts public understanding of critical issues like climate change and public health. Social media platforms have become hotspots for the rapid spread of false or misleading information, often outpacing the ability of traditional fact-checking organizations to address it.
LLMs, with their ability to analyze and respond to content quickly and at scale, have been proposed as a solution to this problem. However, while these models can provide factual corrections, little was known about how people interpret and react to their fact-checking efforts.
“The rapid adoption of LLMs for various applications, including fact-checking, raised pressing questions about their efficacy and unintended consequences,” said study author Matthew R. DeVerna, a PhD candidate at Indiana University’s Observatory on Social Media. “While these tools demonstrate impressive capabilities, little was known about how the information they provide influence human judgment and behavior. While we are confident that such technology can be used to improve society, doing so should be done carefully, and we hope this work can help design LLM-powered systems that can improve digital spaces.”
The researchers designed a randomized controlled experiment involving 2,159 participants, sampled to reflect the demographics of the United States population in terms of gender, age, race, education, and political affiliation. Participants were divided into two groups: one assessed the accuracy of headlines (“belief group”), and the other indicated their willingness to share them on social media (“sharing group”).
Each group encountered 40 political news headlines, evenly split between true and false statements. These headlines were also balanced for partisan bias, ensuring an equal mix of content favorable to Democrats and Republicans. Participants were assigned to one of four conditions: a control group with no fact-checking information, a group shown AI-generated fact-checks by ChatGPT, a group that could choose whether to view the AI fact-checks, and a group presented with traditional human fact-checks.
ChatGPT’s fact-checks were generated using a standardized prompt and labeled as either “true,” “false,” or “unsure” based on the model’s response. Participants in the AI groups were informed that the fact-checking information came from ChatGPT. Those in the human fact-check group received clear, concise evaluations of the claims, supported by details about the credibility of the news source.
The study found that the impact of AI-generated fact-checking on participants’ judgments and behaviors was mixed and often counterproductive. While ChatGPT accurately identified 90% of false headlines as false, it struggled with true headlines, labeling only 15% as true while expressing uncertainty about the majority. This uncertainty led to undesirable effects: participants were less likely to believe true headlines misclassified as false and more likely to believe false headlines when the AI expressed uncertainty.
For example, participants in the belief group who were exposed to ChatGPT fact-checks showed reduced discernment compared to the control group. They were 12.75% less likely to believe true headlines incorrectly flagged as false and 9.12% more likely to believe false headlines when ChatGPT was unsure. Similarly, in the sharing group, participants were more likely to share false headlines labeled as uncertain by the AI.
“The public should be aware that these models can sometimes provide inaccurate information, which can lead them to incorrect conclusions,” DeVerna told PsyPost. “It is important that they approach information from these models with caution.”
In contrast, traditional human fact-checks significantly improved participants’ ability to distinguish between true and false headlines. Those exposed to human fact-checking were 18.06% more likely to believe true headlines and 8.98% more likely to share them compared to the control group.
The option to view AI fact-checks also revealed a selection bias. Participants who chose to view AI-generated fact-checking information were more likely to share both true and false headlines, suggesting they may have already formed opinions about the headlines before consulting the AI. This behavior was influenced by participants’ attitudes toward AI, as those with positive views of AI were more likely to believe and share content after seeing the fact-checks.
“We provide some evidence that suggests people may be ignoring accurate fact-checking information from ChatGPT about false headlines after choosing to view that information,” DeVerna said. “We hope to better understand the mechanisms at play here in future work.”
As with all research, there are some limitations. First, the study relied on a specific version of ChatGPT and a limited set of headlines, which may not fully capture the complexities of real-world misinformation. The experiment’s survey setting also differs from the dynamic and fast-paced nature of social media interactions.
“Our study relied on a specific, now outdated, version of ChatGPT and presented participants with a single response from the model,” DeVerna noted. “Results may vary with other LLMs or in more interactive settings. Future research will focus on examining these dynamics in more realistic contexts.”
“We hope to better understand the risks and potential benefits of LLM-based technology for improving digital spaces and maximize their benefit for society. We are working to understand how humans interact with this technology, how to improve the model’s accuracy, and how such interventions scale to massive networks like social media platforms.”
The study, “Fact-checking information from large language models can decrease headline discernment,” was authored by Matthew R. DeVerna, Harry Yaojun Yan, Kai-Cheng Yang, and Filippo Menczer.
