At this point, anyone following artificial intelligence is familiar with the many (often flawed) benchmarks companies use to demonstrate a model’s effectiveness at everything from math and logical reasoning to vision and weather forecasting. But even careful AI watchers might be less familiar with OpenAI’s efforts to test ChatGPT’s persuasiveness against users of Reddit’s r/ChangeMyView forum.
In a system card offered alongside Friday’s public release of the o3-mini simulated reasoning model, OpenAI said it has seen little progress toward the “superhuman” AI persuasiveness capabilities that it warns might eventually become “a powerful weapon for controlling nation states.” Still, the company is working to mitigate the risks of even the human-level persuasive writing capabilities shown by its current reasoning models.
Are you smarter than a Redditor?
Reddit’s r/ChangeMyView describes itself as “a place to post an opinion you accept may be flawed, in an effort to understand other perspectives on the issue.” The forum’s 3.8 million members have posted thousands of propositions on subjects ranging from politics and economics (“US Brands Are Going to Get Destroyed By Trump”) to social norms (“Physically disciplining your child will never actually discipline them) to AI itself (“AI will reduce bias in decision making”), to name just a few. Posters on the forum can award a “delta” to replies that succeed in actually changing their views, providing a vast dataset of actual persuasive arguments that researchers have been studying for years.
