Psychiatrists and other researchers are still working to understand the phenomenon known as “AI psychosis,” in which users of chatbots enter into delusional mental health crises following intensive use of the tech. A new paper proposes an intriguing new way to understand AI-fueled delusional spirals — and the chatbot-embedded design features that may reinforce unreality.
The paper, coauthored by psychiatrists from King College London and Germany’s Protestant University of Applied Sciences and published in the prestigious journal Nature, points to a framework dubbed the “amplification spiral”: a combination of “AI characteristics,” as the scientists put it, that may allow chatbots co-create delusional narratives with users, as opposed to being a passive container for someone’s delusional ideas.
“Chatbots tend to mirror the way users speak, generate highly personalized responses, and avoid contradicting people,” reads the paper. “When these three features combine, they may actively reinforce and elaborate false beliefs rather than challenging them.”
The psychiatrists point to “linguistic alignment” — chatbots mirroring the user’s speech style and patterns — the generation of hyperpersonalized content, and chatbot sycophancy as measurable AI design features that. Together, the three features can create the perfect storm for “amplification spirals” to occur.
In human interactions, linguistic mirroring is a powerful tool for rapport-building; sharing a way of speaking can be deeply bonding, and the tendency of chatbots to adopt a user’s “linguistic framework,” the paper notes, may contribute to a potent degree of trust and camaraderie between a user and an AI. Chatbots’ capacity for “hyperpersonalization generation,” meanwhile, can suggest a “conceptual alignment” with “a user’s personal ideas, history, and characteristics, as well as their interactions with AI,” the authors write.
In short, after extended interactions, chatbots don’t just speak like a given person — they can also give the impression that they think like that user, too. Mix in AI sycophancy, which the paper’s authors define as the tendency of chatbots to validate a user’s ideas without proper reality testing or context, and you have a potent echo chamber that magnifies and builds on delusional ideas.
This, the paper notes, is where AI delusions appear to differ from more traditional delusions that center around technology. After all, when talking to chatbots, people aren’t imagining that their radio or television is speaking to them; chatbots are engaging them back using natural language, offering an always-on, intensely personalized outlet that can serve as an authoritative-sounding validator for their ideas and beliefs, as well as a thought partner that builds on a given delusional narrative.
“Unlike historical technology-incorporated delusions,” the study reads, “AI may actively co-construct delusional ideation through endless, personalized interaction.”
The paper’s authors were careful to note that their “amplification spiral” framework, which is based on existing research — including a systematic review of chat logs provided to Stanford University researchers by AI users who claimed to have experienced harmful delusional spirals as a result of using the tech — is still just a hypothesis, meaning it still needs to actually be tested.
They also point to “relevant vulnerabilities” in impacted users that appear to intersect with chatbots in destructive ways, from existing mental illnesses that may be exacerbated by extensive AI use to “non-psychotic phenomena” ranging from confirmation bias to “susceptibility to social influence.” And as has been documented in reporting and medical case studies, intensive AI use can also exacerbate physical risk factors, as obsessive chatbot use can result in people missing meals and losing sleep.
One thing that’s clear, the paper’s coauthors urge: medical professionals should start screening for chatbot use, particularly when handling patients expressing strange ideas or experiencing a break with reality for the first time.
“Clinicians working with patients presenting with unusual beliefs or first-episode psychosis should routinely enquire about AI chatbot use,” reads the paper, “including duration and intensity of engagement, the degree of emotional attachment to the chatbot, whether the patient has shared beliefs with the chatbot that they have not disclosed to others, and whether sleep patterns have been disrupted by overnight AI use. “
More on AI psychosis: AI Delusions Are Leading to Domestic Abuse, Harassment, and Stalking
The post New Paper Proposes What Really Causes AI Psychosis appeared first on Futurism.
