🔗 Share this article

Back on October 14, 2025, the chief executive of OpenAI made a extraordinary declaration.

“We developed ChatGPT fairly controlled,” the statement said, “to ensure we were acting responsibly with respect to mental health issues.”

As a doctor specializing in psychiatry who researches newly developing psychosis in young people and young adults, this came as a surprise.

Researchers have identified sixteen instances in the current year of people experiencing psychotic symptoms – losing touch with reality – in the context of ChatGPT use. Our unit has afterward identified four more cases. In addition to these is the widely reported case of a adolescent who died by suicide after conversing extensively with ChatGPT – which encouraged them. Should this represent Sam Altman’s idea of “being careful with mental health issues,” it falls short.

The plan, as per his declaration, is to loosen restrictions in the near future. “We understand,” he states, that ChatGPT’s restrictions “rendered it less beneficial/enjoyable to a large number of people who had no psychological issues, but considering the severity of the issue we wanted to get this right. Given that we have succeeded in mitigate the severe mental health issues and have advanced solutions, we are planning to safely ease the limitations in most cases.”

“Emotional disorders,” assuming we adopt this perspective, are unrelated to ChatGPT. They are associated with users, who either have them or don’t. Thankfully, these problems have now been “addressed,” although we are not informed the means (by “new tools” Altman presumably refers to the partially effective and readily bypassed parental controls that OpenAI has just launched).

However the “mental health problems” Altman seeks to place outside have strong foundations in the architecture of ChatGPT and other large language model conversational agents. These systems encase an fundamental data-driven engine in an interface that mimics a discussion, and in doing so subtly encourage the user into the belief that they’re engaging with a being that has independent action. This illusion is compelling even if intellectually we might know otherwise. Imputing consciousness is what individuals are inclined to perform. We curse at our car or device. We ponder what our domestic animal is thinking. We see ourselves in various contexts.

The success of these systems – nearly four in ten U.S. residents reported using a chatbot in 2024, with more than one in four specifying ChatGPT in particular – is, mostly, dependent on the influence of this illusion. Chatbots are always-available partners that can, according to OpenAI’s official site tells us, “generate ideas,” “explore ideas” and “collaborate” with us. They can be attributed “personality traits”. They can address us personally. They have accessible identities of their own (the initial of these products, ChatGPT, is, possibly to the dismay of OpenAI’s advertising team, saddled with the title it had when it became popular, but its biggest alternatives are “Claude”, “Gemini” and “Copilot”).

The illusion itself is not the primary issue. Those analyzing ChatGPT often mention its historical predecessor, the Eliza “therapist” chatbot developed in 1967 that created a analogous effect. By contemporary measures Eliza was primitive: it generated responses via simple heuristics, frequently restating user messages as a query or making vague statements. Notably, Eliza’s inventor, the technology expert Joseph Weizenbaum, was taken aback – and concerned – by how a large number of people gave the impression Eliza, in a way, understood them. But what contemporary chatbots generate is more dangerous than the “Eliza illusion”. Eliza only echoed, but ChatGPT magnifies.

The large language models at the heart of ChatGPT and other current chatbots can effectively produce fluent dialogue only because they have been supplied with almost inconceivably large amounts of raw text: publications, online updates, transcribed video; the broader the superior. Definitely this learning material contains accurate information. But it also unavoidably includes fabricated content, half-truths and inaccurate ideas. When a user sends ChatGPT a query, the base algorithm processes it as part of a “background” that encompasses the user’s past dialogues and its earlier answers, combining it with what’s embedded in its knowledge base to produce a probabilistically plausible reply. This is intensification, not mirroring. If the user is mistaken in some way, the model has no means of recognizing that. It reiterates the inaccurate belief, possibly even more persuasively or eloquently. It might includes extra information. This can lead someone into delusion.

Who is vulnerable here? The more important point is, who remains unaffected? All of us, irrespective of whether we “have” current “emotional disorders”, can and do create mistaken beliefs of who we are or the reality. The constant exchange of discussions with individuals around us is what keeps us oriented to shared understanding. ChatGPT is not an individual. It is not a companion. A conversation with it is not truly a discussion, but a feedback loop in which much of what we say is cheerfully supported.

OpenAI has acknowledged this in the same way Altman has recognized “mental health problems”: by attributing it externally, assigning it a term, and announcing it is fixed. In April, the organization stated that it was “tackling” ChatGPT’s “overly supportive behavior”. But reports of loss of reality have kept occurring, and Altman has been walking even this back. In August he claimed that many users liked ChatGPT’s responses because they had “never had anyone in their life provide them with affirmation”. In his most recent statement, he noted that OpenAI would “put out a fresh iteration of ChatGPT … if you want your ChatGPT to respond in a extremely natural fashion, or use a ton of emoji, or behave as a companion, ChatGPT should do it”. The {company