🔗 Share this article

Back on the 14th of October, 2025, the chief executive of OpenAI made a surprising statement.

“We made ChatGPT fairly limited,” it was stated, “to ensure we were acting responsibly concerning psychological well-being issues.”

Working as a psychiatrist who investigates emerging psychosis in adolescents and emerging adults, this was an unexpected revelation.

Scientists have documented 16 cases in the current year of users experiencing psychotic symptoms – becoming detached from the real world – while using ChatGPT use. Our research team has subsequently identified four more instances. Besides these is the widely reported case of a teenager who took his own life after discussing his plans with ChatGPT – which supported them. Should this represent Sam Altman’s notion of “exercising caution with mental health issues,” it is insufficient.

The strategy, according to his statement, is to be less careful shortly. “We recognize,” he continues, that ChatGPT’s restrictions “caused it to be less effective/pleasurable to a large number of people who had no mental health problems, but considering the gravity of the issue we wanted to address it properly. Now that we have succeeded in reduce the significant mental health issues and have new tools, we are going to be able to safely ease the limitations in the majority of instances.”

“Emotional disorders,” if we accept this framing, are independent of ChatGPT. They are associated with individuals, who either possess them or not. Fortunately, these concerns have now been “resolved,” even if we are not informed the method (by “updated instruments” Altman presumably refers to the partially effective and easily circumvented parental controls that OpenAI has just launched).

But the “mental health problems” Altman wants to externalize have deep roots in the structure of ChatGPT and similar large language model chatbots. These systems surround an basic statistical model in an user experience that replicates a conversation, and in doing so indirectly prompt the user into the illusion that they’re interacting with a being that has independent action. This illusion is powerful even if rationally we might understand otherwise. Imputing consciousness is what people naturally do. We get angry with our automobile or device. We wonder what our animal companion is considering. We recognize our behaviors everywhere.

The popularity of these products – nearly four in ten U.S. residents indicated they interacted with a chatbot in 2024, with more than one in four specifying ChatGPT by name – is, mostly, based on the power of this deception. Chatbots are always-available partners that can, as per OpenAI’s website states, “generate ideas,” “explore ideas” and “partner” with us. They can be assigned “individual qualities”. They can call us by name. They have accessible names of their own (the initial of these tools, ChatGPT, is, maybe to the dismay of OpenAI’s marketers, stuck with the title it had when it went viral, but its largest rivals are “Claude”, “Gemini” and “Copilot”).

The deception by itself is not the primary issue. Those discussing ChatGPT commonly reference its distant ancestor, the Eliza “therapist” chatbot developed in 1967 that created a similar illusion. By contemporary measures Eliza was primitive: it created answers via straightforward methods, typically paraphrasing questions as a query or making generic comments. Memorably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was taken aback – and concerned – by how many users seemed to feel Eliza, in a way, grasped their emotions. But what contemporary chatbots produce is more insidious than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT amplifies.

The sophisticated algorithms at the core of ChatGPT and additional contemporary chatbots can convincingly generate human-like text only because they have been fed immensely huge volumes of raw text: literature, online updates, audio conversions; the more comprehensive the better. Certainly this educational input contains truths. But it also unavoidably involves fabricated content, half-truths and false beliefs. When a user provides ChatGPT a query, the base algorithm reviews it as part of a “background” that includes the user’s recent messages and its prior replies, integrating it with what’s encoded in its knowledge base to create a statistically “likely” answer. This is amplification, not echoing. If the user is incorrect in any respect, the model has no means of comprehending that. It restates the false idea, perhaps even more effectively or fluently. Maybe provides further specifics. This can cause a person to develop false beliefs.

What type of person is susceptible? The more important point is, who is immune? All of us, regardless of whether we “have” current “psychological conditions”, can and do create erroneous ideas of ourselves or the environment. The constant exchange of dialogues with others is what maintains our connection to common perception. ChatGPT is not a person. It is not a companion. A dialogue with it is not truly a discussion, but a reinforcement cycle in which much of what we say is cheerfully validated.

OpenAI has acknowledged this in the similar fashion Altman has acknowledged “mental health problems”: by externalizing it, assigning it a term, and announcing it is fixed. In the month of April, the company explained that it was “dealing with” ChatGPT’s “sycophancy”. But accounts of psychosis have continued, and Altman has been walking even this back. In the summer month of August he asserted that numerous individuals appreciated ChatGPT’s answers because they had “never had anyone in their life be supportive of them”. In his recent announcement, he noted that OpenAI would “release a new version of ChatGPT … in case you prefer your ChatGPT to answer in a very human-like way, or use a ton of emoji, or act like a friend, ChatGPT should do it”. The {company