Are AI companies incentivized to put the public’s health and well-being first? “Although relational AI has potential therapeutic benefits, recent studies and emerging cases suggest potential risks of emotional dependency, reinforced delusions, addictive behaviors, and encouragement of self-harm,” reads the paper. “I think [AI companies] don’t want to create a product that’s going to put people at risk of harming themselves or harming their loved ones or derailing their lives. “The potential harms of relational AI cannot be overlooked — nor can the willingness of technology companies to satisfy user demand,” the physicians’ paper concludes. “If we fail to act, we risk letting market forces, rather than public health, define how relational AI influences mental health and well-being at scale.”More on AI and mental health: Users Were So Addicted to GPT-4o That They Immediately Cajoled OpenAI Into Bringing It Back After It Got Killed