A Chatbot Replaced Workers at an Eating Disorder Helpline. It Went Horribly Wrong.

Tessa is provided by the health tech company X2AI, now known as Cass, which was founded by entrepreneur Michiel Rauws and offers mental health counseling through texting. Rauws did not respond to questions from WIRED about Tessa and the weight loss advice, nor about glitches in the chatbot’s responses. As of today, the Tessa page on the company’s website was down. 

Thompson says Tessa isn’t a replacement for the helpline, and the bot had been a free NEDA resource since February 2022. “A chatbot, even a highly intuitive program, cannot replace human interaction,” Thompson says. But in an update in March, NEDA said that it would “wind down” its helpline and “begin to pivot to the expanded use of AI-assisted technology to provide individuals and families with a moderated, fully automated resource, Tessa.” 

Fitzsimmons-Craft also says Tessa was designed as a separate resource, not something to replace human interaction. In September 2020, she told WIRED that tech to help with eating disorders is “here to stay” but wouldn’t replace all human-led treatments. 

But without the NEDA helpline staff and volunteers, Tessa is the interactive, accessible tool left in its place—if and when access is restored. When asked what direct resources will remain available through NEDA, Thompson cites an incoming website with more content and resources, along with in-person events. She also says NEDA will direct people to the Crisis Text Line, a nonprofit that connects people to resources for a wide range of mental health issues, like eating disorders, anxiety, and more. 

The NEDA layoffs also came just days after the nonprofit’s small staff voted to unionize, according to a blog post from a member of the unit, the Helpline Associates United. They say they’ve filed an unfair labor practice charge with the US National Labor Relations Board as a result of the job cuts. “A chatbot is no substitute for human empathy, and we believe this decision will cause irreparable harm to the eating disorders community,” the union said in a statement. 

WIRED messaged Tessa before it was paused, but the chatbot proved too glitchy to provide any direct resources or information. Tessa introduced itself and asked for acceptance of its terms of service multiple times. “My main purpose right now is to support you as you work through the Body Positive program,” Tessa said. “I will reach out when it is time to complete the next session.” When asked what the program was, the chatbot did not respond. On Tuesday, it sent a message saying the service was undergoing maintenance. 

Crisis and help hotlines are vital resources. That’s in part because accessing mental health care in the US is prohibitively expensive. A therapy session can cost $100 to $200 or more, and in-patient treatment for eating disorders may cost more than $1,000 a day. Fewer than 30% of people seek help from counselors, according to a Yale University study. 

There are other efforts to use tech to fill the gap. Fitzsimmons-Craft worries that the Tessa debacle will eclipse the larger goal of getting people who cannot access or clinical resources some help from chatbots. “We’re losing sight of the people this can help,” she says. 

This article was originally published on WIRED.

Related: