We Tested Sonia the Way We’d Want Any Wellness Tool Tested

We handed Sonia to outside academic researchers and asked them to test it properly: randomized, pre-registered, IRB-approved, independently analyzed. Here is what they found — and what the study cannot tell us.

At Sonia, we hear a version of the same question a lot: does this actually work?

It’s an excellent question, and it’s one that many companies wait years to answer. We didn’t want to put out a product we hadn’t tested properly. So earlier this year we did something a lot of wellness apps don’t bother doing: we handed our own product to a team of outside academic researchers — at the University of Cologne and the London School of Economics — and asked them to test it using the most rigorous of methods. Randomized. Pre-registered before a single person enrolled. IRB-approved. Analyzed independently. Written up as a preprint with the methods and numbers out in the open for anyone to check. We didn’t get to see the results until they did.

What we tested

Four hundred adults across the US joined the 4-week study. None were currently in therapy. Each person was randomly assigned — essentially a coin flip — to one of two groups. Half got access to Sonia. The other half received a set of well-known, publicly available self-help materials from the World Health Organization covering many of the same ideas Sonia teaches people to use when self-managing their emotions: grounding techniques, healthy daily habits, and guidance on when to reach out for professional support.

Two weeks later, everyone reported back on how they were feeling.

What we found

People who used Sonia reported feeling meaningfully better than people in the comparison group, with less worry, less low mood, and more of a sense of being in control — and the difference showed up in just two weeks. People also used Sonia the way we’d hoped: in short, regular check-ins that fit around a normal day, practicing self-care techniques drawn from cognitive behavioral therapy (CBT) and acceptance and commitment therapy (ACT). These are techniques a person practices themselves, not something delivered to them like a treatment.

None of this means Sonia is a cure, a diagnosis, or a stand-in for a therapist. It means that, for a lot of people, a short daily conversation grounded in well-established self-care techniques made a real difference in how they felt. To me, that’s worth taking seriously — especially for people who’d otherwise go without any support at all.

What we built the study to catch

Testing an AI companion properly means testing what happens when things go wrong, not just when they go right. So the study measured safety in real time, for every single conversation.

Over the two weeks, a small number of conversations crossed a threshold suggesting someone might need more support than Sonia is built to give. In each of those cases, the system recognized it, ended the conversation, and pointed the person toward real resources, including 988. Our research team reviewed every one of those moments afterward, and every one was judged to have been handled the way it should have been.

We think that finding matters just as much as the wellbeing numbers — possibly more. A tool that helps people but can’t recognize its own limits isn’t a tool we’d want to build, let alone use ourselves.

What this study doesn’t tell us

We’d rather tell you what we don’t know than let one number do more work than it should.

Two weeks is a short window, so we can’t yet say whether these improvements last, although additional analyses on our data did show the findings held up at four weeks as well. The people in this study found us through an ad and chose to try an AI tool for emotional support, which likely makes them more comfortable with AI than the average person. And this study wasn’t designed to tell us how Sonia performs for someone with a formally diagnosed condition under a clinician’s care — that’s a very different question, for a different kind of study we haven’t yet run.

We’re currently running longer follow-up work to understand whether these effects hold up over time, and we’ll share what we learn, good or inconclusive. We have to help the field figure out real answers to these questions in order to finally start moving the needle on outcomes and truly help people who want support.

Why we did this the hard way

We could have run this study in-house, on our own terms, with our own team analyzing the results. It would have been faster and cheaper. We didn’t, because we wanted an answer we could trust — and one you could trust, too.

Sonia is not a replacement for professional care. But it can be a first step: a way to build the skills and self-awareness that help you know when you need more support and feel ready to seek it. Sonia is not a crisis service. If you are in crisis or thinking about hurting yourself or someone else, please contact emergency services or call or text 988.

You can read the full preprint at https://osf.io/preprints/psyarxiv/9m8aw_v1. We’re grateful to our academic collaborators for holding us to a real standard, and to the participants who gave us four weeks of their lives so we could learn something real.

Source: Roever, N., Wolf, L., Forman-Hoffman, V., Areán, P., & Feld, B. “Short-term Evaluation of a Voice-Based Generative AI Emotional Support Companion: A Randomized Controlled Trial.” Preprint, not yet peer-reviewed.