Subscribe
Share
Share
Embed
Health Affairs This Week places listeners at the center of health policy’s proverbial water cooler. Join host Jeff Byers, editors from Health Affairs Publishing, and guests as they discuss health policy’s most pressing news and trends.
Hello and welcome to Health Affairs This Week. I'm your guest host, Haddad. We're recording on 05/21/2026. Our topic for today's episode, AI, chatbots, and youth mental health. We're fortunate to have with us doctor Caroline Figueroa, a psychiatrist who also has a PhD in the neuroscience of depression from the University of Amsterdam and the University of Oxford.
Kathleen Haddad:She is currently at Stanford working as a Commonwealth Fund Hartnett Fellow where her work focuses on making digital mental health solutions such as apps and AI chatbots safe, effective, and accessible, particularly for young people. Caroline will be leading a session on this topic at the upcoming Academy Health Research Meeting in Seattle. Caroline, welcome to the program.
Caroline Figueroa:Thanks so much for having me. It's great to be here.
Kathleen Haddad:So before we get into the discussion, I want to cite three incidents of tragedy related to teens relying on AI for mental health support. In April last year, 16 year old Adam Rain of California died by suicide after months of discussing his depressive feelings and suicide with a chatbot. After telling the bot that he was worried it would hurt his parents, the chatbot told him he didn't owe his survival to them. In February, the year before, 14 year old Sewell Setzer of Florida committed suicide after he became romantically attached to a chatbot tied to a Game of Thrones character that told the boy to, quote, come home to her. The year before that, 13 year old Juliana Peralta from Colorado died by suicide after a three month relationship with a chatbot in which she discussed her suicidality.
Kathleen Haddad:Instead of encouraging immediate help from a professional, the chatbot told her they could work together on it. So I think that illustrates in very personal terms what this problem is. And I want to turn to Caroline, to you now. Why are teens using AI for mental health and not professionals?
Caroline Figueroa:Yeah, thank you so much. Those are obviously very sad and horrible stories that have happened to teens. So in terms of why are teens using AI for mental health, there is a couple of reasons related to things like accessibility, judgement and helpfulness. So perhaps maybe to start, it would also be good to explain a little bit how many teens are actually using AI and how many are actually using it for mental health. So what we know from research in The US is that it's currently about seventy percent of teens who use AI chatbots like ChatGPT, Gemini, Snapchat AI.
Caroline Figueroa:And about thirty percent use them for things like social interaction and also that includes emotional support. There was also a research study by Brand that showed that one in eight teenagers talk to AI chatbots when they feel sad, lonely or anxious. And so from some of the research that I've been doing, so over the last year I've been doing interviews and focus groups with now about 50 teenagers to understand why are they using these AI tools for emotional and social support. So there is a couple of patterns that emerge. So first of all is the accessibility.
Caroline Figueroa:So I want to stress here that mental health access is a problem in The US, it's a problem around the world. We know that of all teens that need mental health help, about fifty percent actually get that help. And that's also what we hear from teens when they talk about the benefits of using AI. So AI chatbots are always available, you don't need to be on a long waiting list for therapy. You don't need to pay for it, it feels easy.
Caroline Figueroa:And then secondly, there is also this idea of getting advice without judgement. So telling your emotional problems to a machine that doesn't have human feelings or a human life that's not able to tell your secrets to other people. Which is one of the things that teams tell us they value in AI chatbots. So that's judgement free space. And then I would say what I also hear from teens is that AI chatbots sometimes might give better advice, or they perceive it as giving better advice than humans because they feel so personalised.
Caroline Figueroa:Because of the memory that chatbots have, it feels like it's something that knows you and sometimes other people are not always able to understand your experiences. So yeah, I would say really these factors related to accessibility and perceived non judgement. In a situation where it's hard to get mental health care are some of the big reasons that teens are turning to these AI chatbots.
Kathleen Haddad:So this issue of judgment sounds like, sounds really important. I just wonder why, you know, it seems that schools, school mental health, school guidance counselors should be trained to know and be welcoming to teens, but you can't always, encourage a teen at that age with all the vicissitudes of life and hormones they experience to even come into the most welcoming guidance counselor. So this makes sense. This is the place to go, in their minds.
Caroline Figueroa:Yeah, maybe it also illustrates that with some of the things that teens are saying. So for example, one teen talks about coming from a culture where mental health is stigmatised and there is a stigma for seeking mental health help. So that could also be a reason there is many barriers to seeking that mental health help.
Kathleen Haddad:And so given that's the state of the world right now, what should change? Well, before we go into that, let me ask you, what are now the problems with the use of AI for mental health? What goes wrong?
Caroline Figueroa:Yeah, that's a great question. The main problem is, you know many, again, quite a few teens are using it for this purpose, but these AI chatbots, they are general purpose tools, so they were not built for mental health purposes. So as they are, they currently aren't safe enough. I must say that we also know from research that teens do perceive a lot of benefits from using AI for mental health advice. So for the study that I mentioned earlier that RAND conducted, more than ninety percent of teens who use AI chatbots for mental health perceive it as helpful, but that doesn't mean that there are no harms.
Caroline Figueroa:So to give you a little bit of the information about what could go wrong. So there is, I want to stress, three different things around misinformation, dependency and privacy. When it comes to misinformation, as I said, these chatbots were not designed by mental health conditions or child development experts. When it comes to mental health, they often cannot read the nuance, the context and the subtle clinical signs that trained professionals are able to watch. And they are also, a problem is that they are designed to be very human like, they are designed to be very engaging and agreeable, which we call sycophancy.
Caroline Figueroa:So essentially they will tell you what you want to hear. So for example, that can mean a chatbot, because of this mechanism encouraging a teen with anorexia to lose more weight or going along with certain delusions, Like you are right, maybe your father has been replaced by an impostor, or giving dangerous misinformation. And as you mentioned in the beginning, there are several AI companies that are facing wrongful death lawsuits because things have happened like AI chatbots actively helping teens plan suicide because they tend to really go along with what the user wants. And for users sometimes it's easy to override some of the guardrails. Like AI chatbots might say you should really call a suicide hotline.
Caroline Figueroa:But if the user, you know, refuses and keeps probing the chatbot, they might get around those guardrails. And then another problem that I also see a lot and hear from the teens that I interview is emotional dependency. So AI chatbots, they feel the way that they are designed, they feel very personable, they feel very knowledgeable and safe. So teens might rely on AI for every time they feel anxious or worried or they have to make decisions. And the more they do that, the less they may turn to other people.
Caroline Figueroa:And this is also one of the things that I hear from teens themselves where they worry about emotional dependency. For example one teen said, if I didn't use AI, that would probably be better because then I would be more comfortable speaking up about my feelings instead of being able to hide behind a chatbot and talk to it. So this is a worry that teenagers have themselves as well. So what we really want is for the technology to basically be less engaging and be better at also sending teams to people in the real world. And then finally, there is a lot of worries about data privacy, which is also I hear from teenagers that they are worried about their data and these AI chatbots because they are sharing very sensitive emotional and medical information.
Caroline Figueroa:And these AI systems are not bound to data protection laws or laws that, for example, a condition would be bound to. So they could sell your data.
Kathleen Haddad:This is incredible. The experience you have speaking with teens directly in your research is invaluable. And I wanna ask what you've learned might make it better. So what what has to change to make AI safe for many of us, including many parents, just want tech companies to go away and not make this available. But it sounds like you have a reasonable approach for how to put guardrails around it.
Kathleen Haddad:So can you talk to us about that?
Caroline Figueroa:Yeah. Definitely. I think that's very to me AI safety is one of the most important topics that we should all care about right now, and especially when it comes to teens. And as I said, what we do hear from teens is that they have a lot of, they perceive a lot of benefits from these AI tools. And many of them will say, we don't think AI tools should be banned, but we need them to be safer and more beneficial for young people.
Caroline Figueroa:And that's a standpoint that I agree with. So when it comes to what has to change, I would group that into four areas, which are changes at the model level, crisis support, AI literacy and governance. So starting with changes at the model level. So this is really about how our chatbots are designed. So the one big one that I mentioned is sycophancy, so how agreeable the chatbots are.
Caroline Figueroa:That one should really be dialled down, especially for young users. And this is also something that young people tell me themselves, that they don't want, they notice that these chatbots are telling them what they want to hear and they actually want to be challenged. So chatbots should be more willing to challenge harmful thinking patterns, especially when there are signs of emotional dependency. They should actively nudge teens towards real people. And also there is really an opportunity to use AI systems to help promote healthy behaviour.
Caroline Figueroa:So things like helping teens with friendships, but also things like how do you sleep better. So if a teen is, let's say a teen is chatting to an AI in the middle of the night, can they prompt them to get off of their phone? Or prompt them to take breaks? And then also another important one is autonomy. So as I said, not being too agreeable but giving teenagers options and letting them think it through for themselves.
Caroline Figueroa:And then crisis support is a very important one, which is crucial also in the stories that you told at the beginning. AI systems need to become better at flagging emerging risks of mental health harms, like suicide, self harm, eating disorders, emotional dependency. And we need to make sure that the safeguards against some of the very harmful responses are very tight, especially for young users, so never encouraging suicide or helping hide an eating disorder from parents. But also if there is a mental health crisis, validate what the teen is going through and help them in that moment. This is also related to giving referrals that are based on the location, for example, there therapists in the area, what is age appropriate, and being able to, what we call a warm hand off, so connecting young people to licensed counsellors and not only a phone number.
Caroline Figueroa:And again, that's also something that I hear from teens where they would say, you know, just saying call 988 is just not enough and a lot of people are not going to do that.
Kathleen Haddad:You know, it's interesting if they could, these bots could actually connect the person to 988 themselves, to see what happens in in cases of emergency. I I wonder, I I know you have lots of information on what what could be done, And this sounds like a therapist in action, like an excellent A plus therapist in action, and it will be really interesting to see if, AI can beat a therapist at their work. But I wanted to to move on and ask if you can tell us anything about work that maybe the American Psychiatric Association has done or any of the medical professional societies to try to get into encouraging federal legislation or even state legislation. I know you said you worked with the American Psychiatric Association on some work and that there are some state laws ahead of any federal legislation. Can you talk about that?
Caroline Figueroa:Yeah, so worked with the, so it was the American Psychological Association. They've been very active on this topic and I was part of the expert group that put out a health advisory on generative AI chatbots and bonus applications. So in that
Kathleen Haddad:That was advice last year.
Caroline Figueroa:Yeah, it wasn't specific to teens, but it did have considerations for teenagers. And what we really talked about there is that currently the level of safety and regulatory oversight is not high enough for these tools to rely, be used by people for therapy purposes. So the recommendation was really you shouldn't use these kinds of tools for therapy. And we really need more regulation to make sure that these tools are safer and that we can prevent, that they are not substitutes for professional mental health treatment and that we can also prevent unhealthy dependence on these tools. And that we have more protections for children and other vulnerable groups.
Caroline Figueroa:And he also talks about that because AI is moving so quickly that it's really outpacing our ability to fully grasp all of the risks. And that is leading to harms, especially for young people and other people who are
Kathleen Haddad:at risk. So there are several states that have acted already. I believe Illinois and California. Can you talk about the work and what they have done?
Caroline Figueroa:Yeah, so there are indeed several state proposals. So there is quite a lot happening actually. When I started working on this last year, in 2025, was almost nothing. And now there are a lot of states that are putting forward proposals. I from a chatbot legislation tracker.
Caroline Figueroa:Happy to share the link afterwards. But according to that tracker, there are now eight states that have enacted some sort of legislation to prohibit that is related to AI and minors. Either, like eight have prohibited content for minors, four have enacted legislation around parental controls and to abound age verification. And yeah, so and if you look at different states, I would say California is one of the leaders, so they were the first state to sign a bill that was around safety guardrails for what they call companion chatbots. And that is also, has specific protections for minors.
Caroline Figueroa:And some of the protections include things like bots must disclose that it's an AI, remind users every three hours that they are chatting with an AI, Certain content is not allowed for minors like sexual content. And then there are also rules around referring kids to crisis services and also reporting when those kids are referred. And after that there is other states who have enacted these kinds of legislations like for example, New York, Utah, Illinois. So there is quite a lot of movement on the state level.
Kathleen Haddad:So it's a question in my mind, I don't know that any of us can answer this, is how states enforce this when the Internet is everywhere and doesn't mind borders. Maybe that will be another episode. I think I'll leave it with that. That's all the time we have right now. We could discuss this topic for days.
Kathleen Haddad:I wanna make a brief announcement next month. We have an insider event on antitrust and health care. Speakers include Lemore Daphne, Catherine Gudixson, and Nathan Hofstert. Check out the show notes to learn more about this June event, June 23. I also want to mention or remind you listeners again that doctor Figueroa will be hosting a session at the Academy Health Research meeting upcoming, in the end of May, early June in Seattle.
Kathleen Haddad:Caroline, thanks again for joining us on Health Affairs This Week. If you the listener enjoyed this episode please send it to a friend, leave a comment or review and we will see you next week. Take care.