Subscribe
Share
Share
Embed
Come with us as we talk to the clinicians making AI part of their daily workflows, and using it to build deeper connections with patients.
Welcome back to Trust But Verify, the Evidently podcast. In episode four, we continue the conversation that we started with Mel Molina, a UCSF emergency medicine physician in episode three. In this part two, we dig into health equity, the digital divide, the do no harm paradox, and generally just have the second half of a great conversation with Mel. Hope you enjoy. When you think about health equity and technology, what's the biggest misconception you encounter?
Mel Molina, MD MAS:Luckily, the place that I work at, health equity is definitely kind of at the forefront of everybody's mind. I would say that, health equity can't be an afterthought in terms of like building whatever tool you're building, like it has to be at the forefront and you have to be able to measure whatever the tool is, whatever its impact is on different populations and make sure that it checks out. A lot of people also talk about whether to exclude demographic information to make things a little bit less biased, like exclude race and ethnicity, without perhaps realizing that race and ethnicity is built into all of these other different variables, you know, like language zip code, insurance status. So it's not you're just not gonna like, you know
Kai Romero, MD:Delete that.
Mel Molina, MD MAS:Yeah. I just like, oh, like, it's not biased cause I didn't include race and ethnicity. It's like, it's built into everything. Yeah. Structural racism.
Mel Molina, MD MAS:So, I mean, are I I would say those are the two I I wouldn't say, like, misconceptions, but for me personally, things that I've learned, just, you know, from from some of the initiatives that have been taking place at my own institution.
Kai Romero, MD:In my previous role as the CMO of By the Bay Health, like, at start of the pandemic, we were rolling out a big telehealth platform. And, you know, elder folks who are seriously ill that live in, like, rural Sonoma County turns out don't have great broadband. Yeah. Can't hear what's going on. No.
Kai Romero, MD:Know, often don't have any type of technological awareness, have never downloaded an app. And so starting from this place, like how do you address folks who have any number of additional disabilities around hearing or communicating whatever that may be, neurological issues, can't use their feet well, etcetera. And I think it was this very humbling experience of like, oh, the thing that everyone is saying is the solution for healthcare is actually not going to work for us at
Mel Molina, MD MAS:all. No.
Kai Romero, MD:And I think about that often when it comes to actually the wide array of technical skill that clinicians have. There's the people who are kind of digital natives, tend to be younger, tend to use lots of different types of technology in their daily life. And then there are the folks like my dad, 75, still working as a primary care doctor in a federally qualified health clinic. He was an early adopter in tech in general, so he was able to make the transition to EHR. But there are lots of his peers that were not, and you think they were just, like, entirely kind of removed from their career path because they couldn't make that adaptation.
Kai Romero, MD:And I think we often make assumptions around a certain level of tech capability as a clinician, as a precondition for participation in things that could, like, really make your life better. Totally. But if you can't figure out how to use them, if it's not intuitive to you and you can't get from step a to step b, you're just like out the game. And that feels pretty unfair.
Mel Molina, MD MAS:No. That's the digital divide. That's the digital divide. You know, it's that some of these innovative technologies are going to reach people differently and others won't it won't reach them, the people without broadband or even just MyChart, MyChart being available in different languages or, know Great. You know, people being illiterate.
Mel Molina, MD MAS:Like, how if people are illiterate, how are they supposed to navigate MyChart?
Kai Romero, MD:Yeah. You know?
Mel Molina, MD MAS:So yeah. No. It's it's definitely a a whole thing.
Kai Romero, MD:Yeah. And I think the challenge is it only exacerbates who gets access and who doesn't. Right? So if all of a sudden all scheduling is online, half of the people are delighted because they're like, great. I didn't have to wait on hold for thirty minutes to get an appointment.
Kai Romero, MD:Mhmm. And the other half are just entirely cut out of that process. Mhmm. Most places have obviously workarounds for that, but I think I think about that a lot.
Mel Molina, MD MAS:Yeah. It's a perfect example of, like, if you don't adopt whatever's coming, like, you're gonna be left behind. You know? Like, I I mean, like, I use open evidence every day. It's just helpful to me, you know, and I just feel like if I don't use it or if I don't use ChatGPT or, you know, we have UCSF has a secure version of ChatGPT and Claude.
Mel Molina, MD MAS:And, you know, I use it a lot to help with my writing, and it's great. Yeah. And it's just helped me immensely. If I if I I feel like I have an advantage in using that, but I can imagine like folks who are not still trying to like, you know, write papers the traditional way for me is just really challenging. Whereas here, it's like, I can definitely edit.
Mel Molina, MD MAS:I can do edits, and I can give it an outline, and then it can produce this very eloquently worded paper that exactly capitulates what I was trying to say in a better way than I could have ever have said it. So
Kai Romero, MD:Well, and you know enough about the topic to be a good editor. I think that's the other piece of it.
Mel Molina, MD MAS:That's It's
Kai Romero, MD:like, if you don't have enough of a framework, then you're like, oh, this seems okay. I don't know. But if you Yeah. Do know enough about the topic, then you can kind of be a discerning editor and
Mel Molina, MD MAS:make That's it right.
Kai Romero, MD:I'm curious about the other side of that, which we actually hear a lot about from clinicians, which is like, okay, I really appreciate all of this assistance. And what is atrophying? I think a lot of clinicians are like, man, if I have so much help from AI, is there potentially a part of me, a part of my brain that will become weaker, that will become essentially so reliant on a tool that you plot me into the middle of a clinic in, you know, rural, I don't know, Iowa, with no, you know, access to these tools, I and I would be up a creek.
Mel Molina, MD MAS:You know, I think that that's that's a concern that constantly comes up with the deployment of new technologies. Like, I remember with the calculator, people said that computers people say that I certainly would have concerns about it for someone who's just learning. Right. Like, you know, I think one example that comes to mind is just like, Jen Alpha, who's like growing up with social media and like all of these, you know, computers and iPads and things. What does that do to their brain development in terms of like how to socialize with a human like in person?
Mel Molina, MD MAS:Or be bored? Yeah. Like yeah. Yeah. The attention, you know, like it conditions them to have like a a very small attention span, TikTok, and all these things.
Mel Molina, MD MAS:So I think I'm more concerned about that for the trainees that are coming up. In fact, our residency leadership has not even though residents in our health system technically do have access to AI scribes, our residency leadership is talking about how to mindfully allow that because they don't want the interns and the twos I think to have access to that because they're worried that it might
Kai Romero, MD:They won't learn how to write a note.
Mel Molina, MD MAS:Right. Not just how to write a note but like it does your MDM and it does like your differential and all these things. And so like could it stymie learning in that sense in terms of how to think critically through a differential?
Kai Romero, MD:And how is that different from having a dot phrase with your MDM?
Mel Molina, MD MAS:Yeah. Exactly. Exactly. So that's why I'm kinda like Maybe. You know, I don't know.
Mel Molina, MD MAS:I I don't know. You know, I use a calculator for adding up numbers when I'm filing my taxes and things like that. But, like, am I really missing out on anything by not, you know, doing it in my head?
Kai Romero, MD:Yeah. You
Mel Molina, MD MAS:know? I don't know.
Kai Romero, MD:Like Maybe. Don't think so.
Mel Molina, MD MAS:Maybe. I don't know.
Kai Romero, MD:There's this, there was this thing somebody said. Essentially, like, gyms didn't exist until people start stopped doing hard labor as their jobs. You're stacking cement bricks. You're not like, oh, I need to go bench press. Their point was like, what's that gonna be for the mind once we have begun outsourcing more and more tasks?
Kai Romero, MD:Is it just everyone's gonna have to play spelling bee? Wordle.
Mel Molina, MD MAS:It's gonna
Kai Romero, MD:be your Wordle reps.
Mel Molina, MD MAS:Yeah.
Kai Romero, MD:Like, what will it be that continues to engage your mind? The challenge is, as you said, if you ask Claude or whatever to write something, you look through it, you know you know what it's supposed to be and you know what you want it to be. And then you're kind of tailoring it based on this initial. Whereas the folks who haven't yet learned what their writing voice is, how to kind of come up with a thesis, how to do all these things. I actually had this really wild experience of someone I was interviewing someone for a role, and they were typing my questions into an LLM and reading me the answers.
Kai Romero, MD:I could see them Yeah. Reading me
Andres Krogh-Walker:the happened answers. To me five or 10 times.
Mel Molina, MD MAS:Were they what? Were they a young young person?
Kai Romero, MD:Yeah. And I think I was just like, oh gosh. I felt so much sympathy for this person who was trying to put their very, very best foot forward and did not believe they could do that alone. And Mhmm. I thought, how do we how do we avoid creating this writ large?
Kai Romero, MD:Like, some some of this is gonna be around who the person is and their own internal sense of self confidence. But I've heard the same thing from friends who are professors that kids are just typing the answers into LLMs. And there's and and, like, what's missing is, like, obviously, their own thinking, but it's also like being vulnerable in a public space with an answer that might be wrong. Yeah. Totally.
Kai Romero, MD:That's the thing you're missing. You know? And and so I I wonder a lot about integration of these technologies in a way that, like, allows you to experience the discomfort of having a novel and incorrect idea in the world with your colleagues. Like, that's
Mel Molina, MD MAS:a Yeah.
Kai Romero, MD:That's like, I think an essential part of being a person, and I wouldn't want folks to miss that.
Andres Krogh-Walker:I mean, this has always kind of been the story of computers. Right? It's like I stopped knowing how long division works, but I did for like a a brief moment in my early teenage years understand how long division works and balancing a checkbook. But it seems like it's this shift from collecting knowledge being the goal to synthesizing and actually having an interest in the knowledge being the new goal. The way I've accidentally stumbled into freezing the system for those interviewees that are reading off of a ChatGPT response is just to ask them what their favorite thing is.
Andres Krogh-Walker:Interviewing an engineer, it's like, what's what's your favorite programming language? Just tell me about something you like and that you're fascinated by. And that's full stop always been the question that
Kai Romero, MD:You know, like, snap them out of the
Andres Krogh-Walker:Yeah. At least move them past the demonstration of hoarding knowledge into like the actual interest and intrigue of what they're learning about.
Mel Molina, MD MAS:It's just, yeah, it's just really interesting because it really how you communicate and how you perceive the world impacted by the events that were occurring during your formative years. Know and it's like, you know, like not just like technological things but like you know, 09/11 and like the Iraq war, you know, and like all of these other things. And so I think that part of it is just gonna like be, you know, this new generation that's coming up with all of these tools and, like, how are they how are they gonna be impacted? I don't know. We we we don't know yet.
Mel Molina, MD MAS:You know?
Kai Romero, MD:Well and on some level, you kind of like, I kinda think you just pick your poison generationally. You know, like, how did people self sabotage when I was in my early twenties? Well, like, you would drink too much and like, you might stay out too late and be irresponsible, you know, sexually or do drugs or or smoke cigarettes. That feels like it's not an accurate description of a lot of No. Going on with Gen Z at all.
Kai Romero, MD:And so I'm like, you know what? Like, big picture, they're not doing any of that. So that's pretty good. On the Oh other hand
Mel Molina, MD MAS:my gosh. Yeah.
Kai Romero, MD:On the other hand, like, you know, I think the what's what's kind of true about all of those kind of maladaptive behaviors is that for the most part, you are around other people
Mel Molina, MD MAS:Mhmm.
Kai Romero, MD:Screwing up. And the thing I worry about a lot is folks who've, from such a young age, gotten used to isolation or had it foisted upon them at a really critical time. Yes. How do you do you make sure for those folks that technology is additive, useful, and like not thought of as an exclusive replacement for the discomfort of human interaction?
Mel Molina, MD MAS:Totally. Totally. Yeah. I mean, and I think the kids that lived through the pandemic definitely are gonna have to figure that one out for sure.
Kai Romero, MD:Yeah. I think, I think everyone struggled. I think all kids struggled, but I think especially the older kids struggled. I remember hearing once that, Maria Montessori said that essentially I'm gonna badly paraphrase this, but, like, essentially, once a kid turns, like, 12 or 13, just like send them out into a forest, they're not gonna learn anything academically. They just need to figure out social stuff.
Kai Romero, MD:And the thought of having been fully pulled from that social environment, which is so crucial at that age. And, you know, it took us a long time societally to figure out, should there be parameters on screen time and smartphones? It took us about twenty years to figure out, like, yes,
Mel Molina, MD MAS:there's Yeah.
Kai Romero, MD:Hope we are faster and more thoughtful around the implementation and use of AI and what those guardrails should be. And I'm curious about so you mentioned being thoughtful about, you know, deploying AI solutions to medical trainees. What are some other, like, important guardrails that you think an organization and specifically a health care organization should have kind of in mind when they think about using AI at their institution?
Mel Molina, MD MAS:At our institution in particular, we've thought about it in tiers. You know, like there's like the low tier stuff, you know, like very low risk, low chance of causing patient harm. Like that's first and foremost, right? Like do no harm, right? Yeah.
Mel Molina, MD MAS:And, you know, can it, you know, like example, can it like improve administrative scheduling, you know, not of patients, but just like of shifts, you know, like the admins and departments are using it. That's like some low hanging stuff. And then you work your way up toward, more AI oriented solutions that impact patient care. And those have to be really, really rigorously piloted, before they've, before they go live. And I think our institution has an environment, a high performance computing environment.
Mel Molina, MD MAS:They basically have almost like a simulated environment with patient data that you can run your model or whatever it is that you want to pilot in there and theoretically see kind of how it works, before you take it live. And, you know, that's obviously, after you've, you know, run it with data that, you know, that like a part of the de identified warehouse, make sure that it actually works in theory with like our population and things like that. But I think first and foremost is prioritizing patient safety. And obviously, you know, we want to comply with all of the laws and regulation, the federal laws and regulations that are, I think a lot of them are actually like, they're trying to kind of just figure it out. There's like a little bit of like vague language with institutions kind of some institutions interpreting things a little bit differently.
Mel Molina, MD MAS:For example, with machine translation. I think that we should definitely be careful and mindful about how we're deploying AI, but at the same time, I also think it's a balance between, like, being overly cautious and not actually having anything deployed that could actually be better than what the status quo is. You know, our status quo for discharge instructions for non English speaking patients is give them the discharge instructions in English and then, you know, talk through those instructions with an interpreter and expect the patient to remember everything or like take notes or like what? Okay. That's our standard of care.
Mel Molina, MD MAS:And you're telling me that if I use machine translation for something like Spanish, you know, which is like pretty damn good. Right? It's not like it's Mandarin. Yeah. You know?
Mel Molina, MD MAS:I can't use it for that. And there's maybe a possibility that one or two words maybe don't make sense, but the patient maybe gets the gist still. Like, how is that not better than what we're doing right now?
Kai Romero, MD:Right.
Mel Molina, MD MAS:Like that just drives me absolutely insane. You know, it's just like, so it just, yeah, there I think there's a there's a balance, you know? Yeah. There's a balance.
Andres Krogh-Walker:It's It's a really interesting gray zone we're in as a culture right now and you see people uploading their whole health records to chat GPT Yeah. Just to see what's in it. But it's like there are so many places where people are hungry for better insight even if it's 60% right. Just better insight on what this complicated test result was that I got back that I can't read.
Mel Molina, MD MAS:Yeah, I think that's super important. But also, you know, we get a lot of patients that come into the emergency department, they're like, yeah, ChatGPT told me this and this and this. And it's like, some of it's right. Some of it's not right. And it kind of gets back to the point that Kai was making earlier about like, well, if you're not someone that has medical knowledge and like you can't really sift through that information and know what's accurate and what isn't then, you know, how are they supposed to know?
Mel Molina, MD MAS:You know, I think it's really interesting the times in which we're living in right now.
Kai Romero, MD:Yeah. The thing that I found kind of reliably that AI can never tell you is if something is just absurd. Like, you know, like if it's just totally absurd. So someone's like, well, I probably have this like, you know, blah blah blah blah. And you're like, no, it's just ridiculous.
Kai Romero, MD:That's why it's not that. Like, because there's so many other things that are more normal and reasonable and like, do all these things align, but like potentially, but also it's ridiculous. I remember, once as a med student working with this, she was either an oculoplastics doctor or she was an oncologist that focused on eye cancer. And I was like, do you just constantly feel like you have eye cancer? Because you just see all
Mel Molina, MD MAS:the time? She was like,
Kai Romero, MD:no. Like, I know I don't have eye cancer because I see it all the time, you know?
Mel Molina, MD MAS:Yeah. Yeah. If you could
Andres Krogh-Walker:wave a magic wand and have AI solve one problem in emergency medicine for vulnerable populations, something nobody's cracked yet, what would that be?
Mel Molina, MD MAS:I would say, closed loop follow-up. That is really hard. And not just like trying to get patients, you know, follow-up with their PCP or, you know, specialty care follow-up, but like specifically with patients with unmet social needs. And, you know, we perform whatever interventions like the social worker tries to work magic and get them, you know, shelter for the night or, you know, food stamps or transportation, like, you know, understanding, like, whether that shelter actually received them, you know, or, you know, and I think this is the problem that Find Help was trying to solve. The the closed loop is the holy grail.
Mel Molina, MD MAS:You know, it's like, if we can get the community based organizations that provide these social services on the same, you know, platform that we operate on in the emergency department, then we can communicate with them and, you know, we can understand, like, how many beds do they have available without having to call all of the shelters in the city. It would just be a lot easier to link people to services if we had an understanding of real time availability and we were able to actually communicate with those folks in real time without having to call people on a regular basis. Findhelp tried to do that, and I I just don't think it's very successful. I think it's a great idea, but we've found like the people in the community based organizations, they don't log in to find help. And then and then just logging in is a whole other step.
Mel Molina, MD MAS:I don't have time to do that. I don't think our social workers have time to do that, unfortunately. So what ends up happening is like, they just kind of go by experience. Like they know the shelters, they routinely know who to call to get ahold of the person they need to get ahold of and find the information they need to find. And it's like, and it would just be great if we had this like one system that connected everything.
Mel Molina, MD MAS:Yeah. That way we in the emergency department could feel confident that we're not just sending this person, you know, back into the abyss, hoping that they find their way to that shelter that promised us that they were gonna keep their doors open for the ten minutes it took to get from the ED to the shelter. You know, it's just it's hard, you know, in the ED and not being able to control those things outside.
Kai Romero, MD:Yeah. And the the thing that always strikes me too is like, when you you don't know what you don't know, and so you send them to a shelter, but they have a complex wound that requires private bathroom to manage. And turns out they don't have access to that there. Like, all these things that it would be so good to get feedback on and to know, like, oh, actually, don't send that person there. They can only go here.
Kai Romero, MD:But you feel like in the absence of that information, you just kind of have to release them out to the world and hope for the best.
Mel Molina, MD MAS:Yeah. Yeah. So, you know, I would say, you asked me what could AI solve in emergency medicine? And and I gave you an answer that's not about emergency medicine at all. It's about the net the safety net outside of the safety net that is emergency medicine and how can we catch them and make sure that they stay plugged into the health care system or plugged into the social services that they need.
Mel Molina, MD MAS:It's like an infrastructure thing beyond the emergency department. That connection, I wish it wasn't so siloed.
Andres Krogh-Walker:Well, I think that's a great way to wrap up. Thank you so much for spending time with us.
Mel Molina, MD MAS:For sure. Thank you guys for having me.
Andres Krogh-Walker:You're listening to Trust But Verify, the Evidently podcast. You can find us online at evidently.com/podcast, and you can find us anywhere you subscribe to podcasts. See you next time.