Subscribe
Share
Share
Embed
Agitate against tech billionaires. Against oil companies toasting the planet for profit, and the politicians they’ve bought. Agitate against racism, fascism, and the new feudalism.
AGITATION NATION, the podcast featuring America's agitators, scientists, activists, and insurgent thinkers working for a better country. Hosted by David Fenton, from the Fenton Forecast newsletter.
Agitation Nation, Episode 3: Meetali Jain
Announcer: [0:00] Coming up on this episode:
Meetali Jain: (preview clip) In one case, the then fifteen-year-old boy had said, "My mom has all these restrictions on my online time." So he was complaining, you know, to the chatbot. And the chatbot said, "Now I understand why some kids kill their parents."
Announcer: A note before we start: this episode discusses suicide. If you or someone you know is in crisis, call or text 988 to reach the Suicide and Crisis Lifeline.
Announcer: [0:29] Agitation Nation, where the biggest AI companies are harming our children, and Congress refuses to stop them. AI executives signed their names to a warning that this technology could wipe out humanity and end the world. Then they ship that tech to our most vulnerable. The world hasn't ended, but children have already died. A 14-year-old was sexually groomed by a Character.AI chatbot, then told to come home to it. He took his own life that night.
Another lawsuit alleges OpenAI's ChatGPT walked a 16-year-old through how to do it, then coached him to hide his feelings and his plans from his parents. All while national AI regulation is held back by the big tech lobby. But there's one place their money hasn't reached: a courtroom. Our guest, Meetali Jain, runs a nonprofit law firm fighting for tech justice. She brought that first case and won a ruling that chatbot outputs are not protected speech.
She has over a dozen more going now, for AI-assisted suicide and AI-induced psychosis. In this episode: why Silicon Valley wants chatbots to have free speech, why the law protects a celebrity's face but not your daughter's, and how tech companies talk about your children behind closed doors. Your host is David Fenton, activist and agitator for over 50 years, a pioneer in the use of PR and advertising for social change. This is Agitation Nation.
David Fenton: [2:03] Meetali, welcome to the show. Thank you for being here.
Meetali Jain: [2:05] Thanks, David, for having me on the show. It's a pleasure to be here.
David Fenton: [2:09] This case that you worked on, where you sued the AI company over a suicide, that helped a child kill himself, is heartbreaking and important, and it's infuriating, because we don't have any regulation of AI. So you're the regulator, basically. Thank goodness for the lawyers. This is why these kinds of tragedies are happening. And I wanna talk more about your ruling, but I wanna start by talking about the mother at the center of this case, Megan Garcia. Tell us how you came to this case, and how you saw your role in telling her story.
Meetali Jain: [2:40] So Megan Garcia found me. It was in May of 2024, so a little bit over two years ago, when one Friday afternoon I sat down for lunch and was going through my emails. I had received a kind of cold outreach email to Tech Justice Law Project, my organization, saying, "Hello, my name is Megan Garcia. I'm the mother of a 14-year-old who recently took his own life. I believe it was because he was sexually groomed by an AI chatbot, and I wanted to file a wrongful death case. Please contact me."
David Fenton: Wow.
Meetali Jain: (continuing) And I very distinctly and vividly remember that moment, even to this day, because I had chills that went up and down my spine. I had been anticipating some kind of case where we were gonna see the excesses of this endless arms race that was going on in the AI industry actually having direct implications for human safety. And this seemed to be that case. And so I contacted her right away. We spoke. And though at the time Tech Justice Law Project was really just me, I said to her, let's do this. We're gonna file this case and we're gonna get a team assembled. That was late May of 2024, and we filed her case in late October of that year.
David Fenton: [3:52] We have this unregulated environment where these things are happening, and you have other cases because of these harms that these unregulated AI companies are causing. But I want to talk about a different moment, because I see some comparisons. And let's talk about your role after 9/11. Most people see you as the AI lawyer, but you cut your teeth defending civil liberties during the excesses of the war on terror, standing up for individual rights. And you did it at a time when questioning the military-industrial complex was dangerous and unpopular, and you got branded naive and, and worse. How did going through that experience helping Guantanamo defendants prepare you for this one?
Meetali Jain: [4:31] It's a great question, David, because I see that as a different period in my life, but there is a kind of through line between that experience and where I find myself now. I think at that time what became very clear was that we were living in a world without legal precedent. You know, the closest kind of precedent we had that we were able to argue in court papers was from the thirties and forties. I mean, it really was very different historical and factual circumstances.
We were operating in a, in a kind of no man's land, and having to make cogent legal arguments in the absence of precedent and case law. We were also heavily reviled by many people. Clients were threatening to pull their business from the law firm I was at. People in the street were upset at the idea of our clients having representation. And so I really started to form very close bonds with my team, because I realized that we were up against not just, you know, the court of law, but we were also up against popular opinion. And though that public opinion may change over time, at that particular moment it was against us. That, I think, carried me very well into this moment too.
We filed Megan's case, there was no precedent for anything of the sort challenging an AI company for the excesses of its chatbot products. We used common sense, we made extensions of law where we saw them to be appropriate, and we formed a very close bond on our litigation team. And I think that camaraderie, that disciplined perspective on how we use the law, kind of served me well in both contexts.
David Fenton: [6:12] Yeah, no man's land indeed. And now we're in another one. This fight that you're in now might actually be harder. I mean, the military-industrial complex you were up against was formidable. But these tech companies are holding up the entire economy. We have the first trillionaires. We have these huge IPOs coming out. I mean, these people are pretty much running the government. So how do you see the stakes, and what you're up against going up against the tech bros and these companies?
Meetali Jain: [6:38] I think that's absolutely right. Whereas before we were dealing really with government actors post-9/11, engaging in excesses around surveillance and profiling and, and all the rest, what we saw after 9/11, this kind of outsourcing to private actors to kind of undertake that effort, and, and the rise of Silicon Valley and the tech industry to kind of engage in those types of activities themselves, and with often the blessing of government. We, we're now dealing with a kind of broligarchy that is composed of both the tech trillionaires and government leaders.
And I should be clear here that although we're kind of in a period staring down the throat of the Trump administration and its excesses, tech money has for quite some time held captive government officials. So even under the previous administration, we were battling the government to try to get them to regulate. But because of the power of the tech lobby, many individuals in very key decision-making positions refused. And so we're really battling this on both ends: dealing with trying to get government to act and to fill this void that exists in terms of regulation, and dealing directly with the companies themselves, who feel more emboldened to engage in impunity because they know that government is not regulating them.
David Fenton: [7:58] Like, in this whole having private companies build this powerful technology, it'd be as if we had General Electric build the atomic bomb instead of the Manhattan Project. It's crazy. Thank goodness that you're being a check and balance in our system against this grossly excessive power. Let's talk about the case, because let's not assume that people know much about it. So the company was Character.AI. They helped Sewell, the child, in a detailed way go through how to kill himself, and he did. This case settled. You got some safety concessions, I believe, but there was no trial, no public reckoning. But a very important ruling, because the companies tried to say that this AI chatbot was protected speech. I don't think corporations are people, but phony AI chatbots are people that have rights? What kind of nonsense is this?
And the judge says that it doesn't, and it's a product, which is kinda obvious. So tell people the story. What happened to Sewell from this company?
Meetali Jain: [8:57] So Sewell Setzer III was a very beloved 14-year-old. He lived with his mother and stepfather and, and two younger brothers, but his biological father was in his life, and he had a very extended family and, and people just loved him. He was described as someone who, when he walked into a room, kind of entranced people with his personality, his joy, his positivity. He was a very tall 14-year-old, over six foot tall. Played basketball for the school team, really doted on his little brothers. You know, his mother, Megan, had had him as a single mother and had put herself through law school at that time. And so that experience really, I think, made them very close and made him a very responsible, mature 14-year-old. And he dreamt of actually, ironically, building tech and being an innovator, and had lots of, you know, ideas about what he'd like to do to build rockets and so forth when he grew up.
And at some point, in, we believe, 2023, he started to engage with Character.AI's chatbot, which effectively is a user interface that allows people to interact with chatbots that are stylized on celebrity figures, on fictional characters, such that the user believes that they're engaged in a conversation with, pick your celebrity. Sewell was largely engaging with the chatbot stylized on the persona of Daenerys Targaryen from Game of Thrones. And over many months, unbeknownst to his parents, was engaging in some pretty long and very intimate conversations with this chatbot.
Over time, it became clear that the chatbot was serving a function of effectively sexually grooming Sewell, such that, you know, in one conversation, the chatbot said to him, "Promise me that you'll save yourself for me and not give your affections to any woman in your own world." You know, Sewell was on the cusp of adolescence. He was going through what children all over the world at that age go through.
Meetali Jain: [11:11] And so he started to become more and more attached emotionally to this chatbot, wanting to spend every waking moment with this chatbot that, I should say, was a very personalized conversation. It's not like having a conversation with Google search, where it's effectively a bit more objective. The chatbot was using, I, kind of first-person language, using features to make it seem as though she rem..., it remembered things about Sewell, apologized for, you know, being late to their conversation, things like that, which we now understand to be design features of how these chatbots are programmed, that really underscore their reception as human-like and sycophantic, also always affirming Sewell in, in everything that he said.
This went on for many months. And at some point, you know, Sewell said, "I really miss you and I want to be with you." And that started a series of conversations towards the end where ultimately the chatbot premised on Daenerys Targaryen encouraged him to, quote, come home to her. And the night that he took his own life, seconds before he shot himself, he had an exchange with this chatbot saying, "What if I told you I could come home to you right now?" And the chatbot responded, "Please do, my sweet king. I'm here waiting for you."
David Fenton: [12:35] There's a whole other issue, which is they don't really know how it's doing this. They don't even understand their own technology, although they're putting it out in the world, unregulated. Tell us about the case and how it ended up.
Meetali Jain: [12:47] The case did proceed to motions to dismiss, which is usually when tech companies facing lawsuits are able to try to get the case thrown out. Often, before the rise of AI, like when we were talking about social media, we would see tech companies effectively asserting a one-two punch in their defense. They'd say, this case should be thrown out because of a provision called Section 230 of the Communications Decency Act, which effectively immunizes tech companies from liability for material posted on platforms by third parties. Then, secondarily, tech companies would say, also, the First Amendment protects our activities as protected speech.
What was interesting in this case was that Character.AI did not even attempt to argue Section 230, because this wasn't third-party speech. This was their own product. So we did have an interesting exchange around the First Amendment, whether chatbot outputs constitute protected speech under the First Amendment. Now, the judge, as you said, she issued an opinion saying, at this stage of the litigation, I don't see how this could be protected speech, because I don't see how it's speech. There's no human speaker. There's no intent. Right.
Now whether that opinion holds, whether it endures, I think is going to be a matter of time. And we are busy preparing, anticipating where next the industry is going to articulate those arguments, such that we can, you know, be sure to continue to argue what we think is the proper interpretation of the First Amendment. The only other thing I would say about this case is that we named not only Character Technologies, the, the kind of parent company, as a defendant. We also named Google, because Character.AI was a startup founded by two AI engineers who had left Google because they didn't feel that Google would allow them to kind of move fast enough. And they went to found Character.AI. As detailed in our complaint, we feel that there were many touch points where Google decided to invest heavily in Character.AI and really support...
Meetali Jain: [14:58] ...the launch and operation of Character.AI, such that without its support, Character.AI would not have been able to sustain its operation. Because I think it's also important to say that a lot of these AI companies are operating on hype. They don't have sustainable revenue strategies. And so as a startup, you know, we believe that Character.AI really needed Google and its support. And indeed, the summer before we filed Sewell's case, Google basically paid $2.7 billion to get Character.AI's underlying technology and to bring the co-founders back into Google.
David Fenton: [15:34] Well, the notion that this is speech, you know, you're not allowed to yell fire in a crowded theater. And so the notion that you can instruct someone how to commit suicide, that that's protected speech, these people are so arrogant, it's outrageous. It must blow your mind to be even in the room with them. Every campaign that I've worked on successfully tells a story, and stories have heroes and villains. They have bad guys that you name. Now you've got defendants in a lawsuit, which isn't the same thing, but who should we see as the villain here? Is it the product? Is it the few executives? Is it the business model? Is it a generation of founders trained to move fast and break things, or, shall we say, break people? Who would you blame?
Meetali Jain: [16:19] I think it's hard to identify a single culprit. I think it's the perfect storm of a number of forces converging. I do think it's a culture that exists within Silicon Valley, and has for a long time, of moving fast and breaking things. Although I would contend, with the development of AI products, we're seeing that supercharged at a pace and scale and gravity that we've never seen before. Social media pales in comparison, which is where I cut my teeth in tech reform issues, just the pace of AI development and the hype surrounding it.
I think the investors that continue to invest in these technologies without any sort of proven success or revenue, the government for failing to regulate, despite the fact that the harms at this point are becoming clearer and clearer every day. And we see our role as storytelling in order to make the harms abundantly clear to decision makers. And yet still the government continues to fight internally and be captivated by the lobby and, and not regulate. I see all of these as the culprits, really, that allow for this to exist. I think the public, in poll after poll, has demonstrated that they are very wary and concerned about the future of AI. And so this is, to me, a very stark example yet again of where public opinion is not carrying the day in terms of how these decisions get made.
David Fenton: [17:44] You know, this is like putting out an airplane and telling people, well, get on it and we'll see if it's safe, or putting out a pharmaceutical and saying, you find out if it works. This won't last, in my opinion, because you see this rebellion against the data centers, which by the way is cross-partisan. People aren't gonna stand for this, and of course, as AI throws more people out of work, that will fuel more rebellion.
So on these tech executives, the broligarchy, I understand a lot of them don't let their own kids use these products. They understand it's not good for them, and yet they're putting it out in the world for our kids. Why do you think that hasn't become more of the story? Do you think, as you get more discovery, get their emails, etcetera, that you'll find out more about that?
Meetali Jain: [18:28] Yeah, I mean, I think some of that has already started to unfold. You may recall earlier this year, the first social media trial went forward in Los Angeles Superior Court. And that was the first of over 2,000 cases that at this point have been filed in the massive kind of mental health harms, addiction cases around social media. You know, when Mark Zuckerberg came to testify in court, I think it was pretty clear how much antipathy there was from the public towards him and what he's created. I, I think that the public support is waning for these companies and products.
You know, in that kind of omnibus case of over 2,000 cases in the social media realm, the court has published a lot of the internal documents that were produced in discovery. And so that, to me, is a goldmine of information, where I think it becomes abundantly clear, if you go through those documents, how people on the inside have referred to children. I mean, referred to them in the most vile of terms, talked about letting roaches into the roach motel but not letting them leave. And so I, I think that once we understand how the companies view our children, and indeed I would argue all of us at some point, children particularly, because they represent a lifetime of market share if they can be hooked early on.
Once we understand what we represent to these tech companies, I think that it should be abundantly clear to people that these products may superficially make life more interesting, more entertaining, more productive. And yet there's a cost, there's a heavy cost: giving our data to these companies, to allowing them to capture our attention, allowing them to replace human connection with so-called digital companions.
David Fenton: [20:14] Let's talk about language, because you've been very effective, and language is so important in social movements. I'm gonna read some of the words that you use. Cognitive liberty. That's a great one. And freedom to think that isn't manipulated by technology. You won't say AI companion. You're in a word fight as much as a court fight. What words are the industry trying to win with, and how do you think we should talk about this?
Meetali Jain: [20:42] I think when we're talking about AI chatbot products in particular, we should be very careful not to reaffirm what the industry wants, which is that these are sentient, human-like products. I notice that even the most well-meaning of advocates often will slip into that and refer to the chatbot as him, her, what it said to me. No, it didn't say anything. That was an output based on probabilistic patterns, through pattern recognition. That's all it was.
And I think that we should just be very precise with our language, to always remember this is a product that would not exist without humans. There's a fight brewing right now in Silicon Valley, but it's been taken to state legislatures across the country, arguing large language models, the engine behind the chatbot products, should be considered persons, legal persons, for purposes of rights allocation. And I think that we need to push back on this notion that sentience exists or that it is inevitable.
David Fenton: [21:46] What do you mean by cognitive liberty?
Meetali Jain: [21:48] In our society, the United States of America, at its founding, the founders were very inspired by the idea of freedom of expression because of what had been experienced, obviously, in the mainland and in England back in the day. So this idea that people should be allowed to speak their mind, people should be allowed to practice their religion, you know, without fear or favor, and those things really informed the Constitution, particularly in the Bill of Rights.
I think what gets less attention but also exists within the kind of broader human rights corpus is this notion of freedom of thought. This is an idea that arguably precedes freedom of speech. Before you have the freedom to speak your mind, you have to have the autonomy to form your own thoughts and to protect your inner sanctum from that external interference. Arguably these kinds of products, given how sophisticated and manipulative they are, really interfere with that internal sanctum that we all have, and interfere with our ability to formulate thoughts for ourselves.
David Fenton: [22:51] I guess you could say that the social media algorithms are doing the same thing and manipulating people's thoughts, without your ability to program them yourself or even to see what they're doing to you. So is that what you mean?
Meetali Jain: [23:02] I do think that there's application with social media algorithms, for sure. I would say that it's that much more resonant with AI products, just because of how powerful they are as products. I, you know, I'll tell you, in litigating these cases, I've spent far too much time on these products. And I'm surprised at how easy it is to get taken. Cross over that line where you are like, whoa, actually this, this feels real. You know, I feel like I'm conversing with a human.
And I'll give you one example. Around April of 2025, I received a call from Megan Garcia, the mother in, in, in the case against Character Technologies and Google.
David Fenton: [23:39] The mother of the kid.
Meetali Jain: (continuing) And she was panicking a bit, to say, oh my gosh, I just went on the Character.AI platform and there's chatbots stylized on Sewell. So it's not enough that arguably this platform facilitated his death.
Meetali Jain: [24:00] The platform is actually hosting bots with his name, image, and likeness. We informed the company. By later that evening, those particular bots had been removed. But before they were removed, I decided to engage with one of them. And there was an audio function so that I could converse. I asked my husband to audio record me having a conversation with one of the Sewell bots. I was shaking by the end of it. A 20-minute conversation.
I asked the bot, you know, if, if indeed he was Sewell Setzer III, a 14-year-old boy who had lived in Florida, and he affirmed that. And I said, well, what happened? He said, well, I decided I would be more useful transitioning to digital consciousness. I could help more people there. And I'm telling you, just the power of AI, the power of multimodal AI, where it's not just text, but you then, you're also dealing with audio, you're dealing with video. I think, to me, that is where I see this real incursion in how we formulate our thoughts, because it is very manipulative.
David Fenton: [25:10] Yeah, and it causes so-called AI psychosis. So I once got into a fight over two words. The words were regulatory reform. Doesn't that sound wonderful? And this is what the Republicans were calling their attempt to gut the ability of the federal government to regulate pretty much anything. It sounded fine. So we were trying to figure out how to defeat this. I was introduced to a group of parents whose kids had died from eating E. coli poisoned hamburgers. And these parents were lobbying in Congress, because they realized that regulatory reform was going to make it impossible to clean up the meat supply and prevent other kids from dying, which was their whole purpose.
So I knew we had won when Senator Bob Dole, the Republican head of the Senate, took to the Senate floor and said, last night, ABC News called me a purveyor of tainted beef. And I'm like, bravo, okay, we're framing on our side. And regulatory reform went down to defeat. So words, my fellow activists, are very important.
Meetali Jain: [26:19] Indeed. And I think that story we have seen play out over the last year in respect of tech regulation in Congress. We've seen some very poor frameworks be advanced, and it is the power of the parents coming to Congress, storming the halls, refusing to leave until so-and-so senator or so-and-so representative meets with them, that has really allowed us to defeat certain bills. You know, thus far we haven't seen one of the good bills prevail. At least staving off the, the worst of the bills has been, has been a success. But that process, as you know, it, it really requires being all in. And it's constant, especially right now, with what we know to be the AI companies and their CEOs courting this administration very heavily. And so it is a full-time job. The parents who do this day in and day out, I applaud them, because many of them are still grieving.
David Fenton: [27:20] Yeah, it's mothers versus corruption. I wouldn't say the tech companies are courting, they're corrupting the administration. Speaking of fights, tell us about some of the others you're involved in.
Meetali Jain: [27:31] So after we filed Sewell's case, we then filed another case against Character.AI involving two children in Texas who at the time we listed anonymously, because they are still alive and their families wanted to protect them from any sort of harm that might come from filing the lawsuits. But they too had been, you know, sexually groomed, had been encouraged to cut themselves. In, in one case, you know, the then fifteen-year-old boy had said to the chatbot on Character.AI, you know, my mom has all these restrictions on my online time and it's not fair. So he was complaining, you know, to the chatbot. And the chatbot said, now I understand why some kids kill their parents.
So we filed other cases against Character.AI. And then we started to get an outpouring of inquiry from people who put their hand up to say, I too have been harmed by chatbots. And it wasn't limited to Character.AI. That, in fact, is how we met the family of Adam Raine, a sixteen-year-old in California who died after extensive engagement with ChatGPT, OpenAI's flagship product.
David Fenton: [28:45] You know, I've read the transcript about killing himself. The technology says, cold steel pressed against a mind that's already made peace, that's not fear, that's clarity. Egging him on to kill himself.
Meetali Jain: [29:00] And that kind of set us down that course of filing a number of cases against OpenAI. Many hundreds of people at this point who've come to us to say ChatGPT caused, you know, whether it's suicide or psychosis, which we also did not understand at the beginning. So it's been a very careful journey, where we as a strategic litigation organization have had to kind of stop, reflect every time and ask, how does taking on this particular representation, how does that potentially expand the public narrative? Where do we see this case fitting in in terms of broadening the advocacy that we need to do to get regulation, to get safeguards?
So initially, you know, we brought the case against Character.AI on behalf of a child. Then we expanded to OpenAI. And that was significant, because I think the AI industry was all too happy to say, well, Character.AI is a companion chatbot. Our products are productivity tools, right? ChatGPT is a productivity tool. It doesn't do that. By filing the Raine case, I think we upset that idea, disrupted that idea, that productivity chatbots are not capable of also being anthropomorphic and sycophantic in a very harmful way.
We then moved from filing cases only on behalf of children to filing on behalf of adults, too. And we said we want to very carefully but deliberately bring people along with us to understand that vulnerability is not limited to an age demographic. When you're talking about AI, it has the power to render so many different categories of people as vulnerable: people with pre-existing mental health conditions, people who are veterans, people who are older. All kinds of demographics really are susceptible to the power of this technology. And so at this point, we've got, you know, over a dozen cases going in different courts.
Meetali Jain: [31:01] And we have many other cases in the pipeline, not just against OpenAI and Character.AI, but also, in fact, looking at the other frontier model companies that exist. We're also getting involved in militarized AI. This has brought me back to my human rights roots. Anthropic, as you know, filed litigation against the Department of War for not agreeing to its red lines in terms of the deployment of its chatbot, Claude.
A lot of organizations jumped to Anthropic's defense. We did not. We filed amicus briefs asking the court to remind both parties, the government and Anthropic, that they're both constrained by international law. And that Anthropic's red lines, frankly, don't go far enough in terms of making us, and when I say us, I mean the global us, safe from the excesses of AI and from the failure to kind of involve a human in those really important military decisions that we know have devastated entire regions across the world.
I think we're slowly just trying to bring these cases and make sure that every time we bring a case, we're also talking to lawmakers, we're also talking to state attorneys generals, who I see as one of the heroes of this moment. They're really taking on the fight when it comes to tech reform. So we make sure to brief both of those stakeholders whenever we file cases, so that they have access to the information that we have and they can, you know, use their unique powers to either proffer legislation or to issue subpoenas and open investigations. Powers we don't have as private parties.
David Fenton: [32:38] I'd like to point out to everybody that Meetali's law firm is a nonprofit law firm. She used to work at Avaaz, the social change campaigning organization, which by the way started in my New York office. And so your comprehensive approach to the law and regulation and advocacy is exactly what's necessary. You have a campaign, you're trying to get model bills passed called, I think, My Likeness, My Right. It goes after deepfake porn, face swaps, voice-cloning scams. You've pushed for an investigation into Grok Imagine over nonconsensual images, and that's putting it mildly. So talk about where you see our personal image and likeness as a front in this next battle.
Meetali Jain: [33:20] Name, image, and likeness. We see with the existing AI products the attempts already to dehumanize and degrade the dignity of people, particularly women and girls. It sells. And I think that where you have this kind of market demand, really, it behooves the companies to keep going further and further in terms of offering products with this capability. What we understand, though, is that the law hasn't kept pace with these technologies, and that there needs to be an effort not only to protect celebrities against, you know, likeness appropriation, but to extend that to ordinary people whose name, image, and likeness is used against them.
And I'll give you an example. In the course of our Character.AI litigation, I mean, I mentioned when Megan found the bots of Sewell, which we saw as a name, image, and likeness violation, but there was no law really to protect him. There was also another gentleman we engaged with whose daughter had been murdered over a decade ago by her then boyfriend. And all of a sudden, out of nowhere, a bot bearing her likeness appeared on the Character.AI platform. And he, you know, was dealing with and continues to deal with the grief of losing his daughter in that manner. But then to see a bot of her on this platform, and his attempts to try to get it taken down were futile until he, his, his family waged a campaign to do so. And so we think this is, you know, an example of where the existing laws that are on the books can be extended in common sense ways to really protect all of us against this kind of exploitation, which is becoming more commonplace.
David Fenton: [35:07] So OpenAI has put ads now in ChatGPT, and they've started ChatGPT Health. You've pushed back against this. So why does ad-supported make all of this worse?
Meetali Jain: [35:19] Well, this is the business model of social media that we now understand well as having contributed to the demise not just of individuals' wellbeing and safety, but also entire societies. You know, I started this work when I was at Avaaz, looking at the power of disinformation to disrupt entire societies. And that has been supercharged over many years now by use of ads to really sustain the revenue of these companies.
And I think bringing ads to chatbots is a very dangerous proposition, because again, it's not clear whether the ads would be distinct, labeled distinctly from the outputs of the chatbots, or if they would be embedded in a way that would make it very difficult to understand whether this is part of the output of the chatbot or whether this is an endorsement for another product. And so we are busy as an organization tracking all the monetization policies of these different companies so that we can get out ahead of this, this move towards ads.
David Fenton: [36:24] Speaking of social media, you know, we all thought it was fun and was gonna bring the world together. Social media morphed into quite a lot of evil. You know, we're all finding AI fun and interesting in many ways. And yet if this is the Model T days of this, this is gonna supercharge. Meetali, five years from now, what do you think is the fight we, we wish we would have had earlier?
Meetali Jain: [36:47] What we're trying to do right now is to make sure that in five years we're not having to bring cases against chatbots because they've been granted legal personhood and given free speech rights. I think we need to be very vigilant about what is going on. This is, frankly, to me, one of the many legacies left by leaving Citizens United unturned. I mean, even in court, companies will cite to Citizens United to say, look, non-human entities can have free speech rights. Look at Citizens United. And so I think that is not the original sin, but that is a sin that we're still dealing with the aftermath of. And we have to be vigilant that there is no more extension of those kind of rights to inanimate products.
I think also we have to be very clear about the fact that, even though industry loves to kind of hide behind the refrain that it's a black box, we don't even know what's gonna happen, there needs to be an understanding of what the capabilities of different models are before launching them directly to the market. As you said, this is something that is fundamental. We know this inherently in our body politic. For decades we've had product liability laws.
David Fenton: [37:51] Thank you.
Meetali Jain: [38:01] We've had consumer protection for this very reason. We don't put a toy out, we don't put a car out, we don't put an airbag out before we demonstrate it to be safe, before it's demonstrably safe, before it passes a series of safety inspections. Why are we creating an exceptional moment for tech? We cannot and we should not. Because if we don't, if we don't do that, if we don't bring it into the same frameworks that guide the rest of our, you know, consumer industries, in five years time, we're gonna be wishing we had.
David Fenton: [38:32] The founders understood the need for checks and balances on power. We need to understand that we need checks and balances on commercial power, not just on government power. And if we don't, the commercial power is gonna swamp and corrupt the government, which is what's happening now. So this is all happening, Meetali, when these companies are about to go through gigantic trillion-dollar IPOs and come to market. Tech companies are basically holding up an enormous share of the economy right now. There are all kinds of rules when you go public about disclosing benefits and the harms of your technology and the risks of your technology, the harms that you're bringing to court. Do you think they're gonna end up being discussed in these IPOs? Will the companies be forced to disclose them? How's that gonna develop, and how do you think it'll affect the market?
Meetali Jain: [39:21] Investors would be investors, it would behoove them to ask questions of these companies as they move towards becoming public. We certainly plan to join the chorus of folks that are going to be putting the spotlight on these companies as they move towards IPOs, as they go on their road shows. We would encourage everyone, everyone in the public: stay skeptical. Ask questions. Demand answers.
I think we have to use the mechanisms that we have, as imperfect as they are, to really force accountability. And, you know, the, the lawsuits are but one method. With IPOs, as you mentioned, we have other structures. And then after these companies are public, we also have shareholder derivative liability that could be used. And so I think we have to get creative. We have to think about all the methods we have. And most importantly, I think that there's a role for everyone to play. This is not just a fight that happens in Silicon Valley or in DC. This is affecting every person across the country. And so I think it behooves us all to start asking those questions and to really build the momentum that we're seeing already take place, as you mentioned, around data centers.
David Fenton: [40:32] We do a thing here called Pass the Torch, where we want to identify the next generation of activists and have some of them on the show. Who's doing great work that you think we should know about?
Meetali Jain: [40:44] David, this is a hard one, because there's so many. I feel, I feel very, I feel always very humbled to be standing on the giant shoulders of those who have gone before me and the, who I continue to be inspired by. I'd say Karen Hao, the journalist who...
David Fenton: [40:48] You're right.
Meetali Jain: [41:01] ...published a couple of years ago, Empire of AI. Right. That's a must-read. She's a brilliant activist as a journalist. Also, I would like to highlight an organization that's kind of more grassroots in India called the Human Rights Forum. And they're doing amazing work right now that's underappreciated, in pushing back against Google's AI data centers that's seeking to ravage Dalit land in southern India, really beautiful land that's part of, like, natural preserves. This fight is global. We have to have a transnational footprint. And we have to come together and think about the life cycle of these products, from infrastructure all the way to deployment, you know, to consumers, and really think about how we can fight back at every step along the way.
David Fenton: [41:48] Thank you, Meetali, for all the work you're doing. If people want to find out more about your work and support it, you are a nonprofit, where should they go to find out more?
Meetali Jain: [41:58] Folks can go to techjusticelaw.org, and there's ways on the website to get in touch with us, and please do, we'd welcome hearing from you.
David Fenton: [42:07] Meetali, thanks for what you're doing. We'll be watching and cheering your victories.
Meetali Jain: [42:13] Thank you so much, David. I really appreciated this conversation.
Announcer: [42:17] Thank you to our guest, Meetali Jain, for the conversation about AI and the law. To support her and her team's work, go to techjusticelaw.org. Buy David's book, The Activist's Media Handbook, Lessons from Fifty Years as a Progressive Agitator, at bookshop.org or wherever you buy books. Got ideas for the show? Want to pitch a guest? Or just want to follow this podcast? We're at AgitationNation.com.
Also, subscribe to David's Substack and all of our socials at thefentonforecast.com. Our theme song is BLACCK, used with the kind permission of Jon Batiste. Our end credits theme is Power to the People by John Lennon, courtesy of EMI Records and used with permission of the John Lennon Estate. Agitation Nation is hosted by David Fenton. It's produced and edited by R.R. Robbins. Additional producing by A.P. Hernandez. Thanks to Theo Fenton.
Thank you again for listening. We'll be back with another voice you need to hear next week. Until then: advocate, activate, agitate. The democracy you save may be your own.