Subscribe
Share
Share
Embed
In this episode, Priten speaks with Cody Venzke, senior staff attorney with the ACLU's Speech, Privacy, and Technology Project, about who is actually protecting student privacy when the law has not caught up to the technology. They walk through what FERPA and COPPA do and don't cover, the limits of "FERPA compliant" as a marketing claim, how AI surveillance tools are being deployed in schools without adequate vetting, and where parents and teachers can apply pressure when federal law leaves gaps.
Key Takeaways:
- FERPA was written for filing cabinets, not cloud platforms. Passed in 1974, FERPA still grants parents a right to access every record a school maintains about their child, including data held by ed tech vendors. But it has never been enforced by the Department of Education, and individuals cannot sue under it, which leaves most of the work to proactive parents.
- "FERPA compliant" on a vendor website is a marketing slogan. There is no Department of Education certification program. The obligation falls on schools to ensure their vendors actually limit data use to educational purposes, and parents should ask schools how they define "school official" and what contracts allow.
- COPPA stops at the thirteenth birthday. The Children's Online Privacy Protection Act applies only to sites directed at children under 13, leaving teenagers in what Venzke describes as a regulatory wild west. The ACLU argues that data minimization and affirmative consent should be extended to everyone, not gated by age.
- Flat bans on minors using social media will likely lose in court. The Supreme Court has held that minors' First Amendment rights are largely coterminous with adults'. Venzke predicts that age-based bans will be struck down as overbroad, and argues that regulating how platforms collect and use data is a more constitutionally durable approach than restricting speech.
- School AI surveillance is being deployed without testing. Facial recognition, weapons detection, and communication monitoring tools are sold to schools without proof they work as advertised. Venzke cites cases where students have been outed by large language models that misread diary entries as bullying, and argues that high-impact AI uses should require state-level vetting requirements.
- Removing a student's name from a ChatGPT prompt does not make it FERPA safe. Identifying details like "the only Native American student in fifth grade" can still trace back to an individual. Venzke argues teachers should not be left to vet AI tools on their own; districts, states, and procurement processes need to do that work.
Creators and Guests
What is Margin of Thought with Priten?
Margin of Thought is a podcast about the questions we don’t always make time for but should.
Hosted by Priten Soundar-Shah, the show features wide-ranging conversations with educators, civic leaders, technologists, academics, and students.
Each season centers on a key tension in modern life that affects how we raise and educate our children.
Learn more about Priten and his upcoming book, Ethical Ed Tech: How Educators Can Lead on AI & K-12 at priten.org and ethicaledtech.org.
Priten: Welcome to Margin of Thought, where we make space for the questions that matter. I'm your host, Priten, and together we'll explore questions that help us preserve what matters while navigating what's coming. Today I'm joined by Cody Venzke, a senior staff attorney with the ACLU's Speech, Privacy, and Technology Project. Cody works at the intersection of the First Amendment, privacy, technology, and student rights. We get to talk about FERPA, COPPA, AI tools in schools, student surveillance, social media regulation, and what parents and teachers should know before trusting a platform with student data. More specifically, we get a chance to talk about who is actually protecting student privacy right now when the law has not caught up to the technology. Let's get going.
Cody: Great to be here with you today. I am Cody Venzke. I am a senior staff attorney with the American Civil Liberties Union. I work on our Speech, Privacy, and Technology Project, which focuses on the intersection of the First Amendment, technology, and privacy, both under the Fourth Amendment as well as statutory authority.
Priten: Awesome. We have a lot to talk about today, so I'm very excited to have you on the show. I'd love to start with a conversation that I think matters for both educators and parents. When they think about legislation around privacy for their students, they think about COPPA and FERPA. For folks who aren't super familiar with the basics of those, do you mind doing a quick introduction and maybe telling us the lay of the land in terms of how they fit in these days?
Cody: Yeah, happy to. COPPA and FERPA are longstanding laws. The younger of the two is COPPA, which is now coming up on its 26th birthday, and FERPA is now well over 50 years old. So it's safe to say they were written in slightly different periods than what we live in now, but you could think of them roughly as dividing the world into two halves. One is the big tech online platforms that we might encounter outside of school. And then FERPA is the coverage for data used inside of schools. They both do significant work to protect people's privacy, but because they are laws that are slightly older and written for slightly different times, they also have challenges applying and keeping up with today's rules. For example, FERPA, which was passed in 1974, was very clearly written with filing cabinets that have student records in the central office in mind. But that doesn't mean it doesn't provide protections. And it's important that parents and children understand both the protections and rights that these laws provide so they can take advantage of what is already on the books.
Priten: It is amazing how laws that were not written for this time period are some of the ones that we rely on the most right now in the space. Do you mind giving us an example of what kinds of protections FERPA provides in particular? What it was meant to protect and then how it's been reimagined in the last 20 years or so?
Cody: I'd love to. FERPA is the Family Educational Rights and Privacy Act, and it applies to any education records that are held by schools, so it doesn't apply to records that are held by other entities. But it critically applies not just to schools, but to all of their contractors. So if they're working with a technology company to run a student information system or with something like Zoom to do remote learning, all of that information is going to be protected by FERPA. And FERPA does two things. One is it provides parents and students with certain rights. The other thing it does is it limits the disclosures that schools can make without parental consent. On that first front, the rights that FERPA establishes, the key is access to student information. Parents are allowed to request from a school, "I want to see my child's education record," and that's not just the permanent file that might be required by state law. It's all records maintained about a student by that school. And like I said before, that includes information that is maintained through electronic vendors. Sometimes parents get the runaround from schools and companies saying, "Oh, we don't have that. Go to the company." And the company says, "You have to go to the school." It is your right under FERPA to access those electronic files. I'll also say that in addition to the rights that FERPA provides to students about that right to access, it also limits disclosures. Now, FERPA has a lot of exceptions built into it, including disclosures to school service providers like Zoom or that backend information system, or for research around improving pedagogical efforts, which I know you might be interested in. Those exceptions, though, have conditions that have to be met. So for example, if a school provides student data to a contractor like Zoom or Google Classroom, that information can't be repurposed for non-educational purposes. The vendor is required to adhere to the purpose that the school released it for. So FERPA really does still serve, even though it's over 50 years old at this point and thinking about retirement, maybe. It still serves to provide important protections for students and their families.
Priten: When we talk about the fact that the data can only be used for educational purposes, how broadly has that been defined in follow-up litigation? Questions that come up often are, if a vendor is using data to train an algorithm to better serve their students, does that count as an educational purpose versus, like, marketing Amazon products to them?
Cody: First you asked about follow-up litigation. One of the dirty secrets about FERPA is that there's been very little litigation, and part of that is because the Supreme Court decided in 2001 that FERPA didn't provide families and students the right to sue in court, that the only entity that gets to enforce FERPA is the United States Department of Education. And infamously in ed tech circles, FERPA has never actually been enforced by the Department of Education. It provides lots of coaching and guidance to schools on how to follow FERPA's rules, but it's never actually brought an enforcement action. Now, that's changing a little bit under the Trump administration, which is using FERPA for what I think we can safely call sort of unorthodox purposes. But right now we've never seen an enforcement action brought against a school. That said, it doesn't mean that FERPA has not been the subject of live interpretation and development. For example, these specific provisions that allow data sharing with education technology vendors were updated in 2011. And as far as FERPA rulemaking goes, that was quite the fracas where there were a lot of folks interested who weighed in through comments, a lot of debate about whether or not we were giving up too many privacy protections. But one of the big things that that provision did is it expanded an existing exception known as the school official exception. When you think school officials, you're probably thinking teachers, vice principals, things like that, and that's how that exception allowed sharing of information within the school system. In 2011, the US Department of Education expanded it, rightly or wrongly, to allow education technology vendors to also receive information if they met certain requirements. One of those requirements, as you just suggested, is that they only use the data for legitimate educational purposes. Now, FERPA doesn't really define what is a legitimate educational purpose. It is up to the school when they contract with the vendor to make that clear. So that's one of the things that parents should be aware of when they're interacting with schools, when they're asking questions about technology: how do you define school official? What information are you sharing and for what purposes? And then one of the key things is ensuring that schools have follow-up to make sure those vendors are limiting that use of education data. I will note one other thing: this was widely recognized as a potential problem with FERPA, that it sort of left schools to police what big tech companies were doing with student data. Many states chose to step in, passing laws known as SOPIPA, S-O-P-I-P-A. That provision made it very explicit. If you are in the K-12 space, you can't use information for marketing and research. You can't use information to develop new products or for targeted advertising. So although FERPA is maybe the predominant education privacy law because it's the federal law on the books, it is not the only player, and states have an important role in protecting student privacy as well.
Priten: I feel like in the educational world, this often comes up with marketing language where you see, "Oh, we're COPPA compliant," and that is used as the end-all, be-all sign of data security and privacy rights for students and parents in particular. And you've made it clear that that is not as broad as it's sometimes made out to be. When I think about this, I'm thinking about the difference between having to ask a parent permission before something is done with their student's data versus a parent being able to reach out and advocate for their student. How much of this ends up being in the second part? Because from what you're saying thus far, it does seem like most of the protections are about allowing parents to do something if they wish, more so than providing baseline protections for every student regardless, or requiring schools to do a lot of outreach to parents. Is that accurate?
Cody: Yeah, that is, I think, very accurate. One thing I will note is that FERPA obligations fall on the schools, and there is no Department of Education authorized certification program for ed tech to go through that says, "Congratulations, you're FERPA compliant." You noted that that was a marketing slogan, and I cannot emphasize that enough. FERPA compliance in glossy brochures or on ed tech websites is just there for marketing purposes. It is still the school's responsibility to make sure that their education technology vendors are actually adhering to the letter of the law. "FERPA compliant" on a website doesn't guarantee anything. As for the rights that FERPA awards, or rather maybe the proactive obligations it imposes on schools, there aren't a lot. Schools are required to put out certain policies, like who they identify as an education tech vendor, to give parents an opportunity to opt out of certain kinds of data sharing. But for the most part, it does envision a world where parents are taking a proactive measure to sit down and ask for access to student records. To some extent that makes a little sense, where schools are big operations, we want them to be able to teach and engage in many of the other helpful things that they provide to their students without having to necessarily proactively provide those obligations. But just because FERPA doesn't necessarily require that proactive disclosure or outreach doesn't mean that they should not. There are some really good baseline principles that schools should engage in, such as providing public information about what ed tech platforms they use, what data they collect, and what purposes that data collection is being used for; if they can make those agreements with education technology platforms publicly available; and creating fora for parents to come and ask questions about technology. One of the things that the pandemic made obvious for parents was just how much technology, how much data is being used, and they will have questions. And that has not changed since schools have gone back to in-person over the past three or four years. So recognizing that parents are concerned about technology, that they have questions about technology, and creating avenues and spaces for that is, I think, critically important, even if it's not strictly required by FERPA.
Priten: It's fascinating to hear how much of this requires good actors who have the right information but also are acting in good faith for all of this to provide the protective measures that we hope they do. I'm curious to hear a little bit about the litigation work you do. Given that you were talking about how there's very little history of litigation based on FERPA, what is litigation relying on? I know you mentioned the Fourth and First Amendment. Does that end up being the bulk of the cases that you all work on in terms of privacy rights?
Cody: The First and Fourth Amendment are key to our work. Obviously, as the ACLU, we are constantly looking to the United States Constitution as the beacon of civil rights and civil liberties that we seek to protect. Of course, that's not the only source of our rights, and Congress and state legislators pass legislation to help ensure that people are protected. I think FERPA could be fairly called one of those pieces of legislation, even if it doesn't allow individuals to sue on their own rights. It is a place that we've worked on. Obviously, because FERPA doesn't allow individuals to sue, it's not a place where we've actively litigated. But for example, we provided guidance last year about education institutions' obligations under FERPA and how that intersected with some of the questions that were emerging under this administration, such as: What are your obligations if ICE seeks information on undocumented students? What are your obligations as artificial intelligence continues to be an ever-growing piece of our technological ecosystem? So it is definitely a space where we think about and we look for ways to remind schools of their obligations, remind parents and students of their rights, even if it's not a statute that lets us litigate directly.
Priten: Can you walk us through some of those obligations, especially when it pertains to artificial intelligence?
Cody: Well, I'm actually going to start with ICE for one particular reason. One of the many misconceptions about FERPA is that FERPA is sort of surrounded by this mythos of urban legend of what it requires and doesn't require and what it permits. One of those myths is that it just generally lets schools share with other governmental agencies, and that's simply not true. FERPA has specific exceptions for sharing with specific entities, with, for example, the Department of Education, that require certain conditions to be met, such as a written agreement that requires the data to be deleted after a certain time. But there's no general exception that says schools may share information with any government agency that they choose. Applicable to ICE, it's applicable to your local law enforcement, it's applicable to your local health agency. Schools have to protect student information and follow the law, even if the entity asking for it is another governmental agency. So that's maybe the first thing that we highlighted in that guidance.
Priten: The scenario I'm imagining is, like, the Department of Education requests the data and then they use it to inform ICE. What protections exist for something like that? Is that an obligation of the Department of Education or of the school itself? This might be in the weeds, but personally, I'm curious about how far these protections go.
Cody: FERPA obligations extend to educational institutions. They do not extend to the Department of Education. Now, for that data sharing with the Department of Education, FERPA is explicit. It requires a written agreement that includes purpose limitations. The Department of Education, or its authorized representative, can't repurpose the information and use it for other things such as immigration enforcement. What you've asked about is truly a deeply structural issue. Could a school enforce that written agreement against the Department of Education? We don't know. Would there be other limitations, such as a lawsuit that could be brought under statutes like the Privacy Act, which coincidentally was passed in the same year as FERPA in 1974? So those are all really interesting questions, and ones, unfortunately, and I do mean unfortunately, that are coming to light more and more as we've seen the Trump administration, through the Department of Governmental Efficiency, DOGE, through the Department of Homeland Security, break down longstanding norms and practices around federal data. And so if that ever came to pass, those statutes outside of FERPA might come into play.
Priten: I think one of the things that an unorthodox presidency reveals is how much of this is based just on precedent and norms, and how little sometimes we have explicitly put in protections for all kinds of spaces, and not just the technology and education space.
Cody: I think you put it spot on when you were describing FERPA, where you said it was sort of written with the assumption that there would be good actors at schools. I think it's true of many of these privacy laws, where not only was it assumed that there would be good actors, but that there would be adults in the room following those norms and precedents. And we are in an age where norms and precedent are being challenged.
Priten: I want to turn to COPPA for a little bit because for parents, I think that is slightly more obvious. It is slightly more relevant in terms of technologies outside of the classroom. Can you do the same thing we did with FERPA? Can you walk us through what protections COPPA does provide? Obviously, that was written for the digital space. How much of that relies on good-faith actors?
Cody: Yeah. COPPA is the Children's Online Privacy Protection Act. It was passed in 1999, so it was definitely written for online spaces. I think it was written before many of the technologies that we've seen emerge since then, including the broad, targeted advertising ecosystem, certainly before the emergence of artificial intelligence, before Web 2.0 and social media. COPPA has, I think to put it generously, had to stretch a little bit to keep up with those pieces of changing technology. Roughly speaking, we will say COPPA is sort of the FERPA equivalent that applies outside of the education space. It applies to private operators of websites that are directed to children. So there are really two critical things to note about COPPA. One is that it is not a general purpose privacy statute. It theoretically does not apply to the New York Times or ESPN or general audience websites. It applies to websites that are directed to children. So think Nickelodeon, Disney, things of that nature. The second thing is that when we say it protects children, we do mean that its application only applies to children under 13. So when you are 12 years old, you are protected by COPPA. The day you turn 13, you are no longer protected by COPPA. I will note this is a reason that the ACLU champions comprehensive privacy legislation, because there's no reason there should be a distinct cutoff between we are protected from having our data abused to now we have entered the wild, wild west. But setting that aside right now, that's a project that's underway politically. The protections that COPPA provides primarily focus on parental consent. If a child tries to use a child-directed website, that website is supposed to ask the child, "Hey, how do we get in contact with your parents through email, through a phone number, through any number of means that are identified in the statute?" And then the parents have to provide consent to use the child's information either to create an account, for personalization, yes, for targeted advertising. Then there are additional protections that are similar to what we saw in FERPA, such as a right to review the information that the website has, a right to delete the information, as well as certain rules that the FTC has recently reemphasized that limit the amount of information that platforms can collect, noting that it has to be somehow tied to the purpose for the collection.
Priten: What protections does COPPA provide that are really necessary only for children? Is there anything that seems like it was written that wouldn't apply to protections we would want for adults?
Cody: I would love to see many of COPPA's protections extended to adults in some ways. We're obviously not going to extend parental consent to 22-year-olds or something like that, but for the most part, the provisions that are there, of "we have to get your affirmative consent to use your information," "you have a right to review and delete your information," "we're going to have safeguards around the amount of information that's collected." I'm going to dwell on that for a second. That's a concept we call data minimization, which is the idea that a platform can only collect, use, and share your personal information as it needs to provide the service you requested. So not what's buried in a privacy policy, but if you want to, for example, download and use Candy Crush, there should be no gathering of your geolocation secretly in the background, because there's no reason for Candy Crush to be collecting that. If you download a weather app, yeah, they can use your geolocation to provide an exact forecast, but they're not going to turn around and sell it because there's no reason for that. That basic idea, which again the FTC has been emphasizing as sort of a latent provision in COPPA, is one of the key things that we should see in privacy legislation that's extended for everyone.
Priten: That part of it is interesting. And then I think the 13 to 18 gap is the other part that I'd love to talk about a little bit. When a 14-year-old, or I guess a 13-year-old, but a 14-year-old is signing up for a website, legally do we consider their agreement to the terms and conditions as legitimate the way it would be for an 18-year-old or older?
Cody: As far as COPPA is concerned, yes, they can provide consent under COPPA for data collection. COPPA is just completely silent on what are the rights of 13 to 18-year-olds. There's an overarching question that I am not situated to get into about minors' contractual rights. Traditionally, if you go to law school, they'll talk about the common law, which is basically the law we inherited from England when we broke away, which said that minors can enter into contracts. It makes sense. They can walk into a convenience store and buy bubblegum. That's a contract. But those contracts are voidable by parents. The parents can retroactively eliminate the contract as if it never existed. That might be a little bit more difficult to apply to online technology. But what you've touched on is a really hard question. One of the difficulties that we see in this space is the fact that minors, as they age, continue to accrue independence. They accrue autonomy. Anyone who has interacted with children from the transition from being a 6-year-old to a 16-year-old knows that you can't apply the same regime, whether it's for privacy or content moderation or whatever, to those two individuals, or even that same individual at those two different points in time. That is one of the things that has made legislation in this space so difficult: how do we recognize and codify teenagers' increasing autonomy as they transition into that role as adults in a free society?
Priten: When we think about that increasing autonomy, there's the legal ambiguity of how much legal autonomy does a 13 to 18-year-old have when it comes to entering these contracts. But there's also the level of how much of that autonomy ought parents and educators intrude on. That question has become more and more complicated as the amount of different options open to young children to partake in things has expanded. Historically, a lot more was physically driven by location. Protecting a child oftentimes meant knowing where they were. If they were at home or if they were at school, you knew somewhat what the risks were of them being in that location. That's very different now when you have an electronic device and that gives you access to the entire world and the entirety of the internet. You can't say, "Oh, my child's in their bedroom, they're safe." That's not enough to rely on the way it was maybe 30 years ago. What does that mean for how we think about the obligations of the companies themselves? There's a very fine line here between what parents can do and what these companies can do at the end of the day, and what we want the government to do and to step in. I know that this is a very complicated picture. I'd love for you to just talk through some of what you all are thinking about in that space.
Cody: Let's start with, not the companies, but the government, because we are seeing lots and lots of legislation emerging at the state and federal level about how do we regulate children online. For the ACLU, the first thing that we come to is the free speech rights of both minors and adults. One of the things that gets elided in these discussions is the free speech rights of minors. The Supreme Court has been emphatic, with a couple of very key exceptions, that the free speech rights under the First Amendment of a 6-year-old and a 16-year-old and a 26-year-old are coterminous. They are equal to one another. The exceptions being sexually explicit speech, which the Supreme Court just recently addressed in a case last summer, as well as, I think, in the school context. But beyond that, the Supreme Court, as I said, has been emphatic. One example of that is in the early 2000s there was pronounced concern about what are the negative impacts that video games are having on our youth, with lots of speculation that it was leading to aggression among teenagers, drug use among teenagers, and school violence among teenagers. I don't know if the research ever really supported those theories in a concrete, indisputable way, but that didn't stop states from attempting to legislate in that space. California in particular banned the sale of violent video games to minors, and that went all the way to the Supreme Court. The Supreme Court struck it down, noting that there was no traditional exception for barring minors from accessing violent speech and emphasizing again that the rights of children to access speech are in most instances on par with those of adults. So as we think about this space, as we think about regulation in this space, it's important that we start with that first principle about minors' constitutional rights.
Priten: Obviously, this is coming up most explicitly right now with social media regulations. I do think some folks' intuitive responses are that this is a no-brainer. Of course we should be protecting 14-year-olds from being on Instagram, given all the potential harms and how Meta wants to exploit them, whatever it might be. There are intuitive responses that I think are pretty strong. I think you all have a more complicated view on this. I'd love to hear a little bit more about how y'all are approaching the ability to ban minors from accessing social media platforms.
Cody: We will see as the litigation continues to pan out. My prediction, and maybe you can have me back on to see if we ever see this right or wrong, is those efforts at flat bans will be struck down as unconstitutional for being overbroad burdens on minors' access to speech. And I think for the most part, a lot of legislators don't want to do that. They recognize that minors benefit from being able to speak. I think they recognize that society benefits from actually hearing from minors. I don't think anyone's very interested in going back to Victorian England where minors could be seen and not heard, and are looking for alternatives. We're watching those alternatives. One of the big ones I would encourage entities to think about is data. It seems so wonky that there is some really challenging speech available online and speech that is awful but lawful. It's constitutionally protected, but much of the engines of social media and other online platforms are driven by the fuel of data. The government has more latitude, not unlimited, but more latitude in regulating how our information is used. If they think through certain ways to be able to ensure that minors, with the consultation and support of their parents, sort of decide ways that their data is being used by social media platforms, constructing feeds, for example, or in targeted advertising or connecting them with strangers, all of those focuses on data collection and then its use are ones where the government has more latitude. That's the space they should be exploring, as opposed to age verification and flat bans and trying to regulate the constitutionally protected speech off of the internet.
Priten: For the layperson, helping understand the difference between accessing information and being able to produce your own speech, are those equally protected for minors? I'm thinking, their ability to post on Instagram versus regulating what they're exposed to on Instagram. Does precedent treat that differently?
Cody: Precedent treats them as one and the same, and the answer is, I think, pretty intuitive for most folks. The right to speak means very little if others do not have a right to listen. I hate to cite 1984 because it's overwrought sometimes, but one of the delightful things, oh no, it's not 1984, it's Fahrenheit 451. It's often said that that is a book about banning of books. What's not illegal in Fahrenheit 451 is ownership of books. It's reading of them that's been made illegal, and that's why that right to listen is so key. Because if others cannot receive what is being said, then the right to speak means very little, and that right is equally protected. The right to listen is equally protected as the right to speak.
Priten: Super fascinating to think about how to still, you know, there are real harms we want to protect children from, but not do it in a way that infringes on things that are fundamental to how we view our basic rights and our children's basic rights. I want to turn now to thinking about AI. Maybe let's start more broadly.
Cody: You said we would get back to AI, so I'm glad.
Priten: I have that circled. I want to make sure that we get a chance to do so. I'll start with the broad question of what are the challenges that AI is bringing that other technology has not already brought, and has not already caused you all, or caused everybody, to have to figure out how FERPA in particular, but also COPPA, applies in those spaces?
Cody: One of the difficult things about artificial intelligence is that it is inherently a multifaceted technology. Up until 2023, and I know technologists are going to come after me for drawing a hard line right there, most technology you bought had a very specific purpose, whether it be a social media platform, which you buy with your data through targeted advertising, or Microsoft Word, or an assisted learning program that you might use in schools. That made it easier to evaluate, "Is this a good fit for my family, for my classroom, et cetera? What data does it need? What data is it processing? How is it being shared?" But artificial intelligence is much more expansive. A large language model, which is a chatbot, and that's only one example of a type of artificial intelligence, might be used as a search engine. A recent poll from Pew shows that the majority of teens are using chatbots primarily as glorified search engines, and frankly, I do the same thing. Or it might be homework help, or it might be an unlicensed counselor. The fact that that piece of technology is functioning in so many distinct ways makes it an interesting wrinkle to try to regulate, and not one that we've entirely wrapped our minds around at this point. The other thing that can make it very difficult to regulate artificial intelligence is the fact that they are probabilistic. We don't always know what outcome we're going to get from artificial intelligence, particularly large language models, based off particular inputs. We don't always know how we got from the input to the output, and that makes it extremely difficult to assess: Is this being fair and equitable? Is it being discriminatory? Is it being reliable and trustworthy? Because we don't necessarily know what the internal processes of that piece of technology are.
Priten: The examples that we see the most are in relation to chatbots, both inside the classroom and at home. But the kinds of technologies that use AI these days are much broader than that. Do non-generative AI technologies have a different set of challenges? Some of the things that we said we might touch on are, you know, face recognition technology, technology that polices speech by students, technology that tries to detect weapons in a backpack.
Cody: I think that's a point worth emphasizing. When we talk about artificial intelligence, it's a range of families, plural, of technologies. Of course, since the advent of public-facing ChatGPT, we've primarily thought about AI as large language models, chatbots. But as you just underscored, it can be much, much broader than that, including, for example, computer vision, where they are using cameras. It might be existing surveillance cameras, attaching artificial intelligence to analyze feeds that come through there. It might be facial recognition technology to identify people entering or exiting a school. Is this a known student? Is this a teacher? Is it a stranger? As well as technologies that are trying to, for example, detect weapons. Does the student have a gun underneath a jacket? Or detect aggression. Many of those technologies are untested, despite the fact that they might put students in contact with police when they are suspected, based off a camera on a computer making a guess that they've brought a gun to school. They are nonetheless being sold without adequate vetting. There are other technologies as well beyond computer vision. For example, we know that many schools are deploying technology to monitor students' communications using what sometimes could be just very basic algorithmic technology that has a list of words it's watching for, to more sophisticated large language models that scan chats, emails, documents, and things of that nature to look for signs of self-harm or bullying. That sounds great, but they also make mistakes where students, for example, have been outed based on a Google Doc that was their diary, where they question their sexuality, and the large language model assumed that the use of the word gay must be bullying. So we've seen these technologies being deployed across the school system, often with mixed results and often proving that they're untested.
Priten: Are there protections? Do we have anything on the books? Here's the kind of thing I'm thinking about. Is there protection against the student being profiled by this information? Is there protection against false negative and false positive rates? Is there anything that is currently preexisting that is being reapplied, and is there any push to add any of that?
Cody: It's very thin, unfortunately, and this is one of the situations where we see these existing laws really being stretched to the point where they may not be covering significant harms. There are very few laws, for example, that require significant testing of these technologies. That's one of the things that we have advocated for: that if technology is deployed in a high-impact scenario, like making decisions about whether or not a student advances to the next grade, or identifying whether or not a student is being too aggressive in the hallway and sending them to the principal for discipline, they need to be tested. But it's one of those places where we haven't seen significant extension of existing law to cover those particular scenarios. States here have an important role to step up. We are seeing states really think about ways that they can assure that artificial intelligence that's being used in state and local government is tested, that it is vetted, that companies are attesting to the fact that the technology works the way it's supposed to. It does not have discriminatory or unsafe impacts. So even though we have not seen the federal government, including Congress and the Department of Education, step up and talk about the ways that these technologies should be vetted and tested, states can play a significant role there as well.
Priten: I want to just think a little bit, as we wrap up, about parents and teachers themselves. Some of the things that there's ambiguity around for educators, or there's a lack of clarity and guidance on, is what they can safely do. I'll give you a couple of examples to make this more explicit: what safe uses exist for them that are not either unsafe period, or B, in violation of existing statute. An example that comes up oftentimes is, there are lots of folks who talk about the ease of using some of these generative tools for making IEPs. There's a range of suggestions that I've seen, from, "Don't include the student's name," "Make sure it's a paid account." There are all these little caveats that are supposed to make this a safe practice. Is that true in terms of safety, and is that allowed under FERPA? Is there a certain way to do this that is FERPA compliant on a teacher's end?
Cody: It can be incredibly difficult to assess whether these particular types of technology are sharing, disclosing, and repurposing the information that you put into it. One of the things that can be incredibly difficult is ensuring that you aren't disclosing personal information. Yes, taking the student's name off the record that you were inputting into Gemini or ChatGPT is a smart first step, but it is not sufficient because, for example, if it includes additional information like the fact that this is the only Native American student in fifth grade, it doesn't matter that you remove the name because the school community knows who that particular student is. There are many, many ways that removing the name and other identifiers alone can still allow the information to be traced back to an individual student. What I think is really necessary here is for teachers not to be looking for ways that they, on their own, can find technologies that are safe, but to advocate for themselves and for their students to the school board, to district administration, to state education leadership, to establish the guardrails and the procurement processes. I know your listeners want to talk about governmental procurement deeply here. Ensure that those entities, which are better resourced than your classroom teacher, are the ones who have a process in place for vetting this technology and ensuring that it's making appropriate use of student information. And again, this is one of the things that's unique about AI, because when it was just an online graphing calculator, you didn't have to wonder if it was safe and effective. But now the vetting is not just about data and privacy, but: does the AI do the thing that it purports to do safely and effectively?
Priten: That's very, very helpful. And parents, I know that's a much broader question, but what can parents do, especially given that FERPA relies so heavily on proactive parents, to make sure that their students are protected at school?
Cody: Yeah. One of the things that parents can do is step up and fill FERPA gaps while we wait for Congress to do so, and that means going to schools and asking questions about the technology. What technology is being used? What data is being collected? Can I review the contracts for that? Have you set up provisions for me to access my child's information that's being held by this education technology platform? What rights do I have, including the ability to correct student information that's been collected or held by that platform? What rights do I have to opt out and choose a non-artificial intelligence option? Those are all things that require parents to engage in a conversation with teachers, with school leadership, and with district leadership. I think in many ways it resembles what teachers can and should be doing, which is advocating for a thoughtful governance approach to the acquisition of AI, to the uses of student data.
Priten: When we think about parents trying to protect their children outside of the school setting, when they're thinking about their child's use of their iPhone at home, what are some things that parents should keep in mind in those spaces where there isn't an entity that they can call on to help provide some of those protective forces?
Cody: Even though the ACLU has been consistently skeptical about governmental regulation of minors' access to speech and ability to speak, it is fundamental that parents play an important role in coaching their children in navigating the complexities of living in a democratic free society. That's especially true of online spaces. So I think two key things come to mind. One, of course, is talking to your child and your teenager about online spaces, and ensuring that you are familiar with the spaces that they occupy. That's the applications that they use, the technology that they're using, and walking them through the potential dangers. When should you accept this friend request? When should you post this particular information? So that they are aware that they are encountering individuals who may have unknown motivations, that personal information put out there can have particularly harmful effects. And also recognizing and coaching through: How does the technology make you feel? What is your response to this post? What is your response to the amount of time you've been spending on the platform? In addition to those conversations, the second thing I would recommend is being aware of the options that are provided by these technologies. Many pieces of technology, whether it be hardware like your child's iPhone or social media platforms, provide tools for parents to be able to help their teenagers and children navigate these online spaces. Often they go unused, and that's one of the things I encourage legislators to think about: How can we legislate around setting toggles in a way that they are more used and parent friendly? But parents should, when they have that conversation with the child that says, "I'm on Snapchat, I'm on TikTok, and I love Pokemon Go," ask: What are the tools that those platforms are already providing for parents that they can use to help coach their child into being a well-grounded digital citizen?
Priten: Thank you so much. The amount of information you've shared today, I think folks are going to find so much to sit with and grapple with. I have a list of things I want to set up to spend a little bit more time thinking about, so I really do appreciate your time today.
Cody: Yeah, it's been a pleasure to be here.
Priten: Cody reminds us that student privacy cannot depend only on old laws, good intentions, or vendor promises. Schools, parents, teachers, and policymakers all have a role to play in asking harder questions about data, consent, surveillance, and student rights. This requires us to really sit down and ask: How do we prepare students for the future while protecting their freedom and privacy? For more on questions like this, order my book, Ethical Ed Tech, at ethicaledtech.org. Thanks for listening to Margin of Thought. If this episode gave you something to think about, subscribe, rate, and review us. Also, share it with someone who might be asking similar questions. You can find the show notes, transcripts, and my newsletter at priten.org. Until next time, keep making space for the questions that matter.