Subscribe
Share
Share
Embed
“Research Ethics Reimagined” is a podcast created by Public Responsibility in Medicine and Research (PRIM&R), hosted by Ivy R. Tillman, PRIM&R's executive director. Here, we talk with scientists, researchers, bioethicists and some of the leading minds exploring new frontiers of science. This season, we are going examine research ethics in the 21st century -- and learn why it matters to you.
Artificial intelligence is increasingly being designed to assist us. In this process, AI can influence our decisions and even emulate people we know and trust. These capabilities raise fundamental questions about what it means for AI to act in ways that genuinely benefit people. Today, we are joined by Alex John London, professor of ethics and computational technologies and co lead of the K and L Gates Initiative and Ethics and Computational Technologies at Carnegie Mellon University, and Ida Matson, a PhD student from Stockholm, to explore what beneficence and value alignment mean in the context of AI. Specifically, we'll discuss a paper they coauthored titled The Ethics of AI Systems that Emulate Identifiable Individuals, challenges for value aligned development.
Speaker 1:We'll explore whether ethical principles can and should be built into AI systems from the beginning, and what happens when beneficence and autonomy come into conflict, and what these emerging technologies mean for researchers, research institutions, and the broader research ethics community. I'm your host, Katherine Batzberg.
Speaker 2:And I'm Dan McClain.
Speaker 1:Do you wanna take it away, Dan?
Speaker 2:Sure. We do like to start with a little bit of a intro about your cell. So, Ida, if we could, let's start with you. How did you end up studying in this field? What is your background and what drew you to AI ethics specifically?
Speaker 3:Yeah, sure. As you said, I'm a PhD student. I'm just entering my fourth year, which means I'm in the scary land of just writing dissertation. Basically, this project started in my first year when I was working with Alex through the AI Caring Institute, which is a really cool space for interdisciplinary work. I did my undergrad in cognitive systems, which is this mismatch of CS, computer science, psychology, philosophy and linguistics.
Speaker 3:And then I worked in the automotive industry, actually, with driving assist systems for semi autonomous cars. And that was really useful to think about systems, technology, how they influence human lives, how human lives kind of change in reaction to technologies that are introduced into our lives and thinking about that proactively into the future and thinking about how we should design systems in ways so that the future kind of ends up where we want it to go. Those are some of the questions that I came to CMU wanting to work on. Alex was excellent mentor for this project, and tried to investigate some of those questions in the context of this AI caring institute, which brings together people in robotics, in computer science, in other domains that are specifically focused on healthcare for older adults facing cognitive decline.
Speaker 2:Great. Alex, I know you've been on the podcast previously, but for those who may not have heard that particular episode, do you wanna give a line or two about what you specialize in over at CMU and in this particular paper that you've written recently? I know you've written other papers on the same topic, but if we could focus on this one.
Speaker 4:Sure. So I'm a philosophy professor at Carnegie Mellon, but my work is in ethics, and it's been about ethics in AI and ethics in human subjects research. And the unifying theme is basically innovation and uncertainty. So innovation produces uncertainty and how to try to push innovation in directions that help people and how to manage uncertainty in ways that increase the probability of benefit and reduce the probability of harm. And then as Ida said, this paper came up because I was the co lead of the ethics core for this large MSF institute called AI Caring, whose mandate was to try to develop the fundamental AI that would be needed if we were going to create AI systems that would help provide assistance to older adults as they age and experience cognitive decline.
Speaker 4:And as someone whose parents are aging and experiencing cognitive decline and in the sandwich generation where I have my own kids I'm taking care of and I have my parents that I'm trying to take care of, I can see the value in the goal of the initiative and how many people are using these systems anyway. Using Alexa to turn your lights on and off, using Siri or increasingly using ChatGPT to answer questions about how to run the remote, all kinds of things around around the house. So so then this this paper was, in a certain sense, kicked off by this idea that sometimes older adults relate better to the sound or the voice of their child as opposed to the voice of anybody else. And if you're experiencing cognitive decline and you're disoriented and it's late at night and you're getting out of bed and you might wander and the voice of your child is saying, dad or mom, go back to bed. Works.
Speaker 4:And there was some evidence from another paper that we cite in our paper. That was the case in this one instance. Then emulating the voice of your child seemed like a pretty good idea in the sense that the older adult doesn't wanna wander, the child doesn't want their parent to wander.
Speaker 2:The paper, Catherine and I both read it in recent days. I couldn't put it down after I started it. I think it's really interesting and just there's a lot to talk about, but one of the things that jumped out at me was the categories that you use to identify different applications of AI, which makes sense, although I had never thought about AI applications in these categories before. So I thought it was helpful to see your framework. The paper goes and describes meaningful benefits, justified paternalism, deception, unjustified paternalism, and domination.
Speaker 2:So your case study with elderly suffering from cognitive decline, that would fall in the category of meaningful benefit. Correct?
Speaker 3:Yeah. So that would be a case of justified paternalism. If there is an intervention where basically you have, in this case, the child might intervene to control this Alexa, or whatever the system is, that emulates their voice to their parent while they're attempting to wander at night. So then you have this two dyadic relationship between the child on the one hand, who is controlling the behavior of the parent on the other hand.
Speaker 1:So where is the line between assistance and that paternalism? Is there a line, or is it always an assistive process? So
Speaker 4:I think the the goal here is to provide assistance. And in the example that I raised, it's paternalistic because the older adult is gonna do something that in the moment it seems like they wanna do. They want to wander, but they're not in the right state of mind. Their conduct doesn't really reflect their considered values. They don't wanna leave the house in the middle of the night, not be able to get back in.
Speaker 4:And the system intervenes to get them to do something else that aligns with their true values. They'd like to stay in the house and be safe, but their cognitive state doesn't enable them to act according to those values. So that would be an example of assistance. So one type of assistance can be a paternalistic system that helps you act according to your values when you aren't capable of aligning your conduct with your own values. A major worry in this space as we're trying to develop technological systems with the goal of providing assistance to older adults is that we create technological systems that are unjustifiably paternalistic towards older adults, that they limit their ability to do things not for what is in the genuine interest of that person, but because it serves the interests of their child or or the facility in which they reside.
Speaker 4:So that's in a very concrete sense in which we wanna make sure that the development and the deployment of this technology is assistive to the older adult, helping someone live according to the way they wanna live, and control manipulation domination, using technology to make them live in the way that someone else wants them to live.
Speaker 2:So as long as the technology allows either meaningful benefit or justified, that's that's the outcome you're seeking, is those two categories?
Speaker 4:If you are capable of making your own decisions, then paternalistic like, justified paternalism wouldn't necessarily be the the thing that we were going for. Right? There, we want the system to assist you, that you're using it to do the things that you want to do, to advance your goals and your interests. So this can happen technologically. If you're an older adult and you're on your iPhone, and my father, who is in his nineties, has an iPhone.
Speaker 4:And so he might think that he's doing what he wants to do. He wants to send an email or do something like that. And if he presses the wrong thing, then you can imagine he's in the right state of a mind. He has decisional capacity, but he's just mistaken about the steps he needs to take. A better Siri
Speaker 2:Mhmm.
Speaker 4:That would be like, are are you trying to send this email because that's the wrong thing to do? And it would intervene. And then it would intervene paternalistically in the sense that it's stopping him from hitting the button that would have deleted it, say, or that would have sent it to reply all or something like that. And it's saying, is that what you wanna do?
Speaker 1:So there would be restrictions and frameworks when I say, Yes, you can emulate my voice and you can help me with my family. There would be some kind of consent process that would go on. I think we all spiral off into these worst case scenarios that you hear about. Is there frameworks within what you were thinking of that would be wired into products?
Speaker 3:Yeah. So part of what the paper builds off of is this other framework that Alex and Hoda Haidari published called the nexus and intelligence, where they basically outline this idea of what it means for technology to be assistive. What that means is truly beneficial for a person. It's not just weekly beneficial. It's not just like status quo.
Speaker 3:Now you can, do a new thing, but it actually benefits you in the sense that you're either better able to achieve your life goals or your life plan in some capacity. So in the case of paternalism, because you have this dyadic relationship, in order for that interaction to be assistive for the user, you would have to have consent to act on their behalf if you have the right kind of access to their true preferences, even though they may not have that in the current moment that the interaction is happening. And that's part of the trick with actually designing these products because you have to account for the fact that you build in some type of mechanism to access their true preferences when you can foresee that they will not themselves have access to them in the future, for example.
Speaker 1:It's fascinating. I'm sorry. The spiral of the average thinker into this world. That must have played a part when you were doing automated car development, right? There are preferences of perhaps a user trusting this self driving car that's taking you somewhere.
Speaker 3:Yeah. And that's a really salient example of this and why I think the framework in general, an efficient analogous framework, is really useful. With the autonomous vehicle example, that's a really salient case for this too. A challenge for the automotive industry, just a tangential thing, is to build trust. Because why would you buy a car that you know is going to save pedestrians and sacrifice the driver or the people in the car?
Speaker 3:Why would you buy a car that would do the opposite? There aren't really clear reasons that we can find for designing a system one way or another, but you have to have some kind of interactive querying mechanism, for example. That are defined between a single human, or maybe some representation of a collective preference, and a system. But when we deploy these systems in real life, technologies can stand in some kind of social relationship and mediate social relationships on behalf of people through emulation. So, it's really important then to think about not the relationship between an individual and the technology, but the kinds of social relationships that they enable.
Speaker 4:If I can just follow-up on some of the things that Ida said there. So the original paper, the Beneficent Intelligence paper, in a certain sense, assumed AI was just a resource for a person to use. What does it mean for them to benefit? And that might seem like a very simple question, but it's actually much more difficult to be able to specify what are the conditions that need to be met in order for something to provide a genuine benefit. So one of the issues here, I think, is technology companies can rush ahead to produce a new technology where it's new and it allows you to do something that you couldn't do before.
Speaker 4:And that is not sufficient to say you've created a benefit because there are lots of things that you couldn't do before a technology that are just trivial. Right? Before the keyboard, I couldn't hit the shift bar, you know, the shift button. And like just sitting there and pressing the shift button, it's something new that I couldn't do before, but it's completely trivial. Does nothing to enrich my life.
Speaker 4:And you can create technologies that allow people to do really bad things, and that isn't then itself a benefit. Right? One of the underlying questions from this paper, there were two things about it. One is here, there's a more complicated relationship in these systems that emulate people because there's the developer of the system. Then there's the person who is being emulated.
Speaker 4:So if we go back to the original example, there's the person who is making the system where there we would talk about, say, the home monitor that's trying to keep a person from wandering at night, right, with the goal of allowing a person who experiences mild cognitive decline and has worse periods and better periods to stay in their home longer. So the developer for this example. The deployer might be the child who gets that system and uses it with a goal of trying to help their parents stay in their house for a longer period. And then the person who's emulated might actually be the child themselves. You could have people who develop systems and people who use them really enjoy them, but the person who's being emulated is being portrayed in ways that they would not like.
Speaker 4:So we can understand the worst examples of those things. And because that person's identity is is being being portrayed, doing things that they would not like, they would not value. Whereas I think if you were like, I wanna develop a system to help older adults not wander at night, that illustrates more of the alignment. Right? The developer can feel like they're doing something good.
Speaker 4:The person whose voice is used, it's being used probably for a goal that they would share, keeping this older adult from wandering. And the older adult doesn't wanna wander, and most children who are trying to care for their older parents don't want them to wander. Right? So that's that sense then in which you're trying to have this system that mediates this relationship because it's emulating the identity of an identifiable person using their voice or their appearance then got to be aligned with more than just the preferences of the user.
Speaker 2:So then how do you prevent one of the potential harms? Who is in a position to say, don't build this. Don't deploy this. Or if it's deployed, undeploy it. How does that happen?
Speaker 2:Is that something that you envision, or is that something that we need?
Speaker 3:I think definitely something that we need. The EUA Act is trying to get at some of these things, for example, recognising the fact that for foundation models like LLMs, for example, there is usually a model developer, and then there's some kind of deployment level interaction. Something that we also recognize in the paper is that we conceptualize control as being asserted by one individual. But that's obviously false, because you have some developer of the system, you have some upper layer of maybe the interface, and then you have the person who actually decides to use the system, which in this case will also count as a deployer. So teasing those apart and teasing, for example, responsibility in case of faults, figuring out where that lies is something that we definitely need regulation and policy to address.
Speaker 3:I think for foundation models, for example, this is something that there's a lot of attention to right now and regulatory frameworks trying to get at.
Speaker 4:Many of the regulatory frameworks, they presume there's a single locus of control. And usually that locus of control is the developer in some sense. So if you think about maybe Dan and Catherine would like to have artificial intelligence avatars, virtual instances of them that could do these interviews, do way more, a larger number of these interviews. Right? So you have a developer try to develop a system that would emulate you so that you could do your avatar.
Speaker 4:But then some user, some other person who interacts with it makes the system do what they want it to do, makes you talk about things that you would never talk about, makes you say things you would never say. That's entertaining for them. It wasn't what the developer had designed. They were trying to facilitate your goals. Your goals were to have a certain type of conversation in a certain area for certain purposes.
Speaker 4:But if a clever enough user could get the system to do other things and and it was entertaining enough that lots of people liked it or at least the user liked it, then there's conflict now between what they want, what makes them happy, what the developer intended to do, and what you, the deployer, and you, in this case, the the person who is depicted, wanted to have done. And so each of these parties exercises some control under different circumstances.
Speaker 2:And in your paper, in that section, is that where you have some of the ideas of watermarking or in some cases, cartoonifying, if that's a word? I don't know if that's a word. But something that would make it quite clear that is your avatar or your likeness. I mean, is that the leading thought on that? And how would a watermark work?
Speaker 3:Yeah, I think that there's two aspects to this, building off of Alex's example. One is the question of does the user recognise that they are interacting not with the real person, but an actual emulation of the person? Then another aspect of this problem is, of course, if you permit, for example, your likeness to be represented, what are the types of interactions that you are okay with being represented in, and what are the kinds of controls that you can assert for those use cases. I think in terms of watermarking, for example, there has been regulation recently trying to have foundation model developers insert watermarks for any kind of AI generated content, and these kinds of practices are already put in place. So that's not necessarily for the purpose of the user or the recipient of an emulation to be able to identify, but to be able to trace back content that has been generated at some point.
Speaker 3:So there's different uses for watermarks here. The particular one that we think about very broadly is because watermarks are usually thought of something that's not perceivable for the user, But a watermark like a cartoon avatar would be something that serves the purpose of informing the user that they're interacting with something that is not the real person.
Speaker 4:Exactly. So if you think about what is the harm you're trying to avoid, if you're trying to avoid them being deceived into thinking that they're interacting with the real Catherine and Dan, then making them more of a cartoon can be a perceptual cue that this isn't the real person. This is the AI generated version of the person. But if what you wanna know is where did something come from, like, text that you just published, is that text that came from your fingers working on a keyboard to produce those cause those things to be written? Or did you type your fingers on a keyboard into ChatGPT or some other model that then produced the paper that then you cut and paste?
Speaker 4:If the idea is there's some watermark there that tells us what the provenance was, then that might not be perceptible to the individual, even though that is trying to avoid a certain sort of deception, I suppose, like, oh, you passing off something that isn't your own as your own.
Speaker 2:If we could take a step back for a second, the term SEII, did you coin that? Was that a preexisting phrase? So can you talk a little bit about what that covers and what that doesn't? I think that's that's good, knowledge that that makes sense. I think not all AI voices fall under that category.
Speaker 2:Explain that nuance.
Speaker 3:Yeah. So when we started writing the paper, we were talking about this mass project where we were actually thinking about systems that emulate identifiable individuals as personas, in quotation marks, of the person. That actually has a very different connotation in human computer interaction, where personas is this kind of categorical representation of character traits and so on. I'm not an HCI person, so I'm probably also misrepresenting that, and the full breadth of work that's being done in that domain. We actually moved away from another title, and decided to use this acronym to be very specific about what we mean.
Speaker 3:I think that the watermarking cartoon by means of cartoon appearance is a good example here, because SEIIs don't focus on the art but they focus on the systems that produce the artifact. There's been a lot of really valuable discussion on deepfakes, but those are artifacts of SEIIs. For example, a cartoon method of watermarking could become a harmful deepfake if it cartoonifies a person in a way that they would not endorse. This is a way in which we have to think about dual use, soliciting the types of values that the person being emulated actually endorses. In the paper, we do walk through a few different ways in which we think about the kinds of technologies that could generate these simulation artifacts, We think about them as having both a modality and a type of dynamism.
Speaker 3:An emulation can be visual, you can have an image of something, it can be voice, like we talked about the voice emulation, and it could also be behavioural. If I see my sister walking down the street, I can tell that it's her just by looking at her gait. That's actually a way that you can emulate somebody as well. And of course, then you can have combinations of these. In terms of dynamism, you can think about static emulations, as images, for example.
Speaker 3:You can have libminen interactions, and the example that we use is car directions when you're driving. You might have some set of predetermined responses, but the way that they're put together is dynamic based on your route. Then you can have fully dynamic interactions, like if Catherine and Dan would run a bunch of interviews, then presumably those would be fully dynamic and full use of modalities here. Actually, this SEI AI concept encompasses quite a lot, but it does differentiate itself from two other categories, which we think of as generically anthropomorphic AI. For example, chat GPT or claw has a particular persona, but it is not attached to a living or a deceased human.
Speaker 3:It's actually something that's very much in the process of being developed all the time, and we'll see where that gets.
Speaker 2:However, if I could jump in on that, what happens if you took all of your Gmail or whatever email of the last decade, all of our teams chats, Catherine and our team chats, so that convey personality and you drop it into one of those LLMs, wouldn't it be able to mimic you in a written form?
Speaker 3:Yeah. That would be a way that you are emulated by these systems. How, say, Claude or Chateapiti would actually treat that, I'm not exactly sure. But the way that you can think about it is that an LLM that you are using as a user has a predefined personality in some sense, and this is what we think of how chat GPT generally sounds and how Claude generally sounds. If you just feed it a bunch of context that is provided from your emails, for example, it'll probably be able to emulate you in a good way.
Speaker 3:Then there's a difference of what if we retrain a model based on your emails? Probably that would be something that's more persistently able to represent you as an emulation, for example.
Speaker 1:So for people working in research ethics, where do you see these questions becoming urgent or something they should be considering when it's recruitment or informed consent? I know you're talking about cutting edge stuff that we're not quite there yet, but it's moving so fast.
Speaker 2:The dementia or cognitive decline scenario, since that's involving patients, wouldn't that need to go through an IRB at some point? And if so, what questions should people want an IRB? What should they be thinking about?
Speaker 4:That's ambiguous. Should there be research about the efficacy of these things, which would have to be evaluated, which would have to go before an IRB? Yeah. I think that would be a really good thing. Do systems like this, can they be developed and deployed without ever going before an IRB?
Speaker 4:Absolutely. Because you only go before an IRB if you're doing research and if you're doing research with human subjects. So most products that are out there in computer world are developed and deployed without ever having to go before an IRB. They create a new motherboard. They create a new chip design.
Speaker 4:But lots of the choices that they make about how to make that piece of technology are their decisions, and they don't go before an IRB. So IRBs are a really limited instrument. They really are there and have their effects when people are saying, I am explicitly doing research. I wanna study how this thing works, and I'm involving either the private data, private information, or I'm interacting with a living human person. And there's lots of parts of the world where people can innovate without actually evaluating what they're doing, and IRBs basically have nothing to say about that.
Speaker 4:That's part of the the problem that we talked about in our previous podcast as an advertisement for current for recurring listeners.
Speaker 1:Absolutely. So this is something that it would behoove them to have in place that could review some of this because, otherwise, we're just dealing with the fallout afterwards. I often think about, are we losing some of our interactions with our humanity with each other by inserting technology that would assist us because we can't maybe be with mom and dad in the middle of the night? Is it removing that moment where maybe we should? I don't know.
Speaker 1:I spiraled out with Dan earlier.
Speaker 4:So there's a ton of issues here, right? One very human one that you mentioned, which is am I doing something wrong if I'm not there in the house with my parent and I'm using a piece of technology to try to help them? And, you know, I think that there are lots of cases where it would be great if you could be there with your parent, if we organized our lives in ways where there were villages that were there ready
Speaker 2:to
Speaker 4:take care of everybody as they age. Many of us don't live in that environment. And I think there are ways in which we can use technology. I think my own parents use technology, their wearables,
Speaker 3:their
Speaker 4:phone, to try to make their own life better and safer as they get older. And so I'm not someone who thinks it's intrinsically wrong to try to use technologies these ways. On the other hand, part of the rationale for working on this paper is that there are lots of ways that the underlying technology is being used in that are deceptive, that are that manipulate people, that are harmful. And there is a vast literature of papers about the moral problems with disinformation, misinformation, deepfakes. And part of our concern though is deepfake is already a moralized term for a particular artifact, and that artifact gets produced by something.
Speaker 4:You can make a deepfake by hand, right, with a dark room and Free time. But in free time. Yeah. AI is making it possible for you to make these things quickly at scale. And the same technology that makes these bad things is the same technology that you would use to make good things.
Speaker 4:And part of our concern was the underlying technology is really being developed without any sense of here are the ways that you could use this for good, and here are ways where we're taking very concrete steps to push you in that direction, develop it in that direction. The easiest way to use some of these technologies that emulate identifiable individuals right out of the box is for things that are bad. And so that's a point of the paper and a way to say that's a flaw in the development process. It can happen because maybe all you care about is money and you want to get your product out there. But it also could happen because you don't have a clear enough sense about what would I need to do and actually to do better and I'm motivated to do better.
Speaker 4:And then regulators could also use this as a way of saying, look, what are the ways in which you have taken very clear steps to make sure that your product is going to be used in ways that are beneficial and making it harder for it to be used in ways that are not beneficial? And we've seen recent public conversations where powerful people and organizations have been pretty clear that they are okay with their product being used in ways that are deleterious to and harmful to people. So in that case, a new moral framework isn't gonna do anything to to solve that problem. There there you need public outrage, law, activism.
Speaker 3:Mhmm. I think also one clear takeaway is just the fact that you even recognize that this is what types of social relationships that the technology enables. And so then recognizing, okay, what are the potential stakeholders to this emulation? We think of it as a triadic relationship where each person or individual in this relationship. So the user, the deployer, and the individual being emulated, each one of those could also be collectives or it could be yeah.
Speaker 3:Consist of multiple stakeholders. Like, even recognizing that your technology enables social relationships of this kind, I think, is important because in the example when you just put a Chateappity in your mailbox or something like that, that the way that we agree with the terms and conditions of using Chateappity does not take that into consideration. Consideration. So maybe this is if you're just trying it out by yourself, it's not really high stakes, but in higher stakes applications, we should really recognize that this is a function that's being served. And then how what are the kinds of consent procedures that we need to add to our product offering in order to ensure that we have the kinds of uses that we want and intend.
Speaker 4:So once you see this idea, you start to see it lots of places. So the example that you gave of what if I want to train a model on my email? And then it's like, what do you want the model to do? If what you want the model to do is answer your email and you don't want it to have a little thing at the bottom that says, hey, by the way, this email is coming from my AI and not me. If you want it to appear that you're just answering your email, then you too have a system that's emulating you.
Speaker 4:In this case, you would have been the one that initiated that because there's something that you wanted to do. And your mediating was just over email. And you want it to sound like you because you don't want the people getting the email to say, why didn't they fob me off onto their AI? Why didn't they answer me themselves? Is there some deception that's involved there?
Speaker 4:When you email actual people, how often do you know that they're emailing you back and it's not their research assistant? There are people who have assistants who run their social media account, run their email account, stuff like that. There was a recent news article about a high profile business school where professors were creating virtual digital twins of themselves so that their students could interact with them 20 fourseven or something like this. There are people who want to emulate themselves and have that emulation interact with other people. Presumably, there are other people who don't mind that.
Speaker 4:So that's fine. What we're looking at is where's the surface of cases where it's okay? You might like some of them, you might not like some of them. And where's the surface then where not okay and you wind up in deception or manipulation?
Speaker 2:Well, with that in mind, going forward, are there practical, clear guidelines that could either be self enforced by the community, or maybe if there's an app being deployed, that it won't be posted to the to those platforms unless it hits some thresholds where it is likely not to fall on your deception, domination, unjustified internalism scale. So then it would keep out some some applications that may not fit the bill. Or is it something that's more heavy handed like federal law or federal regulations? I don't know what you see as the path forward.
Speaker 3:I think there's a balance to strike here because I think a problem with these technologies is that some of them are really general purpose. And in that context, there is a debate in terms of regulation and policy, for example, to regulate more strictly data that's being fed into these systems. So for example, if you can retrain or what kind of context that you can provide and the kinds of uses, it's much harder to control. And so probably you will have to pull other levers.
Speaker 4:While we were writing the paper, one of the use cases that I thought was very powerful So if you go to a museum and you have survivors of the Holocaust or something similar, and you could watch a video of people telling their story. If you had questions that you wanted to ask, then you can interact with them. Can ask them questions and they would respond. There's a situation there where if you have somebody contemporaneously where they're like, yes, I'm willing I'm willing to be an interactive you know, have an interactive avatar that would that would still be me answering people's questions. Then there's a limit, though.
Speaker 4:Right? There's a guiding purpose that they endorse, that the developer endorses, that you want the user to endorse. The technical and ethical challenge is to make sure that system then lives on that surface where it's fulfilling the role, the legitimate role of the museum. It's communicating truthfully what the person's life experience was, and it's not just a carte blanche then to have that person represented in whatever way that you want. So there are legitimate use cases where you might say, yeah, I would like that.
Speaker 4:Right? Especially when you, the person, if you are injured or have a disease such that you would like to have your interactions with other people mediated by something that maintains your voice or your way of speaking or your facial features or things like that. So you you're both the person being emulated and the user of the system very immediately. There's a bunch of use cases of where these things are assisting various parties. It's just that it's much more difficult to identify them and then use this technology to advance those goals.
Speaker 4:Whereas right out of the box, it's very easy to have these technologies advance manipulative, deceptive, all kinds of bad goals. There's literatures on a bunch of these different things, but it was disconnected. And this idea of SEII, of a system that emulates an identifiable individual, was a way to say that underlying technology appears in these different cases. Right? The museum, the wandering example, even deepfakes.
Speaker 4:These are all instances of an underlying technology, and some of them are bad, some of them are good. So being able to get a framework to then say, how do we distinguish which ones are the good ones and which ones are the bad ones?
Speaker 3:One aspect in terms of developing these systems going forward is that we maybe think of a consent process or an alignment process as like a one shot kind of thing. But I think the example you bring up, Alex, of AAC, augmentative and alternative communication mechanisms, our technologies, is a really good one because there, for example, if you cannot produce speech anymore, but you have recordings of your own voice, it can produce something that sounds like you if you type it in, for example. Presumably, that kind of use case seems really assistive for this individual, but probably in part because that process of querying preferences of this tight control over when that emulation is used is by you have this iterative loop of defining the use case. I think that's something we want to be thinking more when these technologies are being developed, how we can develop preference querying mechanisms, data control mechanisms that aren't just relying on a one shot consent, but something that you can control over time.
Speaker 1:We could absolutely talk about this all day. I as an end user, products are going to think heavily. Every time one of these new technologies comes out, maybe that's the takeaway that we all need to pay attention and ask the hard questions no matter what role we have in society. Thank you so much. This was absolutely fascinating.
Speaker 1:And if you can read the paper, take the time and read the paper. It was amazing. Mhmm. Good read. Thank you.
Speaker 4:Thank you for having us.
Speaker 3:Thank you so much.