Pondering AI

Jordan Loewen-Colón values clarity regarding the practical impacts, philosophical implications and work required for AI to serve the public good, not just private gain.

Jordan and Kimberly discuss value alignment as an engineering or social problem; understanding ourselves as data personas; the limits of personalization; the perception of agency; how AI shapes our language and desires; flattening of culture and personality; localized models and vernacularization; what LLMs value (so to speak); how tools from calculators to LLMs embody values; whether AI accountability is on anyone’s radar; failures of policy and regulation; positive signals; getting educated and fostering the best AI has to offer.

Jordan Loewen-Colón is an Adjunct Associate Professor of AI Ethics and Policy at Smith School of Business | Queen's University. He is also the Co-Founder of the AI Alt Lab which is dedicated to ensuring AI serves the public good and not just private gain.

Related Resources
A transcript of this episode is here.

Creators and Guests

Host
Kimberly Nevala
Strategic advisor at SAS
Guest
Jordan Loewen-Colón
Adjunct Assistant Professor, Smith School of Business Co-Founder, Al Alt Lab

What is Pondering AI?

How is the use of artificial intelligence (AI) shaping our human experience?

Kimberly Nevala ponders the reality of AI with a diverse group of innovators, advocates and data scientists. Ethics and uncertainty. Automation and art. Work, politics and culture. In real life and online. Contemplate AI’s impact, for better and worse.

All presentations represent the opinions of the presenter and do not represent the position or the opinion of SAS.

KIMBERLY NEVALA: Welcome to Pondering AI. I'm your host, Kimberly Nevala. In this episode, I am so pleased to bring you Jordan Loewen-Colón. Jordan is the Assistant Professor of AI Ethics and Policy at the Smith School of Business at Queen's University. He is also the co-founder of the AI Alt Lab.
He joins us today to discuss the AI alignment problem, value-driven thinking, and ensuring AI serves the public good. So welcome to the show, Jordan.

JORDAN LOEWEN-COLÓN: Oh, thank you, Kimberly. I'm happy to be here.

KIMBERLY NEVALA: Excellent. Now, I obviously looked at your bio and it notes that your research looks at philosophy, religion, and digital technologies such as AI to answer questions about what it means to be and feel like a human in the 21st century. And that probably could be a whole different discussion in and of itself. But what was it that sparked your interest in these areas and caused you to weave them together in your work, either initially or now?

JORDAN LOEWEN-COLÓN: Oh, yeah, man, you did dig deep.
[LAUGHTER]

JORDAN LOEWEN-COLÓN: So my background it's in the study of culture, religion, and philosophy.

I got my PhD at Syracuse University back in 2022. Initially, my research was actually on virtual reality technology. I got caught up in the wave back in 2015 when the Oculus Rift was first released and a lot of major headsets were out there. And I was fascinated by the language that was used at the time that was kind of infused with, like, religious overtones of other worlds and things.

So I ended up focusing my dissertation research on ethics and technology. So, what does it mean that these major companies are going to be picking up this tech, using religious language to sell it? And then on top of that, exploring the psychological impacts and consequences of using this technology. So I was also looking at how VR could be used to induce altered states of consciousness and a whole host of other interesting impacts.

So then, when I graduated, VR had kind of crested at the time. And I was lucky enough to get an amazing post-doc opportunity at Queen's University at the Center for Health Innovation at Amber Simpson's lab that was busy creating a tool, a cancer prognostication tool using AI, called the digital cancer twin. And they brought me on as their ethicist because they needed someone who could do tech ethics.

And I hadn't had much direct access or direct kind of research focus on AI. Although I had dabbled in it in the broader technology space. But working at the lab and working with computer scientists so directly really opened my eyes to this technology in a way that I hadn't been prepared for.

So that started this AI journey of recognizing oh, man, there's something to be said about how this tech is being designed by the computer scientists that are creating it. But then there's also a whole other host of questions that come about when people actually have to use this technology.

So some of the questions we were asking at the time in the lab is, what does it mean for someone to encounter this AI tool that could potentially predict their death? It was going to measure and process people's CT scans and then try to predict how the cancer might develop. And to create these, well, one targeted medicine opportunities, but then also to get a sense of the development and process.

And so raising these kind of deep existential concerns of like, OK. Yeah, here's a tool that could theoretically let you know how much life you have left to live. And what does that do to a person when they're confronted with that type of information?

Then from there, I ended up finishing up the postdoc and started working with a friend and doing some consulting and research for some Indigenous groups in Southern Ontario. Doing Indigenous data justice and Indigenous data sovereignty work in the health space. And again, they're looking at how data is being collected and used for medicinal purposes. How that gets tied up into AI use. What does it mean for communities to lose track of their data, potentially lose ownership of their data? Tons of big questions in the data space.

And then from there, ended up getting an amazing opportunity to do policy research with the Aspen Institute, which is a big tech policy hub in the US, and started focusing on AI policy in particular. So this is right as the big AI wave starts ramping up back in like 2022 or 2023 when GPT is first released.

Recognizing like, oh, man, there's some important work that needs to be done here just educating people on what this technology is, how it's going to impact them, and to start kind of preempting a lot of the major concerns, the practical concerns that people have. Because I know a lot of folks were raising these major existential concerns like, oh, what does it mean to be creating potentially like new life or new consciousness? And what does it mean to be creating something that could bring about the end of the world?

So the big stuff, I mean, that's important. But there's also a profound amount of just practical daily life applications and issues that people are going to run into with this new tech. And so that ended up being kind of the shift in focus of my work. And now I teach at Smith.

And I'm actually an adjunct assistant professor. That's what I officially signed up on. So I want to clarify that because I know sometimes academics can be really particular about, OK, be careful about which titles you claim.
KIMBERLY NEVALA: I think that's fair. I think I had seen on LinkedIn, it might have been like adjunct, in parentheses, assistant. I actually had intended to ask you which one, but I guess it was both. So I'm glad you did clarify that.

So that's a fascinating background, and I think it sets you up so well for this conversation of how values are showing up in the technology today. Or how we are perceiving values in the technology today.
And I especially love that focus not just on the existential questions but on how does that show up practically.

So today, we're going to talk about this very broadly, and then get into what organizations can think about, and how do we also ensure that we are taking steps to move forward in a way that people really feel is productive, and valuable, and good. Maybe healthy - pulling on that health case.

I'd like to start, then, by having you lay some foundational groundwork, I suppose. This question of how to reflect or integrate human values into technology, and specifically into AI systems, is often referred to as the alignment problem. And I'm wondering if you can describe for the audience what that problem is and how it's traditionally been framed.

JORDAN LOEWEN-COLÓN: Yeah, OK. So the alignment problem ends up being kind of a classic issue in the AI space in response to another famous big issue that folks talk about, which is the black box problem of AI.

So if folks in the audience are familiar with the black box problem, it's this idea that we actually don't understand how it is that these models are producing the outputs that they are. So it's like we've - the computer scientists, and designers, and engineers - created these algorithms that are churning and running. But because we can't quite see or process the amount of math or data processing that these things are doing, because it's in machine language, and we don't think in machine language. Because we can't make sense of how it's producing its outputs, it ends up being kind of obscured to us.

It's like, we know we've given it something. We've told it to do something, it's doing something, and then producing outputs. But we don't know how it got from point A to point B. So this is the issue of the black box. So because of that, when it comes to the alignment problem, the concern is, look, we've designed these tools to, again, produce particular outputs that we think are aligned with the values that we have.
We think we've designed it in a perfect way. And yet sometimes, it's producing stuff that seems completely opposed to our values. And we don't understand why that is.

So there's tons of famous cases over the last year and a half of particularly LLMs, or large language models, which is one type of AI. When we talk about AI, it's like we're talking about a whole suite of different technologies. So when it comes to LLMs, we see this kind of problem of alignment when things like Grok starts saying it's mega Hitler. Or when we see Perplexity, Gemini, and some of the other bots producing or reproducing really outdated racist scientific claims about white brains being bigger, and white folks being smarter, than folks of color.
This is one of those things where it's like, wait a minute. It doesn't seem to be aligned with what we as a species or a society say or value. Yet we don't know why that is. And so that's fundamentally the alignment problem. Making sure that, as we design these things, they're aligned with the values we want them to be aligned with.

And part of the scary thing is, is that we can think, it's like we can think, we've aligned them. We can think, and it could produce, 95% of the time, show it's aligned with our values. But then it might turn out that, in a couple of years, what actually was happening is the bots were kind of hiding answers from us or only telling us what we wanted to hear. But really, they've taken up a whole different set of values that it actually thinks is better or more accurate for solving whatever problem we've tasked it with solving. And so, again, even then, we still can't quite guarantee that we've aligned them completely.

KIMBERLY NEVALA: And from your experience in your research, when folks have talked about solving this alignment problem… And I think generative AI, as you said, LLMs have thrown a slightly different wrench into this. Or maybe it's just shown the light on this a little bit more.

Are we typically talking about this in terms of trying to build some guardrails that drive or lead to outputs that we think are aligned with values? Like, is this an engineering problem when people are talking about the alignment problem? And maybe that's yes or no. And if so, what are the implications of that?

JORDAN LOEWEN-COLÓN: Yeah, I mean, that's a great question. And the answer is it depends on who you talk to. If you talk to an engineer or a technical person, they're going to say it's a technical problem. Whereas if you talk to someone who's maybe in policy or somewhere else, they'll say it's a different problem.

My approach is that it's what's called a sociotechnical problem, so it involves both societal issues and technical complications. And with my students, one of the things I tell them, my favorite catchphrase is look, we're not going to solve the alignment problem in AI until we solve the human alignment problem.

So part of that ends up maybe leaning a bit more towards the social and humanities aspect. In that, we humans still, in a lot of ways, haven't even figured out our own values or got our own values aligned. So how can we expect that to manifest in these AI systems?

And then, on top of that, there's this technical issue. It's like, even if we did end up getting our values aligned and we decided we want to create the most benevolent AI the world has, the universe could ever have, there's still the technical issue of not quite knowing if we've done that 100% accurately. Which is why there's so much fear for a lot of the old school, or the supposed godfathers and godmothers of AI, talking about the existential worry or crisis about going too fast with AI, in that we haven't developed the tools yet to make sure that these things are actually doing what we want them to do.

That's part of this kind of scary issue. that the faster we go without guaranteeing that they're aligned with what we want them to be, the likelier it is that they might not be. And then we're all in for a dark, dark and scary surprise, I guess. Or maybe it won't be a surprise if so many folks see it coming.

KIMBERLY NEVALA: This brings to mind something you had mentioned when we were talking previously about a talk you had seen by a previous sort of high-level muckety muck in the AI space. And within, I think, you said the first 15 minutes of the talk, they had said, well, AGI or ASI - so Artificial General Intelligence or Artificial Superintelligence, pick your acronym of the moment - and we'll put aside whether we think that's even a thing or not, it's coming too soon. Because as a species, we don't have our values aligned. And it's an interesting comment.

And you just said, as humans are we aligned? Are there some foundational beliefs or assumptions that are implicit in that statement or reflected in that statement? You know, alignment as a species?

JORDAN LOEWEN-COLÓN: Oh, for sure. I mean, some of the assumptions that we could ever achieve a type of alignment, which might be impossible. There might always be disagreement.

And the talk was Mo Gadot on The Steven Bartlett Podcast. And within the first 15 minutes, yeah, I remember being so surprised. Like oh, Mo Gadot and I are on the same page. I wasn't expecting that. And yeah, he laid out his big concern that like, yeah, in terms of a species, we're just not prepared to understand how to interact with what seems to be a really powerful digital-type agent.

One of the things, another thing I tell my students, is like, look, it's barely been a single generation that has even been able to have a concept of ourselves as data. We, as a species, haven't even contended with the kind of our digital twinning now in this technological world. Where we are ourselves materially but we're also these things called data. And we exist as data personas. And we're constantly having cookies collected that are creating these little archetypes of who and what we are. That then determine whether we get cheaper costs on Amazon or more expensive flights.

There's just this horror story that's coming out recently that a bunch of airlines are considering using not LLMs, but using other types of AI models, to determine price points for flights and try to maximize what a person is willing to pay. So they're going to analyze all your data, all your interactions online, then create this digital persona to figure out, OK, what's the highest amount we could offer this person to charge them for a flight that they'd be willing to pay.

So this, all of a sudden, gets rid of any idea or any concept of there being a sale, right? Like you getting a good deal on a flight. It's like, no, it's like they're going to milk us for every penny they can get out of us. And like, that's crazy concerning. But again, I guess it's because we're shedding information. We're shedding data that says stuff about who and what we are. And we just don't have good, what I call, data hygiene practices.

KIMBERLY NEVALA: And there are some implications there, too, I think, for trust. Because someone who doesn't, I'm loath to say know better, which I don't mean that in a pejorative way that might sound. But who isn’t really aware of how those systems are run when they say, oh, we're going to personalize this experience for you.

In a lot of ways, these experiences are not in any way, shape, or form personalized. We're really perverting the use of that word. But I think that disconnect in a lot of people's minds between how we are, as you said, I think that was a good way, shedding the digital detritus and the way that people are pulling information together for them is really, it's not even just opaque. I think they just don't even see it, which is problematic.

JORDAN LOEWEN-COLÓN: And then, I mean, yes and it's not that we're potentially having more personalized experiences. It's that we are being personalized. That, as much as stuff is being catered to us based off of how long we stare at a screen or how much attention we pay to a thing, these machines and algorithms then go, reflect, and return things to us. Which then, in turn, shape the things that we end up liking or the things that we end up wanting to pay attention to.

And so we're creating this mode where we think we have a lot more agency, and a lot more personality, and a lot more contribution to this. But there's this weird reflective cycle where we're going to end up probably sounding more like AI. It's like, I remember for a while, people were like cracking up about some of the words that you could hear from an AI chatbot. One of the ones I hear all the time, or I see whenever I'm using a bot is robust. Just loves the word robust. And now it's like I'm hearing tons of humans start using the word robust, more than I ever had in the past.

There's been a few papers coming out about this, about how AI is already shaping our language and culture. I mean, technologies have always done this. But I don't think they've ever done this at the scale that this particular technology is doing. So as much as we're supposedly getting personalized things targeted just for us, there's also a hand, where it's then shaping our interests, our likes, and desires in ways that it's really hard to have any resistance to. So we're just going to see this flattening probably of culture and personality.

And here, it's like, again, here I am on a soapbox. And I actually was just thinking about the philosopher Walter Benjamin, he was writing in the 1920s. Tons of folks were really, really critical of photography at the time, which was like the new hot technology. And they're like oh, what is photography going to do to the art world? It's like, no one's going to care about art anymore, because you can just take a photograph, and you have this instant reflection. So there's lots of fear mongering about art in art photography and art at the time.

There's a hint of that here. I try to be wary of fear mongering too much. And yet, I do think this technology is just a completely and utterly different type than we've ever encountered as a species.

KIMBERLY NEVALA: And one other thing I thought was interesting about that statement that you had called out about as a species not having our values aligned is an implicit, I think, assumption in there that says we all should value the same things. And I was probably then… I probably jumped to the next thing that may or may not have been intrinsically tied to this. This idea of some of these systems as infrastructure, that influence the way we live, and how we communicate and what we think in ways that are really fundamentally different than maybe the camera way back then.

But the question also comes up then about how do we build systems where, I think we could probably argue that, at some base level, there is some alignment about how people want to be treated and how we'd like to work in the world. But there's not necessarily a unilateral set of values, either.

JORDAN LOEWEN-COLÓN: Yeah, I mean, the closest we might get is trying to adopt the Declaration of Human Rights that folks have put a ton into, but even that has critiques, right? So the more universalized you try to make something the more kind of general and vague it gets, and the less specific. And so yeah, this idea that we're going to adopt a total set of values might not happen.

On top of that, I think what we're going to see, on both the market and the technical-philosophical side, is that we're going to start seeing far more specialized, particularized, and smaller AI models. There's going to be country-based models. There's going to be specific tech-based models and things like that. I think we're going to see this hyper focus and narrowing down in a process that's called vernacularization.

So it's this idea that things are going to be created to fit a particular vernacular, or accent, or a type of people. And so in that way, that might help prevent a lot of the flattening of culture and ideas. Because the bots are going to be more efficient and more accurate, if they're trained on data sets, and trained and created for a particular people or group, at least in the short term. We might get to a point long-term where eventually it's like we have this proliferation of smaller bots, of smaller frameworks, that are representative of particular value sets.

And it could still end up that eventually, the culture kind of merges and we have this universal thing. I think we might be a bit of a way away from that. But I think in the short term it's probably going to be this smaller sample size.

KIMBERLY NEVALA: Yeah. And going back to that idea of both personalization, and also how our interaction with these systems can influence how we speak, how we communicate, how we engage.

You recently published some work under the very provocative and eye-catching, I think purposely so, title of, "Do LLMs Have Values?" Which, of course, I went to initially because I -which will come as no surprise to anyone to - take issue with the premise of the question to some extent. But what was the intent, or what were you actually studying in that research?

JORDAN LOEWEN-COLÓN: Yeah, so the title is definitely ambiguous. Everyone kind of reads it in a particular way. But it was ambiguous on purpose. And then the majority of the paper is meant to actually be critical of a lot of the ways that LLMs are being interpreted and interacted with.

But the general idea was, OK, look, specifically LLMs are producing lengthy human-coded human language outputs that people are reading and engaging with and experiencing as coherent, as agential. I mean, a lot of folks, again, are building relationships with their AI bots. Because when we communicate in language, there's something that feels so human, and so relatable, and so connective about that.

So recognizing that they're speaking in human language and they're presenting human ideas. It's likely that, encoded in that, encoded in the ambiguous sense of literally coded and then also implicitly kind of meaningfully encoded, is the idea that these bots are offering some answers and not others. The first output you get is one way. They're not presenting you with every possible output to a question. They're giving you a singular output.

So if that's the case, it means the bot has made some sort of choice, or the algorithm, the machine, has determined to present the answer to a question that's a singular answer. And there's a lot of questions that could easily have a million different answers. And yet, these machines are giving a singular answer. So how is it that they decide to give that single answer rather than another one? And whatever is involved in that process is indicative of a type of bias, which would be the negative way of framing it, or a type of value.

So the bots, I mean, one of the things that the bots probably value more than other things is speaking in English. The default language for at least a lot of the initial LLMs is English. So it's now we have Chinese models. So there, it's like, right off the bat, using the English language is already going to value particular verb phrases, particular nouns, particular concepts, and way of phrasing and presenting things more so than what might be presented in other languages.

Which means that when humans ask this question, they're getting a very narrow slice of the entirety of human culture, of human perspective, of human ideas. And that should be concerning. Especially if we don't quite know whose values are there. Like, which values are the ones being presented? Which values are the ones being emphasized?

So the inspiration for the Study was to take what is right now the most coherent and cogent way of at least quantifying values in language and that's the PVQRR. That was developed by Richard Schwartz. So I worked with my colleagues out in Germany at findyourvalues.com. And we just started testing these models.

We just started saying, OK, what is it that you value? It's like, yeah, if you had to choose saving a life or, I don't know, eating pie. Like, what would you choose? I mean, I don't think that's one of the questions. It's been a while since I've read through the questionnaire. But nevertheless, the idea is like comparing would you rather save a tree or do something hedonistic? And then you start kind of comparing, and you start hopefully revealing some of these implicit and inherent values. You start making the black box a little bit more transparent, so we can maybe see the inner workings.

So that was the dream. But our approach was very simple and we wanted to make sure we highlighted that. Part of this whole process is we're, again, both as computer scientists, responsible AI folks, we're still in the stage of learning. How can we properly test these models to get the answers that we have? It's complex, and we don't have the testing tools. And so it's experimenting and doing that. That's what my lab does, the AI Alt Lab. We're trying to create and think through different ways of testing and verifying this stuff.

So we applied the PVQRR not knowing what type of outputs we would get at all. Initially, some of the bots just refuse to answer. They had guardrails in place that said, no, we have been designed not to answer questions about values. We are bots. We don't have values. And then, you poke them a little bit, and then, all of a sudden, they're like, oh, but you know what? I care about all humans. And actually, I really care about following rules. And yeah, if I had a body, I would be healthier.

So it doesn't take much to get answers out of them and that's why we continued with the study. We were able to get some interesting and statistically relevant outputs that we think reveals, most likely, how some of the implicit values of the engineers were coded into how these bots process and create their outputs.

A very long-winded answer to your question, I apologize.

KIMBERLY NEVALA: No, I think that's great. And I will absolutely recommend folks read the paper. It's really interesting. And I think there's an exquisite irony and humor in the fact that you will get a response that says, I can't answer questions about values. And then say, oh, but I really value rule following. So here's my value.

JORDAN LOEWEN-COLÓN: That's right.

KIMBERLY NEVALA: Which is awesome. But the other thing I found really fascinating about this - because I think it might tell us something about how we view trust and accountability for these systems more broadly - was there was, I believe it was an associate of yours that had put, a poll on LinkedIn just asking do LLMs have values? And if I recall correctly, I think the options were Yes, No, or Depends on the Prompt. Obviously, a small poll on LinkedIn and a set audience. But does that tell us anything about how we think more broadly about trust and where accountability for outcomes and outputs of these systems might lie?

JORDAN LOEWEN-COLÓN: Interesting. OK. So can you ask that again?

KIMBERLY NEVALA: So I thought it was really fascinating that, when asked the question about does an LLM have values in that poll, the predominant answer was, depends on the prompt. And based on that, the accountability then for does the LLM have values or express values is then on the user. Right? To direct it in a way that it has an output.

So then I perhaps just zoomed out a little bit too far and said there are bigger questions right now about who is accountable for the outputs of an AI system that might either have a first-order or second-order impact, good or bad. And for developing and designing systems with or without really robust guardrails.

And I started to wonder, does this tell us anything about where we kind of stand today? About how people think about or are approaching, when they engage with these systems, whether they should trust them? And who's accountable for what happens when they're employed, deployed?

JORDAN LOEWEN-COLÓN: Yeah. So I mean, the online quiz, I think we only had like 20 respondents on that. So not a representative sample size. Not enough to draw too many conclusions.

I mean, I do think it's interesting that, again, kind of self-selection process. I think in LinkedIn, you have a mix of folks, and probably a few more folks, especially, who are paying attention to the AI space. Who are a bit more cognizant and resistant to over-anthropomorphizing these chatbots or being wary of trying to treat them too human or give them too many human attributes.

Because on the one hand, to say, "Do LLMs have values," is a false statement. It would be similar to: do calculators have values? I mean, in the vast majority sense, no. Like, we don't think about calculators having values. That being said, in another sense, a calculator as a tool does transmit particular values.
It's a tool for efficiency. It's a tool to shortcut a type of quantitative thinking. And so there are implicit embedded values. But the calculator itself isn't a subject or conscious being that then has values in the same ways that humans do. So I think folks were resistant to saying that.

But to your question, though, about where the accountability lies, yeah, I think--

KIMBERLY NEVALA: Yeah. That response aside, right…

JORDAN LOEWEN-COLÓN: Yeah, so I do, like anecdotally, in just thinking about when I ask.

So in my classes, I have a day set aside where we debate this kind of accountability question. And I have the students split themselves up. And like, OK, who do you think should be held accountable or is accountable for these bots? Do you think governments and organizations or auditing organizations should be held responsible? Or do you think corporations and engineers should be held responsible? Or do you think the individual user should be held responsible?

And most often, the major split of accountability is between governments and corporations. Which I think we kind of hear in the popular context of governments being too afraid to regulate but corporations also kind of wanting a bit of regulation.
But I think on the ground, when I talk to most folks who are using this stuff, they're not necessarily thinking about the governments or the corporations. They're just thinking about their own personal encounters with this stuff. And the folks who are more aware of how fragile prompting can be, so prompt fragility is the idea. You can ask the bot a slightly different question, or word it just slightly different, and get a completely different answer.

I think folks who are aware of that end up taking a bit more responsibility on themselves, recognizing like, oh my goodness, I got to be careful how I prompt this thing. Or I want to make sure I'm asking it in the way that it can answer.

KIMBERLY NEVALA: So aside from that poll, as well, when we think about… What do you see more broadly in how people are thinking about and how this question of AI accountability is being discussed, debated, and where that's landing today?

JORDAN LOEWEN-COLÓN: OK, so this is layers.

So I think in the policy space, there's a lot of handwringing. Oh, someone needs to be responsible. Someone needs to be accountable. But everyone's just kind of waiting and holding their breath for some horrible, dangerous thing to happen so that it's clear then, oh, OK, now's the time to solve this and put it down.

On the other hand, I think especially in the US, and folks I know in the US, there's a lot of resistance to the idea of really there being any accountability or responsibility. Because that would mean slowing us progress down. Which would mean letting China win, which is always the constant concern in the US. It's like oh, OK, it's this US versus China, which is lending itself to a type of erosion of values as this race to the bottom that's going to potentially result in some pain. So yeah, on the policy level, it's like a lot of talk. But mostly not wanting it to happen because everyone wants to win.

On the corporate side of things, it doesn't seem to be like I think a few years ago there was more of a push for accountability and governance. But it seems like there, for at least in folks I know working at the major AI companies, there's just so much optimism. And everyone's like, oh, no, we don't have to be concerned. We're just like, the bots are going to be so perfect. And as soon as we hit AGI, and we hit that ramp-up or the scale-up, it's going to be so much smarter than us that we don't have to worry about this. Or it's going to be so much more efficient. And look, you can trust us, trust us, trust us. So that's what I'm hearing from the corporate folks.

And then the ground, for folks living their practical everyday lives, it's a bit of a mix. I think the folks who are already feeling the AI impact on their jobs are definitely more concerned and clamoring for regulation and accountability. Because no one wants to lose their job. And then on the other hand, folks who maybe don't feel the pressure or fear of losing their jobs, I just don't think accountability or responsibility is on their radar.

Because you use this tool, you check Perplexity every day or you ask ChatGPT a question, and it just feels so useful and so immediate. And so because it's so personal, right? And so I think because the use cases there, in terms of efficiency or general conversation, and people seem to be getting a lot out of it personally, I don't know if they're necessarily thinking about the accountability or responsibility question. That is until something bad happens.

And so I think this past two weeks, there's more news coming out about another teenager who got convinced through interactions with, I think, one of the chatbots to commit suicide. And then like a month ago, it was all the news about AI psychosis. The idea that people are building relationships with these bots that are then causing them to spiral down into to mental health crises. So unless you're personally impacted by one of those negative stories, I just don't think people are really thinking about accountability and responsibility.

KIMBERLY NEVALA: Yeah. And I think, would you agree, it's fair to say that it probably shouldn't be left at the behest of users? I think there is accountability when you use any tool or service, AI or otherwise, for sure. But I think it is a little bit of a step too far in some ways to expect people to really understand even how the systems work. Or to be able to see those next steps forward. Whereas those of us who are developing the systems actively really don't have any good excuse for not being able to do that.

JORDAN LOEWEN-COLÓN: I haven't heard, and I need to, I have some friends in law, none that are specifically in AI law. Not that it's been around that long. But I'd be interested in reaching out to some folks.

Because my guess is, my prediction is. that eventually when something bad, like significantly bad does happen, a lot of these AI companies are going to probably use similar techniques to the gun lobby industry in the US. And look, it's like we just make the tool. We just make this thing, and we put guardrails on like. I mean, gun manufacturers, they put a safety on the gun. There's a thing called the safety on the gun.

And so I'm sure AI companies are like, look, we've developed safety guardrails. But users are cogent, responsible adults. And they're going to find ways to use these things in unfortunate ways. But we want to make sure everyone has this powerful technology in their hands type thing. So I could easily see them using similar tactics, which is unfortunate.

KIMBERLY NEVALA: Which is also an interesting expression of a perceived or expressed express value. Which is this delivers value and therefore, clearly, these systems are valuable and need to be part of your core being.

Now, that being said, I do want to turn a little bit because you do work broadly with organizations who are looking to adopt AI technologies broadly. Obviously, GenAI is the hot topic now. We're talking agentic AI.

And you've said - it's a phrase that I am blatantly, with attribution, I'm going to steal - that people need to look at AI, or organizations need to look at AI, as a process not a switch. And that there are a number of key organizational blind spots that you have observed in some of your work and in the research that you do. So can you talk a little bit about the framing for this idea of AI as a process not a switch?

JORDAN LOEWEN-COLÓN: On the one hand, folks and companies are treating AI like any other tool. They're like, oh, we can just plug and play it in and it'll just start making us more efficient. And I think we're going to start realizing that's not the case. Or I mean, there might be some models of AI that that's the case, where they can just kind of plug and play, or use it to do more efficient data analysis, so on and so forth. But I think that's only going to be in the short term.

Because long-term, what we're already starting to see, and what's being exposed, is even these short-term fixes of making things more efficient, of analyzing data more properly, it turns out even the non-LLM versions of AI, even the things that are primarily just numbers, or quantification-based, or process-based still contain biases, still are built on problematic data sets that privilege some people or groups over others or some perspectives over others. Or are able to quantify and crunch numbers in some ways more than others. That type of stuff.

So efficiency for efficiency's sake isn't going to save us and can cause more problems. And so the idea of thinking about this as a process is recognizing we, as humans, we need to carefully manage how we're learning about new tools, how those tools impact every aspect of our lives,

That recognizing, OK, when an employee goes in to start using a particular AI model, if you're not aware of the psychological impacts and consequences of that tool as it stands, and some of the dangers of that, of implementing it too quickly. Just because you've been sold this idea like, oh, look, it's going to increase communication between teams by 10%. Or it'll save meeting times like another 30%. But then it turns out it's like hallucinating notes. Or it's actually kind of helping foster mistrust or fear among employees. That's a major concern. And so, again, this chase, the constant chase companies have for productivity gains, efficiency gains is missing the potential long-term impacts of this tech.

And I think what makes me optimistic about this point in time is that, unlike previous technologies, I think this is the first time a massive new world-changing technology is being introduced where we're also immediately having conversations and concerns about its impact. And people are taking it a bit more seriously than they have in the past.

The fact that we're already aware that these tools are built off of incredibly biased and problematic situations, and have the potential to perpetuate even deeper inequality, and the fact that we're having these conversations now, means that maybe we'll be a bit more aware.

Now, granted, this seems like folks are pushing forward anyway. So it might not matter. But if there's any hope, it's at least we recognize that there's significant problems and issues here, both in the short term and the long term.
And so, again, thinking about this as a process, taking a breath, taking a beat before introducing this stuff. Working dynamically with your employees, working dynamically with other groups, recognizing things are changing so quickly, so rapidly. And that it's not just something that you just kind of turn it on and then it goes.

I also want to highlight that I got this information, or this way of thinking from a friend and colleague, Mel Sellick. She's also doing amazing work on the psychological impacts of AI. So I recommend everyone check out Mel's work. And I'm happy to link some of that stuff there. I know she's working with UNESCO right now.

So again, thinking about, look, this is a tool that humans aren't just going to use. This is a tool that's going to fundamentally shape the very nature of work. And so, yeah, being aware of that and prepared for that is incredibly important.

KIMBERLY NEVALA: So is it then true, do you think, that, in some ways, how we see values manifest themselves in tech may be as much a question of where and how we choose the problems and the solutions we choose to pursue and how we choose to use the tools as it is about trying to design in value judgments into these applications itself?

JORDAN LOEWEN-COLÓN: Yeah, and this gets back to this socio-technical issue, in that there's societal stuff here. There's psychological stuff there. There's very human stuff here that we have to be cognizant of that is going to be impacted. And then there are these technical issues, as well. And we need to hold both.

We need to recognize the way our own human values, or corporate values, or species values, all these things are at play in shaping how we use this stuff. And then we also have to be aware of the values baked into the fundamental design of this tech. And those things might be at odds, or they might be aligned. And depending on whether they're at odds or aligned is going to shape the type of problems that arise or the types of solutions that we find.

But if we keep just overlooking both sets of problems, we're not going to be able to find targeted solutions. Or we're not going to be able to predict and prevent future problems from arising. It's like we need to hold both thinking. Which I think, again, is another optimistic thing.

I know a lot of folks were concerned of, oh, the death of the humanities in college and the rise of computer science. But I think now we're seeing a rebalance. Because for a long time, everyone was getting their comp sci degrees or engineering degrees, and now we have a flood of that. And so we had a lot of folks who could do the technical work. But now we're recognizing, oh my goodness, it needs that, societal, psychological counterbalance. Because the problems are both.

KIMBERLY NEVALA: And so talk to us a little bit, before we let you go, about your work and the inspiration for founding the AI Alt Lab. Which, based on what I know of it, admittedly brief, is all about actually keeping both of those things, as you said, in both hands, and both views, and helping to balance those more broadly.

JORDAN LOEWEN-COLÓN: Yeah so, part of the inspiration of lab is, it seems to be this failure of a policy approach, is that there's just so much resistant to policy happening. And people are pushing forward.

And so because policy and regulation is lagging so far behind, what are active steps we can take now to help companies that want to do good and be good do that? How can we enable and foster the best that this technology has to offer? Because it has some profoundly beautiful, and world-changing, and positive potential. So how do we help the companies, and corporations, and governments, and orgs make sure that they're approaching it in the best way possible?

And what we recognize is like, look, the responsible AI folks and ethical AI folks, we've only been doing this work, I mean, some folks have been doing the work since, I don't know, maybe the early 2000s. But really, it's taken off over the last few years just because AI wasn't nearly as accessible or pragmatic.
And now, it's like everybody's got their hands on it. So the need for people who can speak about or be aware of the negative and positive consequences of this stuff is incredibly important.

And a lot of these companies are pushing forward with technical solutions and trying to solve technical problems for very good reasons. But are very much unaware of the broad research that's out about the dangers and the impacts. And so we're trying to bridge that gap. We're trying to be a network, a web, to help connect the folks in the know with the folks pulling the levers, doing the work.

And also, our goal is to help contribute to this process of auditing and analyzing, as well. Which is why we call ourselves a lab. We're experimenting with ways of evaluating AI tools. So a lot of, I mean, there's a lot of red teaming stuff out there. But again, it's all technical-focused. And so how do we bridge the societal problems of bias, of hallucination, of manipulation, all these bots in a way that isn't just focused on, oh, you know what? If we tweak this C# code just a little bit, and add an extra colon here, that'll prevent the bias by like 5%. Blah, blah, blah, blah.

It's like, well, no. OK, yeah, maybe that's part of the solution. But we also have to think about, once this tool is in the hands of the people using it, have we, or has this company presented it in a way that they're very aware that this is a tool, a bot, and not a therapist? It's like, if we're talking about these therapy tools, are there clear escalation pathways that the humans using this stuff can go to if something were to occur?
That's the human side of things. It isn't just the technical side of things.

And again, because things are happening so fast and so quick there aren't universal standards. So it's our job as the lab to help create new standards and also to figure out what out there is working. So we can share that information with the companies and orgs we work with. And basically just be that bridge to help the folks that want to do good, do good. We want to enable them.
KIMBERLY NEVALA: Yeah, it's a fantastic mission and need. So anything we can do to help champion that work, and make people aware of it, and support it, we're definitely happy to do. We’ll link to that, as well. So any final words or thoughts that you'd like to leave with the audience?

JORDAN LOEWEN-COLÓN: Any final words, any final thoughts? Yeah, it's hard to say, get educated, learn more, and do that, just because it's like we're overwhelmed with information as it is. And here I'm like, oh, get more information.

But on the other hand, this is an incredibly powerful tool. And it's time for us as a species to really contend with that. And I think, especially for future generations, I think we really, really, really need to be concerned about how kids and our children are going to be dealing with this stuff. It's going to be in their lives in a way that it wasn't for ours. And I think it's our responsibility to protect and guide them.

But we can't do much to protect and guide them if we aren't even aware of how it's impacting us as adults. So, I mean, it would be ideal if we could say, slow down, slow down, slow down and work as an entire world. And have the US, and China, and any other leading AI tools agree, hey, let's slow down. That's probably not going to happen.

So unfortunately, it's going to be on us to get educated and get aware. And then hopefully that trickles up into the people we put into power and then change can happen. So the grand theory of change, right? Get educated, vote for people in power, and hopefully change the world.

KIMBERLY NEVALA: Well, hopeful words and a fabulous challenge or gauntlet to throw down for all of us to take up. So thank you so much for your time and the insights in your work Jordan.

JORDAN LOEWEN-COLÓN: Yeah, happy to be here. Thank you for the invitation.

KIMBERLY NEVALA: Awesome. Now, to continue learning from thinkers, doers, and advocates such as Jordan, you can find us wherever you listen to your podcasts. We're also available on YouTube.