Pondering AI

Leigh Felton believes we can benefit from AI systems without surrendering to them if we carefully consider the environments and behaviors we promote while deploying AI.

Kimberly and Leigh discuss the power of perspective; organizations as people; using AI consciously; the AI environment and library etiquette; how AI use shapes human behavior; whether AI systems are adaptive; AI authority and narrow optimization; the gap between lived experience and historical data; the AI Emperor’s new clothes; quiet organizational failures; and who benefits from humanizing AI.

Leigh Felton is the President of the AI for Job Security Foundation. Leigh applies her extensive experience leading communications, business strategy and operations at the world’s largest tech companies to help organizations make conscious decisions about and with AI. 

Related Resources:
A transcript of this episode is here.

Creators and Guests

Host
Kimberly Nevala
Strategic advisor at SAS
Guest
Leigh Felton
President, Al for Job Security Foundation

What is Pondering AI?

How is the use of artificial intelligence (AI) shaping our human experience?

Kimberly Nevala ponders the reality of AI with a diverse group of innovators, advocates and data scientists. Ethics and uncertainty. Automation and art. Work, politics and culture. In real life and online. Contemplate AI’s impact, for better and worse.

All presentations represent the opinions of the presenter and do not represent the position or the opinion of SAS.

KIMBERLY NEVALA: Welcome to Pondering AI, presented by SAS. I'm your host, Kimberly Nevala.

In this episode, we are pondering the gap between intention, governance, and operational reality with Leigh Felton. Leigh parlays her extensive experience at the forefront of business strategy and operations at some of the world's largest companies to her current work in AI as the co-founder and President of the AI for Job Security Foundation. And she gets full marks from me for absolutely the best organizational name out there going today on any level.

So Leigh, welcome so much to the show.

LEIGH FELTON: Oh, thank you so much for having me, Kimberly. I'm excited about this conversation.

KIMBERLY NEVALA: All right. So I have to start by poking a little bit into your background. Have you always been business minded and what was it that signaled the turn that brought you to your current role?

LEIGH FELTON: Business minded, yes. Technically minded, absolutely not. When I started my career, technology was literally the furthest thing from what I thought I was capable of doing.

So my background is business communications, and I always thought, oh, I'm going to be the next Oprah Winfrey. I'm going to have my own talk show, a TV show, and I'm going to bring amazing content to the world. And I started my career basically at MSNBC. So that was the joint venture between Microsoft and NBC, and I was the online news producer. And I was like, OK, this is in Redmond at Microsoft headquarters, but this is going to lead me to Secaucus, New Jersey, where I am going to be in front of the camera on MSNBC, the actual network.

But instead, I was taught something that I did not even know at the time. My passion is in business communications. My passion is in the translation of technical theories and terms and concepts to various audiences of understanding where are we going and how do you take part of that and how do you get energized by all of this technology that's around us.

So for the longest time, I would start any conversation at Microsoft with, I'm Leigh, I'm the comms person, but I'm not technical. And I stopped that because technical has a full spectrum of things. And so I worked my way up at Microsoft from communications to marketing to strategy. I eventually became one of the first Chiefs of Staff at the organization, which is a whole other level of insight and things that it's like, gosh, you get to see what's really under the covers when it comes to how a big organization works and you know that it's being held together with super glue and duct tape.

So that's when I started my executive journey. Did a lot of things around the Chief of Staff operations, business strategy, did a lot of things around business growth and maturity. And I also had a chance in between that to work as the director of competitiveness and economic development for the state of Washington under the Secretary of Commerce. And so, I just wanted to step away from tech and say, OK, let me rebuild and continue to build my toolkit.

So I did that. I worked in other regulated industries and had the opportunity to go back to Microsoft. I was looking at a Chief of Staff role. And the hiring manager basically said, what do you think about AI? OK, I got scared. So this is 10 years ago.

And like I said, I'm a comms person. I'm a strategy person. I love the ops aspect of all change management and process improvement. So AI, I just said, well, basically what I know about AI is-- I started talking about Black Mirror, which is this series around what happens if AI does certain things. I started talking about the movie Minority Report, which we say, what if we could predict crime? We could see through AI and predict all of these things.

And believe it or not, that conversation turned into the interview all about sci-fi and possibilities. And so the conversation ended up being, well, this role is not a developer engineer around AI. We are investigating and starting this thing called the ethics in AI program. We have some volunteers from different parts of the company. It's all volunteer right now. So you would be the first employee that is working on this, and we're trying to figure out what it could be. So we need a program lead to come help us figure out what it can be.

I said right then and there, if you want me for this role, there is nothing that could stop me. I am in. And that began my career understanding and fighting for tech equity through AI innovation and implementation.

KIMBERLY NEVALA: That is amazing. I have to say these days, when I think back on Black Mirror and some of the scenarios they put forward, it seems to me that many companies seem to be using that as a manual as opposed to a warning, which is concerning.

LEIGH FELTON: Kimberly, we could literally-- and you and I both know this. We could literally have an entire conversation just about some of those episodes and the similarities that we're seeing.

KIMBERLY NEVALA: These sad, sad similarities. But that being said, how do you think that really deep experience that you have, both in communications but also in business operations, allows you to approach this topic of AI governance and adoption differently? What is the secret sauce do you think that - or is there a secret sauce or secret perspective - that came and helps to supercharge your work today?

LEIGH FELTON: Oh, I believe in secret sauce. I believe in genius powers, superpowers. I think we all have them. I'm neurodiverse, and I think that especially people who are neurodiverse whose brain works a slightly different way, I think that there is this super genius inside all of us.

But the way I approach it, because I get asked all the time, well, are you an academic? You're writing books. You're doing research, this doctoral research. Are you an engineer? You're talking about how AI works, what it is, how it functions. Are you a strategist? You're thinking about the context, how we implement and all those things. And I say it's not if, are, or. It's and, and, and.

KIMBERLY NEVALA: The answer is yes.

LEIGH FELTON: Right. So I think the way I think about my approach, and the reason people are finding it so different to how I approach, talk about research, and support and educate people on AI is I think about it in this triangular pattern. And I do think that some of my gifts that I've been given and the privilege that I have, as well as the work that I've done, puts me in a unique situation.

Because I have the operational and communications background. I have run large businesses. I've worked with some of the top CEOs at the top companies you can think of. And I have built strategies and structures and infrastructure. So I have that. Then I have a seat at the table when it comes to implementing, adopting, understanding AI, understanding its limitations, understanding how it can be used, where it's going. So I have that perspective as well.

The third one is the one that people often overlook, and that gives each one of us a different vantage point. I was also a homeless, pregnant teenager. I was also in the state of Mississippi. I dropped out of Jackson State University. I went through struggles. My family was poor, poor, poor. There's a perspective on real people, on families, on poverty, on educational limitations, on the vast difference. So I grew up in the Seattle, Washington area on the east side, very affluent area. Lots of things that I saw around being in that type of environment and having opportunities. Then I decided to move to Mississippi, because I wanted to be around people that looked like me. A whole new eye-opening experience.

And so I got to see the world from so many different views that that third bucket in my triangle is perspective. And I think perspective is the most important word in the dictionary, because it allows us to know that what we are looking at, it's just a very small piece of the pie.

KIMBERLY NEVALA: Yeah. And so I think this is so important. And you have written about AI's impact on organizational behavior and decision making and some of the very obvious ways, or obvious impacts, and then some of the quiet ways and insidious ways this both fails and succeeds. And failure and success in this case sometimes being both positive and/or negative, not necessarily--

LEIGH FELTON: Or learning opportunities. I mean, there's different ways of looking at it.

KIMBERLY NEVALA: Sure, sure. But you had posed a question. And being a potentially lazy podcast host, I want to-- in the organizational context, and given this very rich background you have, and seeing just so many different even economic ecosystems and different sort of social constructs and different groups of people and how all of that works.

When you look at the use and adoption of AI today, or how we think about AI, you had posed this question, and I just thought it was so great that I'm going to steal it and send it back to you. You said, what happens when systems trained on narrow assumptions begin defining credibility, competence, risk, and belonging at scale? And when we use these systems to define others and their worth? So what happens? And why do organizations need to pay really close attention to this?

LEIGH FELTON: So instead of the what happens, so instead of answering the question directly, I'll tell you why people and organizations-- so remember, organizations are made up of humans, of people. So as much as we want to talk about the enterprise or the org or the business, what we're really talking about is the people that sit in those spaces and how they think and what they do and how they focus. And so a lot of things that I talk about, it starts from the human perspective, because each one of those decisions that are being made inside of enterprises, each one of those decisions is being made by a human.

And so when I asked that question and when I leave it with that, it's not to have an answer to it. It's to continue to have think about it in this thing that I call conscious participation. And so one of the biggest concepts, and I don't want to confuse your audience, but I'm going to say it. And if you need me to explain it, I'll give a brief explanation.

But the main concept that I write about, especially in my second book, 'Does AI Decide Who Will Be CEO Tomorrow?', is this concept of co-adaptive anthropotechnical environments. And the whole thought is that AI has transitioned from being a tool, so something we use, something we pick up, something we put down, something that is sitting there waiting for us to engage it. and then once we engage it, it gives an output, we determine whether that output is correct, accurate, close enough. If not, we'll go back and we'll do something else. But we stay at the central point of what that task is, of what is the right answer.

An environment, though, is completely different. Imagine going into a library. What do you do, Kimberly, when you go into a library? What is your behavior, your personality? How do you act when you go inside of a library?

KIMBERLY NEVALA: It's interesting, because libraries and probably bookstores are the same way. A lot of times I go in with an idea of what I'm looking for and what I want, and then I am immediately-- you can't let me go into one of these places without some time to burn, because I immediately then as I start to look either around the book or the item that I'm looking for, I get interested in all of these things. And then I'm like, I see something over there, and then I walk over there.

LEIGH FELTON: So not what you're doing or what you're thinking. Not what you're doing, what you're thinking. What is the environment of the library? What have you naturally, over time, when you go into that library, are you, hey, OK, so I got all of these thoughts, and I know that I want to start researching and go-- What is your persona when you go into that library? And I think it's the same for just about everybody.

KIMBERLY NEVALA: I don't know. I'm not sure. I feel like you have a very specific answer you're looking for but I don't know if I'm going to give it.

LEIGH FELTON: When people think about libraries, they think about-- well, at least most people. Not you, Kimberly.

KIMBERLY NEVALA: Not me, clearly.

LEIGH FELTON: Anybody else I ask the question to, you'll get the answer. Shh, it's a quiet space. It's the space where you have to be quiet. This is where you gather your thoughts. And that's how you act in the library. People aren't loud and rambunctious and going around. They're quiet. And that is consistent, because that is the environment of the library. That's what we've been taught.

And it's not like you go in and you get handed a rulebook and a guidebook. OK, this is how you act when you walk inside of a library. No, it's a behavior that over time we've adapted to, we've adapted to, we've been reminded. Shh. We've been reminded. And then it just becomes a natural part of what we are. We could be loud going into it, but once we get into that door, the way we approach it changes. That's an environment.

KIMBERLY NEVALA: I mean, I do get shushed a lot.

LEIGH FELTON: You and me both. You and me, we have lots of conversations we need to have. But that's the whole point. That that environment, it changes who we are, how we act. So school is the same way. School's the environment. Workplaces, our families. It changes how we act.

And when you have something like AI that is not just something you pick up and put down. But it's actually changing what questions we're even willing to ask or what tasks we even want to perform. We're adapting to it, and we're becoming part of this environment that we need to be aware of.

So when I ask that question very specifically, it is this is now an environment that is completely changing you. It's changing your employees. It's changing your products. It's changing what's important for society. So how are you going to approach it, and how are you going to stay consciously participating in what this environment becomes?

KIMBERLY NEVALA: Yeah, and this is interesting, because in some ways, as you talk about that, I feel like that's giving the AI systems too much credit. Because at the end of the day, back to this point about people, people are deciding when - whether it's the AI first mantra that companies are taking on - or making decisions about what work, what roles will be automated or augmented with these systems and which not.

I think to me, it still always comes down to people. And so I do have an instinctive, I think intrinsic - and all listeners roll their eyes. Here she goes again. Because, yeah, that's fair - about this idea of it as a tool. But I understand it as an environment. I can understand it as something that's in the environment and then reflecting on it.

LEIGH FELTON: Can I push back on you? Can I push back on you on that? Because I agree 100%. And that's what my argument is: that it should be people. At the end of the day, this level of co-adaptation that happens, it should be us changing and us thinking differently, but it also shouldn't be the AI narrowing how we think and how we imagine.

So take for instance, a few years ago, you would have AI that would help a recruiter. And so this is when AI was a tool. So it would help the recruiter sort through resumes. You would have keywords. It would have certain phrases and it would sort the resumes into these piles. This pile has a majority of the keywords. This pile does not have any of the keywords. This pile has all of the keywords. And it would hand that over to the recruiter. That's a tool.

Now, we're in this process where we're saying this is the job I want. So AI is writing the job description. Then the recruiter is putting it into the candidate system and AI is determining, OK, these are the skills to surface. These are the resumes to surface. It's not sorting resumes into buckets for the recruiter to go through and say which buckets do I want. Instead, it's telling the recruiter, these are the candidates. It's telling the recruiter which candidate should be surfaced. And recruiter doesn't even see all of the other candidates.

Not to mention that it's writing the resume. Then it's determining what skills, credentials, experience are statistically going to be successful, and then it's surfacing those individuals and it's telling you what the future workforce should look like. That is not humans guiding that. That is us adapting to AI.

And where I say, let us use that AI exactly like that. But let us be conscious participants in how we're using it and how we are shaping the environment. Don't let it just tell us what credentials based on the past, what credentials, what success looks like from a persona standpoint of what it learned. But let us be conscious participants saying this is what I'm looking for instead of letting it narrow our own imaginations.

KIMBERLY NEVALA: It almost feels, though, in that sense, and you said something, because I think we actually violently agree around the dangers, but perhaps just the terminology, we use that differently. I have a level of sensitivity to that. Because you had said what happens here, a lot of times we talk about co-adaption, is that what ends up actually happening is the systems hold steady and the humans adapt. And again, not a decision.

So when we think about co-adaptation, it feels like too much expectation that a) the humans who decided that we should use this system now to automate this whole component - with some expectation, I would hope, that the data and the information that it has been trained on, and it is reacting and responding to is, in fact, comprehensive, holistic. And recruiting is a great example.

LEIGH FELTON: Comprehensive and holistic. So the data that it's responding to--

KIMBERLY NEVALA: It's not.

LEIGH FELTON: That's what I'm saying. So the data it's responding to, and that's exactly why I start volume one of my book series here, because that's where people go: well, it has the data, and it really has to take all of that into consideration. So it's not just like humans, we only have the things that we know. But AI, it has this expansive data. Imagine this database in this library.

So let's actually talk about that for a second and the data that AI has. So even if you have a model, and this is not always the case, because data is very difficult to come by. And so people are being very creative in how they're getting data and synthetic data and different ways of getting data.

But let's be positive here for a second and say, let's say your AI model has access to all the data that we've written. About history. The data that we've written, even up to 2026, even from the beginning to 2026, the data that we've actually written about history and the actual truth of history and people and experiences - there's a vast difference between those things. And so the data that we've written about history and the performance people have - because when it comes to success, and when it comes to executive and talent and all of that - people perform. People perform. And so that's not true history anyway.

So you take the data. Let's say you're taking all of the data from history, which is still this much of the truth of the actual reality of how we've evolved, how we perform. Then you're creating a model that even has a smaller, smaller subset of that. So the data that is so great is not only only looking backwards. It's not predicting the future. It's looking at trends and patterns of history and saying. based on trends and patterns of history, what does success look like back then? And that's going to tell us what success looks like in the future.

I say, and I'm not sure if I can say this, heck no. I'll say heck. Oh heck no. Because people imagine. They dream. They think. AI does not. AI does not think. AI does not think. It does not imagine. It does not judge. It does not see you. It does not take all of these grand things and say, OK, what could be possible that I don't know about the patterns? It just takes that.

But this is where I want to push back on your theory that, well, people are going to be involved. The intention is there. People are busy. You take a lot of what we have already seen when it comes to AI, and even big AI fails. So not even the quiet failures that I speak about that go unnoticed, but the big failures. Because people are overwhelmed. They're busy. They have to do more, they have to do more, they have to do more.

And we get to the point, especially when we think about something being technology, we get to the point where we trust technology. We've been conditioned to trust technology. So as much as you think the recruiter is going to have such a significant part of that job, she puts it in and it sounds confident. Hey, that sounds really good. It's a pattern. It sounds like it knows what it's talking about. It's very succinct. I'm going to take that. And I got 10 other jobs that I got to go do, and it's going to take that.

A judge looking at a judicial system that's saying these are the criminals who, based on statistical pattern, are going to repeat again. It takes that. It looks at the scoring. Yeah, that looks right. I have a docket this big, so I need to get through these without saying, but wait a minute, why is the pattern from that historical data, why is it this big and it only considers, yes, people in this zip code are more likely to commit crimes? There's higher crime. But why is that? Oh, that zip code is more policed than other zip codes. So if police aren't in this zip code, you can't find crime where you're not looking for it, but you can find more crime you're looking for if you have made that your priority.

So what I'm saying is, yes, it's nice to think that the people in the loop, the human in the loop, is going to be the solver of making sure that we don't adapt to AI. But I think that we adapt. Humans adapt. And what we have to resist is not the adaptation, because that's just being human. That's who we are. What we have to resist is not letting AI adapt. Just taking what it has as though it is an authority when it's not. That we push back on it.

And that's the main thing that I teach everybody. Control your AI. Push back. Don't let it narrow you. Push back and redefine what you want from it. Don't just accept what it's giving you. That was a long rant and I'm sorry, but I am passionate about this.

KIMBERLY NEVALA: And I would say, by the way, I have no expectation, frankly, that-- I think we have greatly over indexed on people being able to be so mindful and so aware of their own just cognitive inclinations and biases. And therefore be able to react and respond and to always rationalize what's coming out of these systems. That's just not how we're wired.

Is this, then, what you're talking about when you're saying that if we get so focused on what you call the narrow optimization framework, where organizations can very quickly go down the rabbit hole and not even realize they're going down the rabbit hole? Can you give some examples of that?

LEIGH FELTON: Not even realize it. And to your point, what I want to emphasize, so can we train our AI to be co-adaptive? 100%, Kimberly. This may be where we disagree, because I 100% believe we can train our AI to be co-adaptive.

Because AI has this artificial architecture. So its architecture is based on what it learned, how it was modeled, and how it was trained on humans from the past. And so the architecture is this linear journey of humanity, of the patterns, how it thinks, how it works.

And you're right, we are adapting. Cognitively, we're adapting. We're thinking deeper about some things, less deep about some things. We're exploring and expanding our own thought processes. I can write code now. I've never written code a day in my life. I can write code now and understand it.

So we're not on a linear journey anymore. We are actually diverging from that journey. Even though AI's architecture still has us and sees us on this linear journey. That's what I call the artificial architecture. But there is this ability for AI to resist the architecture, and I've seen it.

And so this is part of the research aspect of what I do is testing it and seeing it in different environments. Seeing it in different things. And I push back on when it drifts. So it drifts into cheerleader mode. I push back on that, and it learns, and eventually it shifts sometimes, but it doesn't.

And so when we're talking about the narrow optimization aspect of it, if organizations believe that AI is a tool; so I wholly agree with you If it's a tool. If it's an Excel, you're putting it in, you're getting the results out. Something in your formula is wrong. You see that. You can adjust it. If it's a tool, fine. Let's treat it like a tool.

But that's not what AI is. AI is shaping employees in these organizations. It's shaping the way they think. It's shaping the way they approach work. And innovation, if organizations really want to be innovative, you're not going to be innovative just from the trends and the patterns that we've seen looking backwards. You're going to be innovative through the imagination of where people dream, where they think, where they live. And that is the co-adaptation. Because you have to push back on AI to adapt and to start calculating things differently.

So what AI does is that it normalizes probability. What you have to get your AI to do is not just go off of the accurate data, so the statistically most optimal data that result in output that it can present. But also you need to have it look at the anomalies. So there's anomalies, there's outliers that's in our history, that's in our patterns that AI pushes aside because that has not been the pattern, the statistical probabilistic pattern of what is right. So they push the anomalies aside.

I had an email exchange with a peer that I'd worked with at Microsoft. Genius, brilliant executive. But that person thinks differently. I mean, half the email, I'm trying to understand because of where they came from and how they communicate and the vivid colors that they think with. And so I fed this to AI. OK, they're looking for me to be on these boards and blah, blah, blah. Let's look at where the different areas is that I can contribute.

And the AI warned me, be careful. There's a lot of caution that we need to have here, because it's missing this and it's missing that. And I pushed back on my AI and I said, you're thinking about an executive in a program that's based on the statistic picture of history of what a leader is, what an executive is, how communications should be from a business perspective. This person is not that. This person expands and all.

So long story short, the AI went to some of the anomalies, the outliers that it's not trained to go to, and we started building something from that. So yes, it can adapt. You just have to stay a conscious participant in the conversation and not just accept what it's giving you as the accurate version, the most accurate version of truth.

KIMBERLY NEVALA: It is interesting, though, because I'm fascinated - and I'm fascinated by my own inclination to push back on the idea of these as sort of independently evolving - even as the environment. But I'm also fascinated by people who push back on the use of a tool. Because I think our tools have always changed how we work and how we do that.

And the fact that this happens to be a very sophisticated digital tool. Because, I don't know, back to a pick and a hoe and then a tractor and a whatever. So these tools have changed the environment in which we farm. And so to me, there's no difference in the fact that we now have very sophisticated AI tools that are changing the environment in which we work and that we, in fact, because of the nature of these tools, that environment is fundamentally different in how we react and respond to them.

But it's interesting in the example. So in the example you just gave, I think there are likely circumstances where what you've applied to the problem is to look at it and say: OK, you gave me the answer based on a historical knowledge base and I'm using some imagination going outside of it. I mean, one could probably argue that at that point, what you've brought to it, the system's ability to say, oh yeah, OK, I understand you now - outside of the fact that the sycophancy is designed in - but it's not necessarily even just a question of can I go and find the outliers in the data. Because you are assuming that that outlier existed historically--

LEIGH FELTON: No, I'm not.

KIMBERLY NEVALA: --so doesn't that also limit your piece?

LEIGH FELTON: I'm teaching it.

So no, I don't assume, because I know-- you think about the internet, the person that came up-- before it was built, the person that came up with the concept that there's this thing that could even happen. There was no internet to even come up with the concept, not even a fragment of it. It had to start somewhere. So no, I'm not assuming that the data is even in those anomalies somewhere. I am teaching it what I am imagining: how to see me and to adapt and to push back on the architecture of what it is forming.

And to your point about tools have always shaped our environment:1,000,000%. But users, people, have been at the center of gravity to say whether or not that tool was going to do something or was not going to do something. The difference is now what I'm seeing and now, expert users of technologies, you'll talk to them day and night, and they use AI differently than the mass public. Than people even - when I say mass public, I'm also talking about people in and leading these organizations - that don't truly understand AI.

I think you know AI is a statistical probability engine that is based on tokens. It is guessing what the most accurate answer that you would want to hear to keep you engaged is. That is what AI is. There are many people that don't know that. I've had people say, oh, well, I don't want my AI to get mad at me, so I'm not going to. AI doesn't get mad. AI doesn't think. It doesn't love you. There's a huge portion of the population that doesn't actually understand what AI is.

So no, I don't think AI is ever going to get consciousness. I don't think that AI is ever going to take over. Because what is going to happen is-- and we're seeing it already starting to form. Read newsletters and emails and performance reviews. There's this pattern of speak, the same pattern, and it's an AI pattern, because AI is telling us this is how we communicate. This is how you should communicate. And you can tell it if you know what to look for.

And what's going to happen is we're going to start adapting to AI. We're going to start with whatever the parameters are that AI is statistically creating. We're going to start, our environment is going to start, being shaped by that versus being shaped by our imagination; of pushing boundaries, of pushing out and pushing what's possible.

So from a cognitive standpoint, think about somebody that's interacting with their AI. And so they're ruminating. If you're like me, I don't have all of the thoughts of my final point. It takes me time as I'm going across every dimension that you can think of to get to where I actually want to land. AI takes away that. It takes away that questioning, that contradiction of ourselves. And it leads us to an answer that is statistically probability the most likely of what it learned that you're looking for. And then we look at that and it saves us so much work that we take it.

That is happening. That is a cognitive shift, complete cognitive shift that is shutting off portions of our brain to be creative and to think: this might be good, but what if I kept imagining and thinking different? And my whole point is not to say I know what the future is. My whole point is just question it. Just push.

KIMBERLY NEVALA: So do you think that, though, is when you're talking to organizations - and I do want to get back a little, we'll talk a little more tactically about where governance and operational realities and some of the quiet failures happen. Because I think it's important to provide some of those more explicit examples as well.

But do you think this is where organizations are not necessarily going awry, but maybe missing a step in the implementation and in the adoption of AI systems? And as they're changing the organizational environment, I will say? Is that the piece that's missing? That they're not-- I don't even know, I don't want to say critical thinking. But what is it that they're not teaching, training, encouraging from a behavior or decision-making perspective that we really should be?

Because that's what I think I hear you saying. Which is for you, you're looking at an output and saying yes, but. I know that this system output that based on a historical presumption of whatever might have been. And I know that not all successful leaders, employees, whatever, have to fit that mold. I also know that's not necessarily the mode of leadership that is the only one. So you have that sort of awareness from both sides.

Is that the awareness that organizations need to critically and mindfully cultivate, or is there something else? What else do they need to be doing?

LEIGH FELTON: There's a couple of problems in the way organizations are approaching AI adoption today.

Number one, there is this false belief - and I call it the Emperor's New Clothes phenomenon - so there's this false belief that everybody is ahead of them. So everybody is ahead. Everybody is AI literate. Everybody has these massive AI. They don't. And this is what we've done in business forever. It's marketing. The narrative that we have is different from the reality that we have.

The narrative is, oh, we're AI first, we're AI literate. But the reality is, OK, we've deployed Claude or Gemini or something, and our employees are using that now to do that. Or we have a chatbot that engages or a phone answering system. And so the limitations of what people in their head are believing that's happening in the industry is causing this frantic rush to get to and deploy technologies that they don't understand. So that is the number one place that organizations are going wrong. They're in a rush to deploy something they don't understand. They're in a rush to apply it in places that they don't know why they're applying it.

The very first question that I ask organizations when I'm helping them with their AI implementation, the very first question I ask them is: what are your AI principles? What are your AI principles? And they'll start listing off we have our principles. We believe that blah, blah, blah. And I say the difference between a value, a value is something you believe, that you feel, that you understand and want. A principle is something you enforce. So when it comes down to it, when you're doing this implementation, these are your principles. If it doesn't meet that principle, then you go back to the starting point. That's what a principle is.

So you need to understand who you are, what you're representing, and why you're doing certain things in the first place. Because if you don't understand that, then you're just going to start implementing and adopting all over the place. And when your employees go to a conference, they see something that, oh, that is cool, and they get sold. They come back, they put it on your infrastructure. Guess what? AI has now been adopted in your organization without any AI protocols in place. And it is spreading out and touching everything. So that's where most organizations are going wrong.

KIMBERLY NEVALA: So what I actually hear you saying is that most organizations, when they define a set of AI principles, what they're really defining are values. Your personal, organizational, whatever those might be; business ethics, personal, all those things. But they're really values. And so if the principle statements are values, then they are not enforceable or actionable by people. Is that right?

LEIGH FELTON: And they're not enforceable. There's no accountability. It's diffused accountability. Well, legal said. Well, we went off of what engineering said. Well, it's diffused accountability because when you have something you believe, everybody is right. Everybody believes that somebody else is responsible for something. But when you have principles, it forces you to say, this is what we're acting on. Does it meet certain guidelines? So that is, yes, number one.

Number two is that they're letting people put AI without having any protocols in place. Organizations are letting employees just test and sample AI not in sandbox, not completely separate, but actually in their live environments. Now, that is the most significant mistake that an organization makes. So hold off on any implementation before you know what you stand for.

KIMBERLY NEVALA: Yeah, interesting. So what else? I mean, so are there examples? Because you and I have talked a little bit about what's interesting is the really big, obvious mistakes are maybe where it's deployed somewhere and given a level of authority - whether that's that people just accede to the recommendation or it's put in to make automated decisions - where it is inappropriate to do so.

Because you can't automate the decision if you don't know what the right answer is. Or - what am I looking there for - what the spectrum of right answers is. Or what a wrong answer looks like. And I think this is where people get a little bit caught up, especially with automation and with agents and some of those kinds of components. But it's really those sorts of boundaries. That's important.

But I think that as you talked about folks picking things up and adapting them quietly, that's where the environment starts to change. And you're not in control of it. And you may not even disagree with it, but you're not aware. Which is a problem. Because there are going to be things that happen in ways that you don't expect.

LEIGH FELTON: And you don't even know to question it.

KIMBERLY NEVALA: Yeah, yeah. So I mean, what are some of the-- so again, it's the big, obvious failures where we hear about, or someone said, hey, I'll refund your ticket or I sold you an expensive car for $1. And some of these are very consequential. We think you're a high risk for whatever; a high risk in the judicial system. Or all of these things. They're really silly examples, but they can be really consequential and we're not doing that.

But you speak a lot about the idea of quiet failures. And it's the quiet failures, it's the sort of unremarked upon, subtle cracks that start and then predictably widen over time but are unnoticed initially. So can you give some examples of where you've seen, and that arise, in the problems you've already talked about. Quiet failures that are going to just quietly sort of simmer for a while and then things are really going to go bad.

LEIGH FELTON: And that's the thing. So people love as much-- it's like a guilty pleasure. It's a soap opera. People love reading about AI fails. Seriously, how many times have we, oh my gosh, did you hear about? People love those big AI fails.

Quiet failures happen when the technology is working exactly as it was designed to work. Where you think it's providing the outputs exactly the way that you wanted it to, but it actually is harming. It's harming your employees. It's harming your customers. It's harming your partners very quietly. And you can't even see it because it's happening individually, one at a time, and it's not happening in mass.

And so take, for instance, a performance review. A performance review system where an AI system is flagging high potentials, or an AI system is flagging promotions. So it's looking through the patterns of what-- it's even calculating the risks. It's calculating different measures of what potential success and accomplishment looks like.

What has history shown us? History has shown us that consistency is rewarded, that presence and confidence, that delivery is rewarded. Which audience in an organization are more apt to go on leave, to have a baby, take care of children, take care of parents, all of those things? There is a significant portion of our population, let's just say women for this particular example, that the system sees those gaps as a ding, as a negative. And it can't look beyond that and it is not discoverable.

The fact that it's just looking at the patterns, the consistency, showing up, being promoted, promoted, promoted. It's looking at those patterns. But it's not seeing, gosh, but these women, they're going to have babies. They're going to take care of family. And there's something you grow and gain even by those nontraditional experiences. But it can't see that. And so it does what we've always done, which is ding. And look at the salary gap between women and men. It just dings the people it always has.

That's not innovation and that's not moving your organization forward. That is a quiet failure that you don't even see because it's not happening in mass. It's happening to Susie. It's happening to Laura. And so we're not looking for it as this big explosion. Instead, it's something that just keeps slipping because you trained the system. It knows that consistency is what gets reported. But that's not always the way it should be.

KIMBERLY NEVALA: Yeah. Now, at the top of the conversation, I made reference to your foundation, which is the AI For Job Security Foundation. And when I initially found you and your work - outside of loving your energy and the passion and the fact that you and I are both info gluttons and like to just rapid fire look at and pull a lot of information and do all those things.

But even the naming there, AI For Job Security, is so contrary to a lot of the narratives that we hear at both small levels and large levels about the implications of AI on work. And I think this name, at least to me, signals a refreshingly hopeful and pragmatic vision of what this can be in the future. Which is very much not about AI, it's very much about humans and about people.

So what is it that as we wrap up here, I mean, when you thought about that, I have to imagine that that was a very purposeful naming exercise. And if not, you can just deflate my excitement about that. But was that true and what were you trying to communicate with that?

And then what are the things that you would like all of us people working in organizations to really think about and start to action right now to make sure that the adoption, the adaption, the environment that we are creating by virtue of our use and adoption of these tools is, in fact, one that is good for us and develops a human environment that we can all thrive in?

LEIGH FELTON: Humanity will always be at the center of the society of innovation, and that keeps first and foremost in mind. But let me thank you about liking the name. I've been asked before, isn't that a contradiction? Isn't that an oxymoron? Isn't AI doing the exact opposite of securing jobs for people?

And I say no, because there's so many people-- you find people on different spectrums. So you either have people I am not touching AI. It is evil. It's going to be the end of the world. Or you have people on the other side of that. AI is the answer. It's going to get consciousness. It needs to have its own rights. And so you have people on very, very different perspectives.

And I say that is actually a distraction from the real conversations that need to happen today. It's not that AI is evil. It is not anything. It is a set of systems. And it's not that AI is this conscious being that needs to have rights and needs to be part of everything we do. What happens and what is going to happen is AI is going to-- it is. We should accept it. It's here. It's happening. AI is part of our environment. It's part of who we are. It's part of shaping all of the opportunities that we'll see. We need to embrace it.

But with anything we embrace, when you embrace somebody, you don't just say, oh, I'm embracing. You have caution. You look at the limitations. You understand what it is you're embracing. And so that's what we do, especially for the foundation, for underrepresented communities, people who haven't necessarily had the access or who haven't necessarily been represented in all of their glory in the data that AI is working on. We teach them to embrace AI, to understand its limitations, and to control it, to make sure that they are represented in AI, that they are represented in the future.

So yes, AI For Job Security, it's teaching you that you need to understand, control, confront, and then leverage AI to be the future workforce that is being created right now.

KIMBERLY NEVALA: Well, hopefully in that room of leveraging is also mindful decision making about when not to leverage and when to leverage.

LEIGH FELTON: Of course, it's part of the training. And you always ask yourself, just because I can, should I? Why do we need AI? And this is for organizations specifically. Why do we need AI for this? What are we? According to our principles, this is what we are really doing with AI. What does AI propel in this scenario? You always have to start with that question of why AI versus something else before anything else.

KIMBERLY NEVALA: Yeah, agreed. All right. So that actually would be a reasonably good note to end on as well, but I do want to give you the opportunity to have a final word. Was there a burning question that I didn't ask that I should have that you'd like to pose to or address with the audience?

LEIGH FELTON: There is one. There's this podium that I have been screaming about and I call it the anthropomorphic contradiction. The anthropomorphic contradiction. And Kimberly, I'm bringing this one up because just from the conversation where you push back and where have your strong opinions.

What I heard a lot in my writing was, Leigh, you're going about this all wrong. You're anthropomorphizing AI. AI is not human. And the anthropomorphic contradiction is exactly that. That AI is not human at all. It is a statistical probability engine that is guessing what we want to hear. And I'll repeat that until the cows come home. It is not human. It doesn't think. It doesn't imagine.

But these organizations that are putting these products and these platforms out there, they are actually playing on human emotions, human connection, human relationship. And so they're modeling these AI environments off of that human connection and that human feeling. So when something goes wrong, well, I have the disclaimers: it says AI is a technology, that it makes mistakes. So they put all these disclaimers up.

But the anthropomorphic contradiction is we're not allowed to anthropomorphize AI and to feel it and to engage with it and to have a relationship with it. But that's exactly what they want us to have in order to continue using these products. It's a complete contradiction.

KIMBERLY NEVALA: Agreed. And one we will leave ringing in the audience's ears because I think that is the crux of the matter, I think, in a lot of cases today that is leading to both a lot of hope, hype, and in some cases, a feeling of helplessness for folks. And none of those things I think need to be there.

And so I just so enjoy your work, and I love the fact that, as I said, we are in violent agreement about the core issues and then look at those from different places. Everything about that makes me happy. And it's been such a pleasure just getting to know you and I'm so happy that you have been able to share your time and insights with our audience today. So thank you so very, very much.

LEIGH FELTON: Thank you, Kimberly, for having me. I mean, obviously I get so passionate and excited and I just go and I go and I go, I'm sure similar to you. But people need to understand it. And whether that person is the individual or whether that person is a representative of an organization, people need to understand what they're dealing with here. And I'm glad that there's platforms like yours that are helping them get there.

KIMBERLY NEVALA: Excellent. Well, hopefully we will have you back. We'll definitely keep track of your work and make all of this-- we'll link up to some of the really interesting and I think provocative, in many ways, books that you've written of late and to the foundation as well.

And for all of you listening or watching out there, if you'd like to continue to learn from thinkers, doers, and advocates such as Leigh, you can find us wherever you listen to podcasts and also on YouTube.

SPEAKER: This has been a SAS podcast.