WEBVTT 00:00:01.760 --> 00:00:35.760 Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm. 00:00:35.760 --> 00:00:37.440 Now onto the show. 00:00:41.280 --> 00:01:11.920 Welcome to another episode of the Practical AI Podcast. I'm Chris Benson, your co host. And this week, we're going to bring you three more of our speakers from the Midwest AI Summit. They are three speakers who are dealing with AI issues on an everyday basis that many of us are coping with. Those three speakers are Don Barger from the International Mission Board, Mark Lay from Nobelis, and Darren Taillie from Evercore Technologies. 00:01:12.240 --> 00:01:19.995 Next week, I will be back in the usual format with a guest speaker, but in the meantime, I hope you enjoy these three. Thank you very much. 00:01:19.995 --> 00:01:50.805 I want to introduce our first speaker, Don Barger. I've I've known, Donnie for a a a long time, and he's taken kind of the whole journey of Prediction Guard with us. And I've... I'm so impressed with how he has stewarded the International Mission Board forward in terms of their AI journey and adoption. He's the director of innovation and artificial intelligence at the International Mission Board. 00:01:50.885 --> 00:02:08.065 He spent a lot of years in Latin America and different parts of the world, with really deploying strategies rooted in orality and storytelling. He's gonna be talking about building AI trust, what we said yes to and what we didn't. So welcome Don Barger from the I and B. 00:02:08.065 --> 00:02:25.820 Appreciate it. Yeah, so I've known Dan for a while. The first time I met Dan was at a much smaller meeting than this that he was actually leading, Again, another meeting he was leading about AI. And at that meeting, I was the dumbest person at that meeting. It was hands down. 00:02:25.820 --> 00:03:01.020 I walked in, I'm like, I don't think I knew what I was coming to. And I could say that if it hadn't been for Dan at that meeting and then some other meetings that we met at, and we sketched out some stuff in a hotel in London in the lobby, and next thing you know, we're really doing a lot of crazy stuff. So every organization that's building AI eventually runs into questions about, Should we be doing this? So that's what I want to share with you today, the journey that I've been on, who I am, the oddness that I am the one that's speaking to here. I'm not your typical AI presenter. 00:03:02.285 --> 00:03:40.405 Normally, you would have somebody that has a degree in computer science or data science or physics, and I will tell you that I have an undergraduate degree in accounting and two degrees in theology. So if that surprises you, I think that it should encourage you. Because the reality is, for the first time in a long time, maybe the first time ever, normal people can build incredible things. And I've been so excited to see how our organization has embraced using AI in ways that I wouldn't even have dreamed we could do three years ago. So the problem is what's responsible. 00:03:40.485 --> 00:04:10.485 And so the question we need to ask a lot of people is, is this the right thing to do? The responsibility question is the one that builds trust. It's the question in an organization that's 181 years old, that if you show that you're building trust and you're trying to build trust, they're a lot more open to changing things. So almost four years ago, I'll tell you how we got started. Almost four years ago, before I sat down with Dan in London, I had another group of people in London that I was meeting with, and we were looking at what we believed about AI. 00:04:10.590 --> 00:04:41.540 One of the guys on my team had just finished his PhD in digital ethics and computer ethics, and AI was not really as popular of a topic then, but we sat down and said, Let's talk about what we believe about AI. And what will we do and what will we not do? Before we ever write one line of code, let's come up with a charter about what we believe AI is. And we did that. We spent a couple of days with eight people and we hammered out this agreement to what we would do with AI. 00:04:41.540 --> 00:05:23.885 And I can look back to that time as something that's impacted everything that our team has done since that time, because it allows us to know what is possible and what we're going to say no to without even having to think about it. And there's been multiple times that we've come across issues that people have asked us to do, and we've been able to say, No, we don't think that's what AI should be doing. So the conversation that most organizations skip is, Just because we can do something doesn't mean we should do something. And so we look at really hard what we believe about AI. Let me just give you some examples of things that we said we would do and what we wouldn't do. 00:05:23.885 --> 00:05:41.540 The easy things would be drafting and editing, summarizing and research. These are things that all of us do. Translation, we use a lot of translation tools. We do a lot of really good work with translation, saving a lot of money in translation. Training materials and learning support and content. 00:05:41.620 --> 00:06:26.270 We create code and we always have humans in the loop when we write code. The timing to ship new products has gone from years with lots of contractors that don't really understand what we need to be able to ship it ourselves and prototype in a day and ship within months really good product. So the things that we said no to for us were really important, was AI presenting itself as a human in ministry. But I would just say anytime somebody's expecting to be thinking they're talking to a human, we don't ever want to misrepresent AI as speaking for humans. Final decisions about people that would be made solely by AI, We don't ever want to see that. 00:06:26.270 --> 00:06:59.430 Monitoring people without their knowledge, that's happening all around us. We don't think that's something we should be involved in. Autonomous actions in high risk and high stakes context, we have people serving all over the world, and we want to make sure that we're not giving AI the autonomy to make decisions about that. But then finally, probably the one that we've said no to the most is handling what I would consider soul care over to an algorithm. And from us, from a Christian standpoint, I don't want AI to be taking on pastoral responsibilities. 00:06:59.430 --> 00:07:48.315 I don't want it to do counseling because I really think that people need to be interacting with other people. We use AI a lot to help screen things and to help encourage people, but when it comes down to counseling people and sharing with people, we really want that to be a human. I like to see AI as actually freeing me up to do more of that, not replacing me in doing that. Why I lean so hard into this is that we drew the boundaries before we wrote the code. And I think that if you've not thought through what you believe about AI, the ontology of AI, what you would believe about how you treat AI and what you let it do, I challenge you to stop writing code, stop building, and at least spend some time with some people and think about what you're going to do and what you believe about AI. 00:07:48.395 --> 00:08:16.765 I don't think a lot of people are doing that. I don't think Sam Altman spends much time thinking about this question of what we should be doing with AI. I don't think Elon Musk thinks about that. I think we need to be leading out in helping do that. So the soul care question, like the pastoral care, the prayer, the counseling, discipleship, deep personal relationships, crises, dealing with people that are dealing with a crisis, we want people to be dealing with those. 00:08:17.085 --> 00:08:39.505 AI can prepare the support, but it can't really replace the person. So we had an ask that come to me, and somebody, one of the teams asked me, Hey, we have this urgent need. We have people who really need counseling, but we don't have counselors. There's no way to get them counseling. Can you help us build an AI chatbot that would basically become their counselor? 00:08:39.585 --> 00:09:04.300 And I didn't have to think long and hard about it because we'd already made that decision before. No, we can't help you with that, and let me share with you the charter, our AI charter, about what we believe about AI. And I shared that with them and they still pushed back and said, You don't understand, the need is so great, something is better than nothing. And my response was, Something can be worse than nothing if you're doing the wrong things. And so we said, No, disagreed. 00:09:05.055 --> 00:09:33.880 They may still be doing that, but I'm not helping them pull that off. So I would just challenge you to think through what it is that you're going to say yes and no to as you're doing that. So the no is easy because the no asked long before that person had asked me to help them. I'll just confess to you, and Dan knows me, I am famous for hopping on an airplane and building a solution from one place to another. And so the idea of building a chatbot as a counselor was really intriguing to me. 00:09:34.195 --> 00:09:54.340 And it would be easy for me to go home that night while we're watching Netflix and prototype it, and then the next day send in and say, Hey, we built it. What do you think? But because we had talked about this beforehand, I was able to say, No, this is not something we're gonna do. Second case study. So one of the surprising tools that we built last year, it's been out for about a year and a half. 00:09:54.340 --> 00:10:17.340 It's got around 2,500,000 users in about a year and a half. We've not had any paid advertising for it, it's just word-of-mouth, is something called Faithbot. You can find it at faithbot.tools. So we have several tools that we've built for the church on this website, faithbot.tools. But the Faithbot was sort of like my attempt to create Christian... 00:10:17.340 --> 00:10:32.540 A chatbot that would always be grounded in faith. It's going to answer your questions from a faith perspective. And so it's going to be grounded in things that are important to Christian faith. And so we built this chatbot and it's just taken off. It's amazing to see the growth. 00:10:33.715 --> 00:10:45.475 It's in 50 languages. So we've had a couple of versions of it before. The third version I built on a flight from Norita... No, from Denver to Norita. I got on the plane thinking, This is what I'm to build. 00:10:45.475 --> 00:11:08.855 I'm not going to sleep. I landed in Narita, before I landed, it was live. It was pushed to the droplet, live, published, and people were swapped over from the old faith bot to the new faith bot. The crazy thing is this thing has the ability to do so much. So we had to make some decisions early on, and one of the examples would be, I don't want Faithbot to pray for people. 00:11:09.175 --> 00:11:31.965 The first time I was playing with it on the plane, the new version that I'd built, and by the way, there's 51 languages, and each of them use a different LLM. They all have different markdowns. They all are built uniquely with different worldviews, so it's not just one thing that just translates. It's like 51 tools built into one tool, and it knows where you're entering from, and it automatically loads in that language and worldview. So it's really cool. 00:11:31.965 --> 00:11:48.610 But I was playing with it, and it offered to pray for me. And I thought, Well, that's cool, but then I realized that's kind of creepy. AI can't pray. Why am I letting AI pray? And so then I spent the next hour trying to unprogram the ability for it to pray and making it where it couldn't do that. 00:11:48.610 --> 00:12:24.165 So AI is not really praying, it's just kind of mimicking the training data that we gave it. The decision point number two was we also don't want people to just define all of their needs in an algorithm, so we want to point people to people. And so we intentionally, when you use it, it will try to connect you to someone else, particularly if you're hitting it with a crisis. So one of the things we learned early on is a lot of people who use this are people who are struggling, people who are being abused, people who are self harmed, thinking about killing themselves. And it really was shocking to me to read some of the... 00:12:24.485 --> 00:12:57.365 Because they were flagged and I would go back and make sure the AI was responding well. I don't know who they are, but I get the data to make sure the AI is responding well. So we built it so that when that happens, it's always going to recommend, try to get you in touch with somebody else, and it's going to try to connect you to help. And if you don't know of somebody, it can recommend services and crisis centers that it can point people to. I'm also notified of anything that is critical, and I can hop into the conversation and take over for the AI. 00:12:57.760 --> 00:13:10.965 That was very possible when we had 5,000 users. It is not possible when you have 2,500,000 users. So we do a lot of checking. The other things are lines that it won't cross. I would say this is for everything we build. 00:13:11.045 --> 00:13:33.230 We let it know that there are certain things that you can't do. So we don't want this to ever be hateful or coercive or abusive. So we want to build our tools in ways that are edifying and helping people. Doesn't also replace, It's not going to give you a medical diagnosis. I'm shocked at the number of people who go there and ask it for financial advice from certain parts of the world. 00:13:33.230 --> 00:13:58.110 And so we don't give financial advice. We just programmed it to say, Hey, stick to what you're good at. Don't try to do anything you can't or you shouldn't. I hate I have to say this, but if your pastor is going to AI to help you write a sermon, you might want to find a better church. But there's a lot of people, as I visit with people and I teach in universities, I find that this is more common than I was realizing. 00:13:58.110 --> 00:14:15.165 So it'll actually explicitly not do that, and it'll actually kind of lecture you on how you shouldn't be doing that. And then finally, it's not for sale. We're not trying to make money off of this. We don't charge for these products. We do this through people who love it, they like it, and they say, Hey, I'd like to sponsor it. 00:14:15.165 --> 00:14:28.090 That's what we've been able to do. It's also meant we had to keep rebuilding to make it more and more economical. We're careful with the data. Originally, I didn't archive anything. It was gone once you log out. 00:14:28.090 --> 00:14:49.440 But a lot of people want to go back, because they're accustomed to ChatGPT, They want to go back and they pick up their old chats. So we built it out where you can sign in if you want to, but you don't have to sign in. You can always be anonymous, and you can just keep that anonymous by choice. So last few things here. I would say that you should evaluate against trust, not just feasibility. 00:14:49.520 --> 00:15:23.230 So when you're choosing what you're going to do, you're going to evaluate how's this going to be received by the people you're working with. So a lot of how we've been able to build trust in a very conservative, very old, very risk and change averse organization is we've proven time and time again to make really good, trustworthy decisions. And so we've worked hard at building trust within our organization. So whether or not that's you working within your own organization or if you're selling AI products to other people, it's really important to build that trust with people. So I'm going leave you with two questions. 00:15:23.230 --> 00:15:39.155 First is, as you're looking at building AI and you're asking yourself, what are the things that we're going to do and what will we do and what will we not do? Always start with how might we? So don't start with... There's two types of people. Some people are like, 50 reasons why you can't do something. 00:15:39.315 --> 00:15:56.300 I'm probably the other guy who gives you 52 reasons why you can do something. Always be that person, even if 49 of the 52 things don't work. Be positive and try to do things. And then the last thing I would leave you with is, what will we never do? So make that decision early on. 00:15:56.300 --> 00:16:23.020 These are the things that we're just not going to do. I really don't care if ChatGPT offers this new feature or if Grock offers this new. We're not going to go there, because we don't believe that this is what we should be using AI for. So finally, I've actually found these limits, these clear limits are actually freeing me up to be able to do way more than I could do before. I have a guideline set that says these are the things that we will and we will not do. 00:16:23.180 --> 00:16:32.045 And as long as I stay within those boundaries, it's actually way more freeing than it was whenever I was always wondering, should we do this? Could we do this? Thanks. 00:16:32.045 --> 00:16:47.860 There we go. Thank you so much, Donnie. That was a that was a great start to things. I can attest to the fact that, well, one, Donnie is on a lot of flights, more more than I can imagine. But every time there's a flight, I I get a cool message like, hey. 00:16:47.860 --> 00:17:24.650 Check out check out what I built, which is which is really exciting. So, that that was a great way to start out today. I'm thrilled to introduce our next speaker, Mark Lay. He's, vice president and CIO at Noblis, a science and technology organization serving the federal government where he leads IT strategy and delivery and keeps infrastructure secure and resilient. And I've I've had the privilege of of interacting with a bunch of people at Nobles, and and partnering with them in certain respects, and it was great to be in a room with with Mark at a whiteboard. 00:17:24.650 --> 00:17:46.410 That's my favorite thing. Ask amazing questions, and he's gonna talk to us about, how you can orient yourself as AI is moving fast and what are the actual decisions you need to make as we... We're all in this environment where AI is moving faster than we can keep up. I'm thrilled that Mark's here to talk about that. So welcome, Mark. 00:17:46.410 --> 00:18:09.535 Thank you. I really love the leash concept. It's amazing. And as a CIO, I wanna take it up on another level. I have four decisions that I'd like to go through, and it really starts with when I talk to fellow CIOs and CTOs, we all say AI is moving at the speed of light. 00:18:09.535 --> 00:18:23.650 Every time we turn around, it's different. It's evolving. I meet with a vendor one month, their pitch is different the next month, the features. They're all better, but they're different. And so from a CIO perspective, that's really difficult to manage. 00:18:24.365 --> 00:19:09.720 And I think it's difficult because traditionally, we've had a business problem, process functions, and codified how to address it. And now we're in a situation where we are turning over to technology an objective. We're putting guardrails or a leash around it and having it go off and produce an outcome. And that's a scary thing, and it totally changes, I believe, the way you need to look at it. And so what I'd like to go through is four questions or decisions that I think are really critical to sort of at a NUCs level above some of the talks earlier, just how you should look at AI across the organization. 00:19:09.720 --> 00:19:50.690 The first decision is really about what should I delegate to AI? And my premise here is this idea of turning over activities to an automated system is very similar to what we've done for many, many years, and that is manage people. We have to delegate activities to people. And my premise is when you're in front of ChatGPT or another tool or you're trying to decide to build a system, you need to decide what you're gonna delegate to that AI and how much of a leash you're gonna put on it. And I really think there are two dimensions of that. 00:19:51.135 --> 00:20:11.110 There's what reward you're gonna get out of it and what risk. On the reward side, you're trying to get something better or something more efficient. And if you don't get either of those, you really shouldn't put it in AI. And then the question is, on the risk side, is there a risk of an error? And what's that risk of an error? 00:20:11.110 --> 00:20:39.080 Or is there a low risk of an error? And what's the consequence of that error? And so really, if you take this a step further, you can look at it and say, if there's better outcomes and less burden to do it, that's a clear gain. But then the next question is, is there risk in getting that gain and how do you address it? If there's no risk, like when I go online and explore things, that's down in the lower left side. 00:20:39.080 --> 00:21:02.855 The upper right side is an area where you may not... You might have to put guardrails very tightly around something. And earlier, Heidi made a point about some guardrails are are deterministic guardrails. I think deterministic guardrails fall up in... Or in that left lower quadrant where you have high risk of something happening. 00:21:05.000 --> 00:21:40.330 So the first idea is look at everything as you're delegating just like you would delegate to one of your workers. They can make mistakes, but you wanna control those mistakes and you wanna keep them in balance. You wanna put a leash on what you're delegating. And I find this an easy framework to sort of look at how much of a leash you have to put on it and really where you should put the AI and where you're gonna get the most value. The second The next area is really the idea of who's accountable when AI acts. 00:21:40.330 --> 00:22:05.560 And this is an area where I think a lot of organizations have oversimplified this. And the first premise is that accountability can't be delegated. I mean, humans can delegate accountability to other humans. But with AI, we should not be delegating accountability. So somewhere up the chain in the RACI chart for the A, there always needs to be a person. 00:22:05.640 --> 00:22:31.920 And there are four areas where I think this is important. It sort of covers the landscape. The first is protecting the data. And one of the things that we've done in our organization, because we deal with a lot of compliance from the federal government, different agencies, and security, we actually spent the last year identifying over a thousand datasets that we thought were relevant. I don't think we got them all, but we started. 00:22:32.160 --> 00:22:50.375 And then we identified a data store, person who owns that, okay, and is responsible for understanding what the data is, what the quality is, what the timeliness of it, and also what what... How does it have to be protected? Who can see it? Who can't see it? If you don't have that... 00:22:50.375 --> 00:23:17.935 I mean... And and one of the things that came out of this is even people that you know they know they own the data, our HR department, our their finance department, they were so traditionally programmed to say, well, the data's in the accounting system or the data's in the HR system, it protects it. Okay? As you go to agentic AI and these tools, you can't live in that type of world anymore. You have to have somebody who owns the data and knows the data. 00:23:18.015 --> 00:23:41.245 In the end, after a year of work there, we got down to about a hundred and hundred hundred and fifteen, 120 datasets. And these are not data files. These are datasets of information, tabular, or even reports that were critical to our operations of our business. And we had about 30 data stewards of those. But that was the first lane. 00:23:41.245 --> 00:24:25.160 The second is this whole concept of validating the output. I mean, you can put guardrails on it, but depending on what you've delegated and the risk of error, there's a whole activity of data output quality. And I think when you look at is it a platform or it's a centralized tool, who's checking that data quality? And sort of going back to the platform versus the enterprise system, I like to use the analogy of we've been living with this for many, many years. Somebody earlier said Lotus one, three, which I'm old enough to remember, but Excel is a platform that's been deployed to all our employees and finance teams. 00:24:25.240 --> 00:24:44.575 And it really... To me, it's the equivalent of of AI. It allowed before Lotus one two three and Excel, there weren't tools to manipulate numbers or math the way we can. That changed the world. I see AI doing the same thing in terms of pushing it out. 00:24:44.815 --> 00:24:52.990 But if you think about Excel as opposed to a centralized Power BI tool where you have structured reports, 00:24:52.990 --> 00:24:53.230 when 00:24:53.230 --> 00:25:08.735 you give the when you give the finance team an Excel spreadsheet, they can now build macros. They can build traditional things that that they only know and only their teams know how to run. So... But... So who controls that? 00:25:08.735 --> 00:25:28.985 Who manages the output? You can't centrally manage that decentralized resource, second. Third, which I think Yvonne also mentioned, is monitor the behavior of the agents. You gotta watch them, which is a totally new concept. But from managing employees' work, we already know how to do that. 00:25:28.985 --> 00:25:59.715 It's like, how do you take that principle that we've used for years and think of it now in an automated fashion and where it would apply? And then the last area, which sort of comes back to my Excel example, is owning the outcomes. Who's accountable for the outcomes? We teams where we have a contracts team where we have 20 contracts administrators. One of the early career contract administrators built an app to pre process all... 00:25:59.795 --> 00:26:18.140 He went and got everybody's knowledge from the twenty, thirty year contracts administrator and built this system that would do a first pass on the contracts. Okay? We didn't do that in the IT department. They did it in their group. That has to be managed and maintained. 00:26:18.540 --> 00:26:43.000 How does that how does that... How do you keep that going? So there is that business ownership side of it and keeping track of that. Again, I like to use the Excel analogy, at least with the finance department, they understand they own the macros. The last area, which is another area that I don't think a lot of people are talking about, but CIOs worry about it a lot because it's costly. 00:26:43.000 --> 00:27:13.310 The question is what technical debt are you creating? And there's a whole new different type of technical debt, And the first layer of it comes from centralized or distributed platform and what I was already talking about with regards to departments building their own applications. That's the first piece of it. But the second piece of it, and I've had experienced this, we rolled out certain tools where we let teams go off and build apps and share apps. Well, guess what? 00:27:13.710 --> 00:27:59.640 The vendor changed the tool such that we went to the next version of it and none of the old apps converted. Okay? So there's a whole level of when you're rolling out a technology, whether it's a control plane layer tool, whether it's just a user interface app or whatever in the technical stack, the technical stack is still evolving, the data you're using is evolving. So there's a lot more to worry about in terms of what dependencies related to data or other systems or other things in the value chain of delivering that. And then the second piece of it is, once you know that, can you reverse it? 00:27:59.640 --> 00:28:18.685 What's the cost of reversing it? Actual cost, time, whatever. And then the last piece is, which is I think every organization is starting to hit, is do you have the capacity to evolve? I mean, a lot of people have talked about the idea of... Or the cost of tokens. 00:28:18.685 --> 00:28:47.435 If I build something, gonna is be too expensive for the value I'm getting out of it? But the other point is, let's say the technology changes and I need to put something else in place. Either that centralized IT team or that decentralized group that's leveraged, do they have the capacity to keep running what they deployed? These are probably the hardest questions to answer in a fast moving environment. So those are the four I have. 00:28:47.435 --> 00:29:14.195 I just want to close on this. It's sort of an adaptation of Drucker's management theme, which is AI can do a lot of stuff. There's nothing there, and it can produce a lot of output. But it's really how you choose and address the four decisions before of whether it's worth producing and the cost of that. So thank you. 00:29:14.195 --> 00:29:45.705 Thank you so much, Mark. Appreciate it. Calling an AI model or creating your first agent to do a basic task can be really easy with modern tools. But it's a whole another thing to create an agent that's tied into all of the different systems that it needs to access in order to deliver real ROI over time in your organization. That's why I love what SERP API is doing and why I've used it in the past to connect valuable resources into my agents. 00:29:45.705 --> 00:30:30.590 SERP API handles the complexity and delivers search results as structured data that your application can use, whether that's from Google, Bing, Amazon, Walmart, YouTube, or hundreds of other APIs. They also have built in enterprise security and compliance. They're SOC two type two, SOC three, and ISO 27,001 certified, making it a strong option for organizations where security and compliance matter. I encourage you to visit surfapi.com/practicalai to get started with Surf API for free and get 250 searches every month. That's surfapi.com/practicalai. 00:30:32.985 --> 00:31:00.800 So next we have, I... I'm I'm excited to hear this kind of story and journey from, from Darren at AeroCore. Darren is a senior data analytics manager at AeroCore Technologies, which is local here to our surroundings, but has an... Has this incredible technology to perform, washes on jet engines all around the world. So they do this all around the world. 00:31:00.800 --> 00:31:42.375 And so what's been interesting to see is how Darren has been able to, practically roll out agents into their organization, which a lot of the organization isn't building these agents, but certainly benefits from the insights that these agents are are able to put together, from this, you know, wash data and and work orders and other things from from all over the world. So, this is super practical, and, I would encourage you both to, you know, of course, listen, but connect with Darren after his talk as well to to follow-up on that that really practical journey. So welcome, Darren. 00:31:42.450 --> 00:31:49.410 Yeah. There we go. Yeah. So my name is Darren Taillie. I am the senior data analytics manager for a company called AeroCore Technologies. 00:31:50.050 --> 00:32:06.655 As Daniel said, we wash jet engines. We are an incredibly analog company in a digital world. There are no digital components on the wash trucks whatsoever. So my job is to bring some of the digital world to our company. And as Daniel so eloquently said, we are all over the world. 00:32:06.655 --> 00:32:28.945 Right? So we have about 11 domestic sites at the major airports here in The US. Started deploying internationally and overseas, sites in Japan, in Riyadh, and in Brazil. That expansion is growing every year as our technology really grows to help the big airlines in it, you know, especially in a time where your fuel prices are going up. No one can deny that. 00:32:28.945 --> 00:32:53.815 And so that means your airline tickets are going up to reflect that, and we're trying to counteract that as best as we can. To give a quick bit since, you know, this is a very niche thing that we do, again, we wash jet engines. We use a patented nucleated foam technology to penetrate and clean the insides of jet engines. Right? So you got a lot of carbon buildup, a lot of fouling that happens inside these engines. 00:32:53.815 --> 00:33:22.305 Our job is to go in multiple cycles, go in and clean that stuff out. Where that extends to what I do, I look at the before and after engine performance data, determine the efficacy of the wash itself. But looking also into long term trends, looking into predictive maintenance type things, trying to find the bad apples that are out there for fleets of aircraft, and determine how we can help the the major operators. Like Daniel said, this is a very practical presentation. This is what I have done over the last year. 00:33:22.385 --> 00:33:39.260 I think these kind of presentations fit in this kind of format very well. You have a ton of people who talk about how and the safeguards and and what... How important all of those things are. But I think sometimes you get left coming out of here being like, well, what can I do? How do I apply this to my company? 00:33:39.260 --> 00:33:57.255 What do I take home? So I'm gonna give you two very, very clear examples of what myself and my team did in order to bring... Using AI to bring better information and better data to the rest of our team for decision making. So first, we're gonna talk about... So the biggest challenge we have is access to data. 00:33:57.670 --> 00:34:15.065 As I said, we are an analog company in a digital world. How do we get the information to the key decision makers in time for them to make those decisions? The other one being work order verification. I'll talk about that one a lot more. That's a very nuanced problem, but I think it'll probably a little more interesting than the first part. 00:34:15.065 --> 00:34:27.290 So access to data. Does anybody still know write SQL on, like, a daily basis? Yeah. That's actually more than I expected. So, again, analog company, digital world, not a lot of people know how to write SQL. 00:34:27.290 --> 00:34:50.205 Most people don't have access to the database intentionally as deletes happen a little too easily these days. But then that leads to the analyst bottleneck, as we like to call it, which is the request comes to me. I have to go in, pull the data, merge the data, analyze the data, send the results. And we do that for forty hours a week, and we've done no long term growth. We've done no extra tasks. 00:34:50.205 --> 00:35:23.500 We've done no research. All we've done is fill out how many washes did we do in Atlanta last night, 55 times for, you know, different employees. And then that also leads to another lovely coined term, which is workaround sprawl, which essentially is if I... Because, again, as you didn't hear in my title, not developer, not SQL person, so all of this turns into pandas.read s CSV with group buys and location filters. Right? 00:35:23.500 --> 00:35:38.595 So those get very complicated. You look into my Jupyter notebooks and you see cells and cells and cells of just every random pull that I had to do that week. Go to the end of the week, determine what's important, delete everything else. That just gets spread around. No one else has access to that. 00:35:38.595 --> 00:35:54.690 It's not useful to anybody. So those are kinda all the challenges that we particularly faced in our data. To mention also, our data analysis group consists of me and one other employee. So here comes Daniel with his big toy. And first one, so Agent Forge was our... 00:35:54.690 --> 00:36:02.075 Was, you know, really a big key for us to be able to deploy some of these tools out there. Right? So the first one... Alright. Well, if I just give it... 00:36:02.075 --> 00:36:10.715 This file to the agent, it probably can tell me all the answers about all the work orders from the last week, month, year. Okay. Well, then there's MCP servers. Okay. Cool. 00:36:10.715 --> 00:36:20.890 So, like, now I can just query the database directly. Well, you still have to go into agent forge. You still have to go in and and pull the information out. No one likes an extra login. You know, that's really hard. 00:36:20.890 --> 00:36:35.285 Boom. Now you can put it in Teams. And I think my phone's blown up three or four times already with guys using the agent that we put into Teams in order to access the information. So that ability to just ask, very simple text query. You know, how many... 00:36:35.285 --> 00:36:41.045 I forget which one I put up there. Oh, yeah. Return me all the work orders for this particular tail number. There was an issue. Something happened. 00:36:41.525 --> 00:36:49.700 Alright. Here's all the information. Here's the actual query. Here's the results. You know, fantastic for, you know, our team to be able to go in, find that history. 00:36:49.700 --> 00:37:14.685 Right? History is always the most important part. We've gone through so many different systems to be able to see all the work orders that are on that aircraft, when they were, who did them, where they were, and then obviously, all of the technical data that we gather from every single wash. Once they got moved into Teams, very high adoption. Everybody's using it all the way through, and it's getting it's getting a lot a lot of hits and seeing that in the analytics, and it's been great. 00:37:14.685 --> 00:37:41.565 It's it's truly relieved, you know, burden off of our analysis team to do more things that are beneficial to the company. You know, things that used to take thirty minutes, an hour, a week for complex stuff, the system now does in a couple of minutes. If you gave it a really complicated one, you know, this came back in about three seconds. So it does a really, really, really good job of getting the decision makers the information they need. So, yeah, the big one, self-service. 00:37:41.565 --> 00:38:19.340 Right? Getting our decision makers that information that they particularly needed, getting leadership to not have to ping me every time they need an answer to a question, and also kind of freeing up myself and freeing up my other analysts to be able to do these bigger projects, different types, you know, bigger reporting, bigger research projects, bigger experiments through our data, and how important all of that is, but we weren't really able to find the time to do it, and now we're able to go do that. So that's kind of on the simpler end. I know that there's a lot of the ability for people just to kind of connect Claude directly to your data in all those different ways. We talk a lot about safeguards and guardrails and like that. 00:38:19.820 --> 00:38:39.235 We thought that building it through AgentForge with the MCP servers provides those guardrails, also giving you the access that is wholly required. And now kind of the more interesting one. So this is what call work order verification. Right? So hopefully, most of you have not had to spend a night in an airport, but if you ever have, that's when all the maintenance happens. 00:38:39.970 --> 00:39:14.320 So guys are out there on every plane doing a whole ton of work, including our team who work shifts from about 7PM till about 5AM doing these engine washes out on an aircraft. We do anywhere between, let's say, fifteen to 30 washes in a night across the very different sites. You can get about six washes in a night at a particular site. Typically, it is one technician out there doing the wash on the particular engine. He's out there with his phone trying to get all the data that we need him to gather, writing down all the information that he needs to collect, also making sure that he pumps the right amount of fluid into the engine to wash it. 00:39:15.040 --> 00:39:23.295 There's no one out there to validate his inputs... His or her inputs. Sorry. So errors happen. Fat fingering happens. 00:39:23.295 --> 00:39:37.810 In the middle of the night, 5087 looks a lot like 5078. It happens. So how can... But when that comes back at the end of the month, well, we don't have the right billing counts. We don't have the right aircraft types, and we don't have the right numbers. 00:39:37.810 --> 00:40:07.930 What happened there? We have had a long system of using these pictures that the operators have to submit every night, which are the first three kind of shown there on the top picture on the right, which would be a picture of a whiteboard that we've built on the side that acts as a pseudo form that the operators have to fill out, plus taking a picture of the aircraft, and then taking some other pictures of things of the... We call effluent, but it's the the crud that comes out the bottom of the engine. Is that last picture? I'll love to talk about that with you afterwards if you're interested. 00:40:08.730 --> 00:40:41.110 But our real key is, you know, verifying remotely the work order and the wash done every night. So, right, our challenge, and here's what we came up with for that solution. The other side of this would be that we do have a quality control plan in place for someone else to verify the work orders based on the pictures, looking at them remotely, other site leads from other spots, other professionals. We kinda kept noticing that the same mistakes kept being made over and over. The scenario I just described where somebody looks at a five zero seven eight and thinks that it's 5087, logs it in. 00:40:42.070 --> 00:40:51.105 That would be flagged when five zero eight seven comes in the next night, and you're like, well, we just we just did this one. Why is it coming in again? Oh, that's a repeat work order. Okay. Delete. 00:40:51.105 --> 00:40:53.905 Come to the end of the month. No. No. That one was real. It was... 00:40:53.905 --> 00:41:09.330 The other one was wrong. How do we fix that? So this is where a vision model came into place. Luckily, because of our use in AgentForge and some of the work with Daniel, and big thanks to Sharon who really helped me build this and deploy it. So the crews take those photos. 00:41:09.330 --> 00:41:28.905 They upload them in their... In the power app that we use for them to generate the particular work order. Every day at 08:15 in the morning. Agent goes in, scans all the pictures, verifies it against the information put in into the app. You know, the best part of this would be those end numbers are, you know, are pretty big written on the side of the aircraft. 00:41:28.905 --> 00:41:47.615 Vision model does pretty great at returning them. It also actually does pretty great at returning that little tiny number in the top of the tail when we needed to do that as well. So vision model goes in, looks at the aircraft, identifies it, returns it. If it aligns with what was input in the app, checks it off. If it doesn't, it returns a flag. 00:41:47.695 --> 00:42:00.575 That flag gets sent to me and says, hey, something's wrong with this work order. Please go have somebody, a human, go look at it and see what's going on here. Sometimes it's the wrong picture. Sometimes it's a cloud or a dark night, a bad picture, something like that. Compare and flag. 00:42:02.210 --> 00:42:17.875 That's the exact prompt that I wrote. I... Sharon did help me with that as well. But just kinda back to the, you know, how many times I had to go through and simplify that prompt to make sure that it didn't say, well, I think this aircraft is blah blah blah. I said, no. 00:42:17.875 --> 00:42:30.060 You need to return n and these numbers, and that's it. And how many times we went back and forth fighting about that to make sure that it was correct. But the results... So this was wildly successful. So successful that it has been... 00:42:30.220 --> 00:42:50.585 It goes out to leadership every day at noon with a list of all the work orders done in the previous twenty four hours across every site, both commercial and in the military. It also then flagged... Was expanded to flag other abnormal procedures, other text query... Or text inputs that the operators put into the app to say, okay. Well, I had this issue. 00:42:51.545 --> 00:43:17.385 We worked around it. We fixed it, but just so that everybody knows we had this issue during the wash. Or things like we did this for a smoke and fume resolution that the operator asked us to do, or we missed a cycle, or we missed a wash. All that information gathered. Agent also now goes through those notes, flags them for anything related to smoke fume events, flags them anything related to abnormal procedures, and then goes through and does a whole bunch of other checks. 00:43:17.385 --> 00:43:34.970 Right? We found that the same things were being missed over and over again by the human QC going through and checking these work orders. You know, we work across the middle of the night. There's weird things with what happens when work orders cross midnight, and all of a sudden work orders start taking thirty six hours, and you're like, well, no. That's that's probably not true. 00:43:34.970 --> 00:43:46.265 It was probably four hours. It just got crossed over midnight. How did that happen? So checking all of those type things. So again, like I said, wildly successful, expanded to all work orders. 00:43:46.985 --> 00:44:13.735 Flags are currently just set and tracked, and the team goes through and resolves them as needed. We are building in a resolution system as we speak. So those will actually be flags that need resolution by a human to go through and check and either identify that they are real, fix the problem, or say that they weren't really a problem at all. The biggest result of this, I think, would be a... An op, as we talked about in the last presentation, which was fantastic, where your AI agents fit. 00:44:13.735 --> 00:44:28.110 You know? And this was a big workload reduction and a better outcome in both of these cases. Typically, at the end of the month, we will have a big meeting between finance, data, operations to say, okay. Let's make sure we got all the wash counts right. Let's make sure that all of the... 00:44:28.670 --> 00:44:38.875 Every work order is what we thought it was. Every, you know, all the billing goes out to our customers correctly. We canceled that meeting this month. There were no incorrect work orders. There were no re... 00:44:38.875 --> 00:45:08.275 There's no reason to have to sit and go through everything. They're talking about five, six person hours to go through that meeting, and instead, you kind of just went off and did what we should be doing, development, strategy, looking into the future. So in summary, again, a very practical intention of what this presentation was. We built a text to SQL agent in order for leadership to have better access to our data. Also, to alleviate the load off of our analysts of answering every simple question that came through. 00:45:08.915 --> 00:45:30.535 That also allowed for, you know, better context for our leadership, also better time for the analysts to go do more strategic tasks. Our vision verification has been, like I said, wildly successful in reducing the amount of work required for everybody to go in and fix work orders. And then the daily digest that comes from that. Right? Here's the track. 00:45:30.535 --> 00:45:37.575 Here's every work order. Here's every issue we had last night. What needs to be fixed? What doesn't need to be fixed? What can we do better? 00:45:37.575 --> 00:45:55.140 Upcoming projects. This is fun. So obviously, making the operators... Making sure that all the information that they put in is correct. So looking into more visual models of gathering that data by itself, maybe not requiring the operator to put as many things into the app as required, or not doing it at all. 00:45:55.185 --> 00:46:08.625 And say, I gather all the information from the truck, from the pictures, from other methods? And then the other one being the... Let me try this. Vision of truck valve temperature pressure settings. So, again, like I said, analog in a digital world. 00:46:09.180 --> 00:46:28.925 There's no computer on that truck that washes the engines. It is a guy out there with mechanical ball valves and switches, and that's it. So we wanna make sure that the right settings are getting put in, but again, don't wanna complicate the truck. So can we use AI vision in order to ensure that all the right settings are put into the truck every night? Thank you. 00:46:28.925 --> 00:46:38.930 Thanks, Daniel. Thanks, everybody. We've had a wonderful journey in the AI space, specifically with the Prediction Guard team, and for me and and my team. So, just a thanks to you. 00:46:38.930 --> 00:46:47.730 Thanks, Darren. Yeah. That was great. So so wonderful. For all of you, I mean, we've got a lot of Practical folks here. 00:46:47.730 --> 00:47:12.010 You know what it takes to get something from idea into daily use, and Darren's done that and got that into daily use in their team's environment. It's being utilized. The value is realized that this is this is the anti, you know, the MIT study. It's like ninety five percent of, AI pilots fail. Turns out we just need more Darren's, I think. 00:47:17.015 --> 00:47:37.380 All right. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the AI developments, and we would love for you to join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show. 00:47:37.620 --> 00:47:46.925 Check them out at predictionguard.com. Also, thanks to Break Master Cylinder for the Beats and to you for listening. That's all for now, but you'll hear from us again next week.