Subscribe
Share
Share
Embed
The Sensible Entry Point for Your AI Implementation
Research shows the biggest corporate concerns when it comes to implementing AI are exactly what you expect them to be: security, reliability, and ease of use. With the launch of AI CTRL, we’re addressing each of those head on with a suite of offerings that protect your proprietary data, give you diverse public and private model access (including ChatGPT, Dall-E, Claude, and others), and a simple, private interface and API hooks to inject AI capabilities into new and existing applications. And standing behind it is Expedient, an award winning full stack cloud service provider with over two decades of guiding organizations of all sizes and industries through digital transformations and navigating dynamic technological shifts that offers so many opportunities for growth.
Join Bryan Smith, CEO, and Brad Reynolds, SVP of AI, as they discuss how AI CTRL is built to help enterprises enjoy the benefits of AI while deploying it responsibly at scale. They’ll discuss not only the technology itself but also the thoughtful approach you can take to earn quick wins and prove ROI.
A 30 minute live Q+A will follow.
Research shows the biggest corporate concerns when it comes to implementing AI are exactly what you expect them to be: security, reliability, and ease of use. With the launch of AI CTRL, we’re addressing each of those head on with a suite of offerings that protect your proprietary data, give you diverse public and private model access (including ChatGPT, Dall-E, Claude, and others), and a simple, private interface and API hooks to inject AI capabilities into new and existing applications. And standing behind it is Expedient, an award winning full stack cloud service provider with over two decades of guiding organizations of all sizes and industries through digital transformations and navigating dynamic technological shifts that offers so many opportunities for growth.
Join Bryan Smith, CEO, and Brad Reynolds, SVP of AI, as they discuss how AI CTRL is built to help enterprises enjoy the benefits of AI while deploying it responsibly at scale. They’ll discuss not only the technology itself but also the thoughtful approach you can take to earn quick wins and prove ROI.
A 30 minute live Q+A will follow.
Creators & Guests
What is Expedient: The Podcast?
"Expedient: The Podcast" is your gateway to the inner workings of technology and innovation, presented with unparalleled clarity and expertise. Each episode is an invitation to join the luminaries of Expedient along with special guests from the forefront of the tech industry. We delve into the latest advancements in cloud computing, the evolution of data centers, cybersecurity trends, and groundbreaking developments in AI and machine learning. This podcast strips away the complexity of the technology landscape, offering listeners an exclusive look at the real stories of challenge and triumph, innovation and leadership, that are driving our digital future.
But we don't just stop at presenting groundbreaking ideas; "Expedient: The Podcast" is about building a community. It's for the IT professionals charting their course through the ever-changing cloud environment, and for the tech aficionados keen on decoding the future of digital infrastructure. Our episodes provide the essential insights and perspectives to keep you at the forefront of a world in constant transformation.
Tune in to "Expedient: The Podcast" for a deep dive into the technologies and ideas propelling us towards tomorrow. Experience the journey through the eyes and voices of those shaping our technological landscape, all presented with the authenticity, insight, and forward-thinking Expedient is celebrated for. This is not just a podcast; it's your insider's look into the technologies transforming our lives.
00:00:00:00 - 00:00:26:22
Bradley Reynolds
I'd like to welcome all of you to the launch of expedience first AI product, our secure AI gateway. I'm Brad Reynolds, the expedient SVP of AI, and I'm excited to share with you both our product and the philosophy that we believe helps align with helping you deliver quick and measurable value AI within your organization, while also setting you up for long term success.
00:00:26:24 - 00:01:00:08
Bradley Reynolds
This product is a tangible counterpoint to all of the AI hype and buzz out there. It's something your organization can implement within a few days and start showing results. In this presentation, we'll cover getting you out of the starting blocks. and some of the trade offs you are looking at when picking AI related solutions. Our goal is to show you how to start delivering measurable value with AI in short order.
00:01:00:10 - 00:01:30:22
Bradley Reynolds
After the presentation, we're going to have a Q&A session with myself and the experienced CEO, Brian Smith to chat about your specific needs or other general AI topics. So today we're going to cover a little bit of overview of where it kind of the AI market is today. Then we're going to cover why order matters. Why it's important to take a crawl walk, run approach in terms of implementing AI strategies.
00:01:30:25 - 00:01:57:20
Bradley Reynolds
Well, then talk a little bit about AI applications versus AI platforms and some of the trade offs as you're making decisions in terms of your strategy. After that, we'll talk about the expedient secure AI gateway, which will cover the crawl phase of the gateway and initial stuff, as well as some of the walk phase, which is use cases of how you can build applications on top of the expedient secure AI gateway.
00:01:57:22 - 00:02:04:01
Bradley Reynolds
And then we'll leave you with some takeaways that you can use in terms of your future planning.
00:02:04:04 - 00:02:38:00
Bradley Reynolds
I wanted to cover a way or a study from the Boston Consulting Group in terms of how you can maximize the ROI of your AI investments. So as everybody's starting out, what's the quickest way to get value out of AI? Boston Consulting Group did a study last year where they gave 10% of their organization access to an AI chat tool without any specific data integrations or any specific instruction, and told them to use it in their day to day.
00:02:38:03 - 00:03:06:07
Bradley Reynolds
day to day, work product. Huge results, as you can see. So they saw that they had a 40% higher quality in output from their knowledge workers, and a 25% increased output in terms of what those those folks were able to do with the basic crawl phase of AI chat. Huge, huge productivity uplift and relatively small investment time to enable it.
00:03:06:09 - 00:03:42:26
Bradley Reynolds
So it begs the question, why haven't companies rolled out those tools to all employees? Well, the reality is those tools aren't secure, and security is the number one concern that we hear from CIOs in the organizations that we talk to. They have concerns over the leakage about over private data and the lack of security and enterprise controls. There's also another study by Salesforce that happened in the last couple of months that says 50% of your employees are using AI tools.
00:03:42:26 - 00:04:09:12
Bradley Reynolds
Currently. It's a huge gap inside of your organization and essentially a huge, gap in terms of the security posture of what's going on. So as we try and think about this, we're we're, that that is a huge kind of, thing that we need to address. The second kind of point in terms of why we haven't rolled it out across an organization, is that the AI gateway, or.
00:04:09:12 - 00:04:34:05
Bradley Reynolds
I'm sorry that public AI tools are not tailored to your company specific information. They're just a model that's out there that's trained on the general public information. But to make business decisions inside of your organization, you need to be able to couple them with data internally. But to do that, you still have to cover the point number one, which is you have to have the right security wrappers and infrastructure inside of your organization.
00:04:34:07 - 00:04:54:17
Bradley Reynolds
So one people are in the experiments is they're thinking about, let's try this, you know, a copilot thing. The problem with that is, is that you have a bunch of applications and a bunch of dollars that are siloed, and you have no, no standard policy and employees that are bringing their own AI. And for me, AI devices.
00:04:54:19 - 00:05:19:02
Bradley Reynolds
So that kind of dovetails into the third point, which is okay, that's great in terms of an experiment. But when I start scaling this thing out, what's the cost of these systems and most of the systems that are out there in terms of the initial crawl phase of AI, the the chat tools that are copilots are ChatGPT is their take or pays.
00:05:19:07 - 00:05:48:03
Bradley Reynolds
So whether they're 20, 30, $40 a month, it doesn't matter how much your company is using them. You have to pay a relatively high ticket volume. And so as you're thinking about it, it's very important to think that how do we make sure that the AI tools that we're implementing are very much usage based so that as we use up or used down, we're able to kind of scale and and can and condense as our organization changes.
00:05:48:05 - 00:06:16:20
Bradley Reynolds
So that's great. So we need these all of these pieces together. But like the big question is is okay where and how are we going to start with with with generative AI? So I, I use the analogy that's kind of back to some of the documentary films that happened in the 2000 about climbing Mount Everest. So I, I correlate this to the launch of ChatGPT in 2022.
00:06:16:23 - 00:06:38:11
Bradley Reynolds
Those documentaries on and scaling Mount Everest showed the beauty of the mountain, the fact that it was the largest mountain in the world, and all of the trials and tribulations that went along with it. Everybody got excited about Mount Everest and a lot of people decided, hey, you know what? That's something I want to climb. And the same way with ChatGPT in 2020 20 or 2022.
00:06:38:17 - 00:07:01:11
Bradley Reynolds
We found that, everybody was excited about the possibilities of AI. but it was a relatively daunting mountaintop. It seemed very accessible, almost like a mirage, when you actually started peeling back the layers of that onion. You found out that there's a lot of steps in the middle. So if you were going to climb Mount Everest, you have to have a training program.
00:07:01:14 - 00:07:19:27
Bradley Reynolds
You have to have a diet program. You have to climb small mountains, work your way up, and even when you get there, you have a lot of base camps and higher camps and things that you have to get through to get to the pinnacle. And so a lot of people lost their motivation and ended up sitting on their hands and deciding that that wasn't a thing for them.
00:07:20:00 - 00:07:44:15
Bradley Reynolds
Well, the reality is the promise is there. But the important thing is that you have to break this process down into bite sized steps to be able to achieve it. So the beauty is that we all see the possibility. The difficulty is we need to break those possibilities down into small incremental crawl walk, run steps before we can actually get to the summit.
00:07:44:18 - 00:08:23:22
Bradley Reynolds
So, we can help you with that training plan because the our philosophy about how we approach AI is to give you a starting point, give you the training plan that gets you more and more experience with it, and then eventually gets you to the summit of implementing AI widely across your organization. the goal here is to give you an easy button that helps you along the way so that you can see incremental and reinforcing short term, short term wins that also position you for long term strategic value.
00:08:23:24 - 00:08:43:17
Bradley Reynolds
So how do you end up on the winning side of the AI? How do you end up at the summit of Everest? Order matters. It's really important not to just dump yourself off at Base Camp two on Everest and decide you're going to climb to the summit. You have to start in terms of training and the basics and the building blocks.
00:08:43:24 - 00:09:15:00
Bradley Reynolds
So our philosophy, which you'll see through the presentation, is a crawl, walk, run approach. And executing in that order allows you to provide incremental value to your organization. The first step is to empower your users, your employees, with access securely to the right AI tools. Then, as after you do that, you can then kind of bring data in from other sources inside of your organization.
00:09:15:00 - 00:09:48:01
Bradley Reynolds
Because remember, you built a secure platform and framework to build off of. After that, you can simplify the implementation with an adaptable platform. so it's very important when you're implementing an AI solution to be able to adapt to the changes in the AI ecosystem, because it's very difficult to predict exactly which technologies in which vendors and which capabilities are going to be important in one month, six months, 12 months, or five years from now.
00:09:48:07 - 00:10:18:01
Bradley Reynolds
So making sure that your implementation is strategic, adaptable, and able to work with the changes in the AI landscape is extremely important. And then the last part is to make sure that as you're making these initial investments where you're learning about AI and getting experience with them, you're keeping it as usage based as possible so that you're able to appropriately invest without having to make large take or pay commitments to AI.
00:10:18:03 - 00:10:40:09
Bradley Reynolds
as you're as you're in your learning prospects to take this back to something from the mid to mid 90s, mid 2000, when internet was first taking off. At that point there was a wide disparity of search engines and web browsers and all kinds of technologies that you could possibly use at that time with all of those entrants.
00:10:40:10 - 00:11:09:26
Bradley Reynolds
It was very difficult to figure out who was going to be the winner long term. The early days of the browser war. It was Netscape. Everybody thought Netscape was going to be the winner. Who was using Netscape now? So as you're thinking about your AI strategies, you have to think about is that how do I hedge my bets across lots of different capabilities and so that I know that I have capabilities with the winners as it is, as they as the market continues to evolve.
00:11:09:28 - 00:11:35:19
Bradley Reynolds
So thinking about that and thinking about the previous slide of all the different applications that access the internet, it's important to talk about the dichotomy between, platforms and applications. So an easy thing, like in the previous slide is to pick certain applications and say, hey, this is going to be the application that's going to be the winner.
00:11:35:21 - 00:12:06:04
Bradley Reynolds
But it's a short sighted decision, because what happens is you end up with a bunch of different silos of applications that aren't all interrelated and don't necessarily have the same security posture, framework and adaptability on top of them. You end up just creating a larger problem than you are able to, manage long term. So our suggestion to you is to think about AI in terms of a platform.
00:12:06:04 - 00:12:37:15
Bradley Reynolds
First, that then applications are built on top of. So, you know, the embedded control that you have inside of a single platform allows you to have a single posture that works as you start building things up inside of your organization. And you can have this, a singular governance framework and all that type of stuff. So the advantage of having a platform versus an application is you have a standardized security foundation, but you also have enterprise controls over it.
00:12:37:23 - 00:13:10:07
Bradley Reynolds
Enterprise controls, like the ability to observe the capabilities in in all usage points, whether they're somebody using a chat application, a home built application, or another API generated application or API accessing application. It's also scalable because you have a single point or an easy button that you build all of your infrastructure on top of. That doesn't mean that that easy button is relegated to one particular AI model, or one particular AI technology.
00:13:10:09 - 00:13:49:00
Bradley Reynolds
The platform is adaptable so that you have the ability to use the best and brightest AI technologies as they come and go through one single easy button in one single AI interface. So it's very important now to not be caught in the siren song of applications. We'll give an example in terms of copilot. Copilot is an application that is integrated into a lot of different Microsoft capabilities, but you don't have full visibility and control over all of the different pieces inside of those individual applications.
00:13:49:03 - 00:14:16:19
Bradley Reynolds
So the difficulty there becomes as you scale it out and as you want to do more with AI, you end up with a bunch of different application silos that don't all work together in concert. So how do we solve that? We solve that with product and we solve that with an AI brought in. It's a natural evolution path as you experiment while preserving your economics.
00:14:16:22 - 00:14:44:20
Bradley Reynolds
This is a place where you can learn and experiment economically, and it's built around a crawl, walk, run philosophy. We want to help you get value early and incrementally, and provide you a way to spin that flywheel up as you start building out deeper and more sophisticated AI applications through a single platform. So what are the components of the secure AI gateway?
00:14:44:21 - 00:15:19:04
Bradley Reynolds
The AI platform from expedient, we're going to touch on some of the different capabilities as it relates to the success pieces. Before, in terms of security. It's very important that it integrates into the access and authorization capabilities your organization already has. So whether it's Active Directory one login or some other single sign on component of your organization. But more importantly than that, that's really the stepping stone for the next piece, which is role based access control.
00:15:19:06 - 00:15:56:09
Bradley Reynolds
It's critical in the security aspect of how you implement AI, because not only does role based access control say you have access to certain AI capabilities, but it also in the next phase of your AI implementations gives you access to certain data capabilities. So if you think about it, if somebody in your organization say your health care organization needs access to patient type information or HIPAA related information, and they need that from a data source internally, and they need to have the AI models reason on it.
00:15:56:12 - 00:16:25:18
Bradley Reynolds
Because you have implemented secure authorization. We know the role of that user, and we can scope their capabilities so that they have access only to certain models that are HIPAA compliant and only certain data stores that a health provider should have access to. So essentially, you've taken the whole world of AI possibility and narrowed the scope down to the access control that that person needs to have.
00:16:25:21 - 00:16:52:00
Bradley Reynolds
So as you think about AI inside of your organization and the different folks who might be using it, the notion that being built from the ground up, like the expedient, secure AI gateway, we can scope the role access to that individual to make sure the right people have access to the right AI and the right data. We also have the ability to redirect public AI tools.
00:16:52:07 - 00:17:37:12
Bradley Reynolds
So in the notion before of having employees using unauthorized public tools that don't comply with the audit and security framework of your organization, those can be blocked and redirected to the expedient AI tool so that you're essentially taking something like open AI or copilot and directing it over to the expedient AI tool. So you gain control over the access level in terms of adaptability, the notion that we mentioned earlier in terms of the multiple model support is really important long term, because it's not within the purview of most enterprises to understand the whole model, model, ecosystem and AI ecosystem.
00:17:37:15 - 00:18:04:18
Bradley Reynolds
Expedience job is to curate a select set of models that you can access that have different qualities so they fit for your application. Some models can deal with large amounts of text. Other models can be HIPAA compliant. Other models are great texts summarizes. So we will provide that buffet of models and give you access through that single easy button endpoint to access that whole buffet.
00:18:04:20 - 00:18:26:27
Bradley Reynolds
And the important part of that is you don't have to stay on top of it. We'll qualify that and and be able to filter it so that you're able to use the best models available. And as those models change, as new versions come out, we're on top of it, making sure that you have access to the best models available in terms of security.
00:18:27:03 - 00:19:05:03
Bradley Reynolds
We also have the ability to block PII information as well as company specific confidential information. So the notion of making sure that you have controls and capabilities over what is being submitted out to AI so that you can protect confidential information or PII information for your company. Additionally, we have observability capabilities built directly into the gateway. So this is auditable usage logs which tie both into cost management and understanding ways to improve and audit compliance inside of your organization.
00:19:05:05 - 00:19:32:20
Bradley Reynolds
So this becomes very important as you scale out AI, you have visibility both into how your users are interacting with it, but also how your applications are interacting with it. Important as you scale out your costs and important as you scale out your ability to, improve your operation as you interact with the kind of AI applications in terms of some of the adaptability components there, there.
00:19:32:22 - 00:19:54:23
Bradley Reynolds
We have both the ability to access the secure AI gateway through a chat application that we provide. So think about it as a similar to ChatGPT front end. So your your users and employees will have the ability to have a very familiar interface to access their AI models and their AI, their AI enabled data on the back end.
00:19:54:26 - 00:20:21:01
Bradley Reynolds
But that application actually lives on top of the secure AI gateway. It accesses the secure AI gateway through API calls in the same way your own internal low-code. No code applications or, you know, purpose built applications can access it. So the gateway is that framework of security and control that sits under any application that you build, including the chat framework.
00:20:21:03 - 00:20:49:28
Bradley Reynolds
And then also importantly is that the economics of the secure AI gateway are tied to usage. So as opposed to having a large take or pay, where if you had a thousand employees, you'd have to pay $30,000 a month for copilot. We have a platform fee and then everything is uses based after that. So you don't have to worry that you're you're paying for a whole bunch of AI capabilities that your organization isn't yet ready to use.
00:20:50:01 - 00:21:12:00
Bradley Reynolds
What you're paying for is a platform with the usage based pricing model on top that allows it to scale up as your needs and capabilities scale up in terms of AI. So I wanted to go through some of the deployment options of how does this thing look in the wild and how you can step forward with a crawl, walk, run approach after the deployment options.
00:21:12:02 - 00:21:40:18
Bradley Reynolds
Well, to give you a quick demo of some applications that we've built internally on top of our own secure AI gateway, to give you a sense of how it looks and what the impact was. So in terms of the secure AI gateway, we call it level one. It's an API access or chat interface with a confidential data filter built on top of this with a PII redaction and, confidential information redaction.
00:21:40:21 - 00:22:06:00
Bradley Reynolds
Additionally, you'll see in the middle the security control layer. This is what was referenced when it was the notion of redirecting public AI tools, so that everybody has to be routed inside of the organization to either the API access or the internal chat interface. The core part is the secure AI gateway, and we're hoping that your organization can get used to AI via chat.
00:22:06:02 - 00:22:35:07
Bradley Reynolds
That's like the killer app to start out with. But once you get that enablement and buy in and democratization of AI through your organization, you now have an easy format to start building applications, because that chat that you're using is built on top of the secure AI gateway, which is the AI endpoints. You can use to build ten lines of code AI applications to enable workflows and processes to get solved within your organization.
00:22:35:10 - 00:23:16:21
Bradley Reynolds
You can choose any different model, from our model or our models that we offer. And if you want it all bundled together, we could tie that all into the API and load it all together. And what's coming next down the road? So with the secure AI gateway and the chat application built on top of it, the next step as you build out, which we'll call more of the lock phase, is the ability to tie data from your organization securely into the secure AI gateway through a bunch of data connectors and the ability to host your own internal, customized, controlled private models.
00:23:16:24 - 00:23:47:14
Bradley Reynolds
So this is like the next step. But as we talked about earlier, order matters is very important to start economically with crawl phase, which is get folks inside of your organization experience with the chat capabilities, and start writing lightweight applications on top of the secure AI gateway. Once you start doing that, then you're now going to open up the capabilities to being able to integrate with external data sources.
00:23:47:14 - 00:24:23:28
Bradley Reynolds
So this is your proprietary data as well as private models that could be hosted locally for you and are directly, dedicated to your organization. Why does that order matter? It matters because if you don't have the right secure foundations to build on top of, you shouldn't be tying in your proprietary data to it. So the secure AI gateway and getting capabilities and comfort with that is the first kind of component of the deployment option, so that you're ready to go to the lock phase, which is the next phase coming down the pike.
00:24:24:00 - 00:24:57:09
Bradley Reynolds
Here's a demo of our chat user interface. So as I mentioned, this is an application that we provide for free on top of the secure AI gateway so that your users will have a ChatGPT like experience to be able to access model securely as well as down the road. Integrate to your own proprietary data set. As referenced in that earlier BCG report, in terms of the quickest time to value implementing this chat user interface.
00:24:57:09 - 00:25:12:28
Bradley Reynolds
On top of this, the secure AI gateway is the quickest way you can get to value to start showing value inside of the organization and ROI to then spin the flywheel up for future applications.
00:25:13:00 - 00:25:34:29
Bradley Reynolds
So here's a basic log to how we integrate, or how you would log in to the chat interface. And this would be branded for your organization. This just happens to be the one that we use internally. So you can see the acesso options in terms of the Active Directory or one log in. Now when you actually log in you have the ability to change.
00:25:34:29 - 00:25:57:22
Bradley Reynolds
You have some capabilities. Change the model, set the models that you have access to as a user are determined by that role based access control that I talked about previously. In this case, it's it's one of our executives that's logged in. They have access to the entire model set. But the model set has sets of public models, but it also has sets of private models.
00:25:57:23 - 00:26:23:03
Bradley Reynolds
You can see in here the expedient particular models are privately hosted models. So in certain circumstances, say if was a technical support representative or somebody in the health area, you might want them to, be in a private model only, capability. But because you've integrated into Active Directory, you have the ability to limit that by role. So this model set, this is the full buffet menu.
00:26:23:03 - 00:26:49:15
Bradley Reynolds
You can tune and limit it by the roles that you have specified. and so the next kind of piece of it is the administrator has the ability to select whether PII filtering is on. So this is, as we mentioned earlier, you the ability to filter PII or company confidential information by default. This is the admin level.
00:26:49:21 - 00:27:18:12
Bradley Reynolds
But you can also buy a rule based access control. Say whether this applies to all users, a specific set of users roles of users or having the ability to toggle it on and off based on a particular kind of level of user. So, for instance, if it was a technical support representative, you might say that PII filtering is always going to be on because you can't possibly have public or confidential information leaking out.
00:27:18:14 - 00:27:31:15
Bradley Reynolds
Here's an example of how that works. So we input a prompt into the system. And we see things like account numbers that are in there.
00:27:31:17 - 00:27:56:10
Bradley Reynolds
So we it will it will be able to redact that type of cap, account information, on the fly, and then give them a, an ability to pop up to say that they, that that information was redacted. Additionally, you have the ability to change the model. So if, for instance, you were running a private model, you could put in the same prompt.
00:27:56:15 - 00:28:24:05
Bradley Reynolds
And because it stays locally, you find that the private model is able to process it and not have to redact that information. So the rule based access control can control both the model access and what the files, can be accessed by the each user. Additionally, inherent inside of the secure AI gateway is the observability capability. So we log all transactions back and forth.
00:28:24:08 - 00:28:49:18
Bradley Reynolds
and we we are able to align with your organization's data retention policies. It's very important that we have the ability for you to cost manage your, your usage of your users, but also cost manage it so that you have an understanding of what your spend is going to be in terms of using AI capabilities. And as I said, our product is usage based in terms of its, how you access these models.
00:28:49:23 - 00:29:19:16
Bradley Reynolds
So staying on top of it lets you make sure that you're in the right usage bracket for the amount that you're you're using, but secondarily past the economic drivers of it. We're able to monitor the adoption of the, secure AI gateway. So as you're rolling this out inside of your organization, you can see which groups are using it heavily, which groups might need some enablement or help, and essentially how to increase the empowerment of your organization.
00:29:19:20 - 00:29:47:18
Bradley Reynolds
So the observability helps on both the audit side, the cost management side, and increasing the empowerment and adoption inside of the organization. Still in the crawl phase, the next phase that we're going to get to is the concept of some use cases or one specific use case that we implemented at expedient in terms of a low code interface via the secure AI or the secure AI gateway API.
00:29:47:21 - 00:30:09:01
Bradley Reynolds
So as we saw previously, we were accessing it through the chat interface, and that was just a bunch of API calls. In this case we're going to say, okay, let's take it a little step further. Let's take it into the walk phase of crawl, walk, run and start to build some more custom applications, but do it in a lightweight fashion.
00:30:09:03 - 00:30:41:23
Bradley Reynolds
So we're going to show you an example of what we call the rag ticket Summarizer. So Rag stands for retrieval augmented generation. But simplistically it's pulling in proprietary data from our company and then melding it to an AI model so that we're able to, come out with some sort of a solution. In this case, our CEO asked me how hard would it be to create a tool where everyone on the escalation list of a ticket could get a ticket summary.
00:30:41:25 - 00:31:07:26
Bradley Reynolds
Now, normally, our staff creates an executive write up in a technical summary to get the team up to speed quickly. If an issue gets escalated, a lot of things probably happen in the tickets lawn. So there's a lot of information to summarize. And we were doing this in a manual fashion. So we were looking to auto generate a summary from both a business perspective which is the left column, but also a technical perspective from the right column.
00:31:07:28 - 00:31:42:20
Bradley Reynolds
In just a couple of hours. We had a low code tool mocked up using our internal low code system called tool and the secure AI gateway API on the back end. And now in a few seconds, you can get an executive summary or a technical summary just by pressing a button. And so this saved us a significant amount of time, but also shows you the power of connecting into your internal company data and pulling that into the, the AI system so that it can summarize it on the fly.
00:31:42:22 - 00:32:10:01
Bradley Reynolds
Here's an example of how the back end of that implementation actually looked. So we pulled data from a data source. So in our in our case it was our data gateway to our support management console. We then used our own AI gateway internally which used a private model so that data wasn't going out to the open internet. And then we used our low code environment so it could be a low code environment.
00:32:10:01 - 00:32:38:14
Bradley Reynolds
Or we could have just written an application in Python on the fly in 20 lines of code. but those things combined together was what sits under the hood of that low code interface. So to give you an example of the second level. So the walk level of ROI. So we already talked about crawl. Crawl meaning get high ROI with relatively low time investment by implementing the chat level on top of secure AI gateway.
00:32:38:19 - 00:33:01:21
Bradley Reynolds
Now we're talking application level on top of secure AI gateway. So from conversation from Brian to myself took about two hours to implement a rough version of this. Ten hours to train and have conversations as to is this the right format? Is it? Is it looking good? Are the people who are going to consume this? Happy with it?
00:33:01:24 - 00:33:31:06
Bradley Reynolds
And after that it was done. And so we took a look at how much time we were spending human wise, to manually create these executive and technical summaries. And we looked at we were saving about $2,400 or 20 400 hours annually in terms of shift hand-off as well as executive escalation. Not to mention the fact that our quality and consistency of summary increased greatly.
00:33:31:08 - 00:34:00:22
Bradley Reynolds
How much does that cost us or save us $168,000 annually, based on the labor costs of 20 400 hours in the in the support system of our organization. So the key here is, once you get past the crawl phase of getting folks empowered and capable on the chat interface, you're now opening the door to a lot of these very lightweight use cases that you can implement on top of the secure AI gateway.
00:34:00:25 - 00:34:27:03
Bradley Reynolds
And this is one of a a multitude of use cases that we've implemented internally. But as you're looking at it, as business leaders and process owners internally to your organization, this is the walk step of crawl, walk, run. What are the lightweight use cases for existing processes? We're already spending a lot of manual effort on. And how do I solve that with AI?
00:34:27:05 - 00:34:52:27
Bradley Reynolds
But I have to do it with the secure AI gateway underneath because it has to be the have the secure, capable, auditable underpinnings to the to the capability. So one of the questions now is with those use cases, what's different about the experience? Secure AI gateway vis-a-vis things that are out on the market. So we have a couple of key points here.
00:34:52:29 - 00:35:19:08
Bradley Reynolds
Think of us as the easy button that accelerates time to value. But some of these points are points that are highly differential in the marketplace. Key point at the top open. Hedging your bet on multiple technologies. So the notion that by using the secure AI gateway, you're getting access to a whole bunch of AI tools on the front end and a whole bunch of data capabilities on the back end.
00:35:19:10 - 00:35:44:22
Bradley Reynolds
So the notion of I have to pick the winner at the starting gate is a very difficult process. Like picking the winner of the Kentucky Derby is very difficult. But what if you could just bet on all the horses? So that's the way that we've built our technology. We're going to put all of those highly capable AI horses in the in the starting gates for you, and help shepherd you to the ones that fit your particular use case.
00:35:44:24 - 00:36:10:18
Bradley Reynolds
And when you start building out applications at scale, you'll find that different horses and different components fit your capabilities better than others, and you have access to all of them. So the notion that we've essentially, hedged your bets in terms of your AI technologies and as they grow and we stay on top of that as an organization to make sure all the right horses are in the gates for you as AI grows and transforms.
00:36:10:20 - 00:36:48:04
Bradley Reynolds
The second key differentiator is the role based access control, deeply built into the secure AI gateway. So it's role based access control. We know that we can scope access both from an AI brainpower perspective as well as a data perspective, to the users and applications of the system. So the notion that that can be done in a way that you have firm control and observability of what models and what data do they have access to and which users have them, is something that's in it is is incumbent to designed from the ground up.
00:36:48:04 - 00:37:10:09
Bradley Reynolds
And that's part and parcel to the secure AI gateway and any applications that you build on top of it. Being able to integrate PII and confidential data filtering in at the beginning is important as well. because you need to make sure that that you have the strict security controls over what data could potentially be leaking out to the internet.
00:37:10:12 - 00:37:33:25
Bradley Reynolds
Bringing AI to your data is also an important thing. So a concept in the secure AI gateway, which will be launching in the next few months, is the ability to connect to data of yours where it lives. So a differentiator relative to, say, hyperscaler is the notion that a hyperscaler wants all of your data brought to them, and then their I can act on it.
00:37:33:27 - 00:37:58:11
Bradley Reynolds
Our philosophy is we will access your data where it lives and be able to pull that into the system in the same way that we did the use case example on our ticket. Summarizer. We don't want to create a data movement, project for you. We want to help bring AI to the data where it lives, so you can start getting value out of your data via AI as quickly as possible.
00:37:58:14 - 00:38:22:26
Bradley Reynolds
And importantly, is pay for what you use. So the notion is we have a platform, but we have on top of that the usage of the AI is based on a, it's on a usage base. So the notion that you don't have to pay a large take or pay amount just for the access to the AI system, and hope that folks inside of the organization are using it.
00:38:23:03 - 00:38:50:16
Bradley Reynolds
We have it structured so that you can be assured that you're getting what you paid for when it comes to your AI investment. So how can I take advantage of this now in terms of our platform is secure AI gateway. We have two components one one component is the platform pricing. And so this is small medium large based on the price of or the size of your organization.
00:38:50:19 - 00:39:16:06
Bradley Reynolds
And the important part here is that this is a flat fee. So this is just getting access to the secure AI gateway. Chat comes for free on top of that. and then you then have the ability to start using the AI capabilities on top of it. Each of those levels includes a bunch of AI, usage already built in so that you're not starting out of the gate buying usage.
00:39:16:10 - 00:39:38:00
Bradley Reynolds
But as you increment up and start using more, as you start adding on applications that have additional AI usage, we have plans that allow you to increment up your spend directly proportionate to your usage of the AI system. And we don't expect some some large take or pay kind of commitment in terms of how we bucket our usage for the system.
00:39:38:03 - 00:40:05:22
Bradley Reynolds
So in summary, for the secure AI gateway, in terms of time to value, in terms of getting beyond all the buzz in the hype of AI, you're quickest to time to value is to give everyone in your organization access to chat. That's how you empower your first crawl phase of AI. But the counterpoint to that is you have to do it economically.
00:40:05:24 - 00:40:53:29
Bradley Reynolds
So the notion of having a platform with usage based pricing on top of that is a way that you can crawl and get access, give access to everyone inside of your organization without breaking the bank. So additionally, take away order matters deeply. Start with crawl, then the walk, then to run crawl phase is get folks using AI chat capabilities inside of your organization, but with an eye on the fact that as they start using it, as they start realizing what's possible with AI, they're then going to be thinking about what are the use cases that fix things inside of my day to day.
00:40:54:01 - 00:41:17:05
Bradley Reynolds
They'll use the chat to figure out whether it works and then they'll then you'll start operationalizing that by building lightweight applications like the ones that we showed, that have relatively large ROI, relatively small time investments. That starts to be in the walk phase of things. As you do that, you start to get experienced and then you become going on to the run phase.
00:41:17:05 - 00:41:57:08
Bradley Reynolds
But the important part about secure AI gateway is that we give you the crawl and the walk phase of that component, because it's very important to start slow and work your way up incrementally and economically. Platform versus application. Starting with a secure platform that you can then build applications on top of and ensure that you have a cohesive and secure framework that you're building everything on top of is preferable than accumulating 30 different AI applications that you don't have full control over.
00:41:57:09 - 00:42:20:20
Bradley Reynolds
Visibility, and then you end up having some sort of an operational catastrophe because each of them have their own security policies, procedures and capabilities. So think about it deeply that, while the crawl space may be getting empowerment around, chat the phases later as you scale up in terms of the platform is the capability in terms of room to grow.
00:42:20:21 - 00:42:58:09
Bradley Reynolds
And so that platform kind of capability and I the last kind of takeaway for your organization is that you want to keep your options open in terms of AI technology. The smartest AI technologists in the world can't tell you what the state of AI is going to be in six months, 12 months or five years from now. So as you're making bets, it's better to make a bet on a platform that gives you access to all of the options, as opposed to choosing one one technology that locks you into a particular way of operating.
00:42:58:12 - 00:43:20:26
Bradley Reynolds
And so in that way, you're strategically positioned to climb that mountain. So thank you. That is the, the the launch of the expedient secure AI gateway. And I'd like to open it up for some questions.
00:43:20:29 - 00:43:51:10
Bradley Reynolds
Okay. Let's let's take a look at the questions. So we have a question from, about does any of the, part of the AI control product help clients produce useful workflows through AI, or is that up to the clients to figure out and what advantages? so we'll take that as as one. Ryan, did you have any comments on that one?
00:43:51:12 - 00:44:11:15
Bryan Smith
Sure. So I think, you know, part of it is, when you talk about the crawl, walk, run is a lot of the crawl phase is trying to manually figure out those components, because when people try to jump right to the workflows, that's where we've had challenge or we've seen challenges where people try to go too fast and they're anticipating what they think they need.
00:44:11:15 - 00:44:30:24
Bryan Smith
And they often, you know, find some roadblocks where it's not delivering the underlying results. So part of the way that we help in the workflow is, in the way that we've rolled it out. So you can manually test things when it comes to actually, you know, writing the software, for, your own company and, building those components.
00:44:30:27 - 00:44:44:24
Bryan Smith
And we wouldn't be implementing, or making, changes into your software. And we do have partners that can assist with those specific components. You know, we would have all the tooling, that would really enable, those capabilities, other things you did.
00:44:44:26 - 00:45:08:18
Bradley Reynolds
Yeah, certainly. So, most organizations have developers internally that are experienced with Python. And so in terms of the secure AI gateway easy button, it's it's a set of API calls. So which model you call and what how do you pull data on the other side. It's very accessible to somebody with some basic programing skills. They don't need to be AI experts.
00:45:08:25 - 00:45:37:15
Bradley Reynolds
So when you're looking at building out those early workflows or those low code, no code capabilities, you might be able to start solving some of those problems, like the ticket summarizer in 10 to 20 lines of Python code. So being able to spin up some use cases is relatively easy. On the lightweight workflow side. At that point, you'll start to get more experience with what you need and then figure out, okay, do I want to build up the more sophisticated applications internally?
00:45:37:15 - 00:46:04:19
Bradley Reynolds
Or to Brian's point, do I want to engage an AI specific partner to help me build some of those workflows? For me? okay. Another question. What advantages does AI control have over using free chat, GPT or Microsoft Copilot, or any other new offerings that pop up? Yeah. So, there's, there's a lot of great consumer products out there.
00:46:04:19 - 00:46:30:11
Bradley Reynolds
And then obviously copilot is is more of an enterprise product. So I guess it depends on what your goals are. If the goal is you want to have an AI strategy for an enterprise organization, you need to have a level of security controls and framework around it. So like the free versions of X, Y, or Z aren't going to meet your needs other than the thing that it could me would be just to get you a little bit of experience with it.
00:46:30:11 - 00:46:57:10
Bradley Reynolds
But kind of when you want to take it to the enterprise level, you have to have a secure AI gateway type framework around it. So that's probably on the free side on the side of well, what about copilot? The difficulty with copilot well, one, it's really expensive. It's a take one of those take or pay scenarios where if you give it out to a thousand employees, that's $30,000 a month and, are all 1000 using it.
00:46:57:10 - 00:47:27:05
Bradley Reynolds
What you're paying for it, whether they're using it or not. So that's a that's a difficulty with it. But maybe if you peel that onion back a little further into copilot is you don't have a lot of control about what's going on. And to the last takeaway point, I said, you want to keep your options open. The problem is Microsoft is really married to open AI, and it's actually kind of opaque as to how copilot integrates into open AI internally.
00:47:27:08 - 00:47:51:18
Bradley Reynolds
So if you said, hey, why we really like this, to go to another competitor, Gemini or X or Y or Z, you don't have any of that capability. You're just like, well, I guess whatever PowerPoint has built into it has to be the thing that works. And what you're going to find is that's not adaptable enough. For as things change, you're making a bet based on what you think the future is going to be in 4 or 5 years.
00:47:51:24 - 00:48:05:17
Bradley Reynolds
And it's those those are tough bets for anybody to make. And so I find your, your you're too monolithic, in terms of copilot, let alone the kind of expense model that's associated with it.
00:48:05:20 - 00:48:30:27
Bryan Smith
And the only other thing I would add is if you compare AI to really any other enterprise application, if you're a company that's running an Oracle and a CRM, and you wouldn't encourage your employees to go just download and start using, you know, another CRM on their own that doesn't have all the controls and the buying and stuff from, the business and or having a different one by department.
00:48:31:00 - 00:48:49:06
Bryan Smith
it just wouldn't be very efficient. And so I think that's the first, first is the mindset pieces that enterprises need to think of AI as another enterprise application and put the right type of kind of control, security, everything around it so that they feel comfortable rolling out to their entire base. And that's that's probably the biggest difference.
00:48:49:06 - 00:49:15:26
Bryan Smith
And specifically in their the, the function where it can our, tooling can filter out confidential and private information so that you feel comfortable rolling it out to a larger group of employees. And from, like your example, you know, the thousand person company that that's rolling out. Copilot. And we've seen those situations where this type of platform is, you know, 70, 75% less costly, than trying to do that.
00:49:15:26 - 00:49:21:06
Bryan Smith
So you have a lot more willingness to give a bigger pool of people access.
00:49:21:09 - 00:49:56:11
Bradley Reynolds
Yeah. And I to that point, the wider you can democratize this within the organization. That's that's how the that's the rising tide that raises all boats because, a little too often the AI, because it's a somewhat complicated technological process, it gets homed into the smartest technological people inside of your company. But the people that know where the problems are that need to be fixed are the subject matter experts who might not be the most technically astute or AI forward folks, but those are the things.
00:49:56:12 - 00:50:17:11
Bradley Reynolds
Those are the folks that you need to get access. They're going to know what problems need to be solved. They use the chat to figure out how those problems can be solved and then parse, you know, package that off to the technology folks to say, hey, let's turn this into a workflow so that we can solve this problem going forward, just like we did with the ticket Summarizer.
00:50:17:14 - 00:50:38:24
Bryan Smith
Yeah. And I think that's actually, you know, some of the enterprise controls around the logging and stuff are can actually be turned back around to into a tool. When you look to do that, that walk or run phase, you know, because you can go back and ask the tooling, to ingest logs and understanding what are the most common, things people are asking which departments are using this most.
00:50:38:27 - 00:50:46:07
Bryan Smith
And to help focus your development time for where you're going to have the biggest adoption and in turn, impact in the business.
00:50:46:09 - 00:51:09:18
Bradley Reynolds
Okay. So we have a question about, how do I apply AI in a systemic decisioning system loan management process and workflow automation? So is a great question. And it leads really into the crawl, walk run type conversation. because.
00:51:09:20 - 00:51:31:14
Bradley Reynolds
the person who asked this question obviously sees the Everest potential of AI in terms of like transforming their organization. But the question is, is, okay, those are somewhat complicated things to do. How do you break that down into bite sizes to go from? And so the way that we suggest attacking that is to in the same way that we attacked our ticket.
00:51:31:14 - 00:51:54:00
Bradley Reynolds
Summarizer. So the way that the ticket Summarizer was attacked was somebody brought up this, question about, hey, can we it was Brian, you know, how do we how do we how do we get an executive and a technical summary for some ticket that might have 400 pieces of information in it? Okay, so I didn't know how easy that was going to be to do.
00:51:54:00 - 00:52:18:01
Bradley Reynolds
I just took our chat system built on top of secure AI gateway and started cutting and pasting just the giant ticket and asking it to put things out in this format that we wanted. And I know that's not the systemic decisioning system, but conceptually it's the same thing. So if you take a mountain of data and you give it the instructions of what you would like, hey, I would like it to decide about these things.
00:52:18:01 - 00:52:39:13
Bradley Reynolds
Or here's the parameters you use the chat piece as you're like, cut and paste, almost like a manual test of whether that works. And what you'll find is it often doesn't work exactly the way that you thought it would. And so you're going to be doing a variety of manual processes to kind of hone that in until you can make it manually work.
00:52:39:16 - 00:53:01:26
Bradley Reynolds
Once you can make it manually work to say, like 90%, now you're in a spot where you want to start automating and an hour walk phase would be do some lightweight things to automate. Once you get those automations, then kind of operationalize it. So in terms of the output that this person is asking, that's more in my mind a run phase.
00:53:01:28 - 00:53:20:14
Bradley Reynolds
And so all you do is you say, hey, that's what we want to get to, or that's what we want to see if we can get to what's the crawl phase look like, and then how do we start there? How do we validate and then can we take it to the next step with some of these AI desires that people have and they see the possibilities, some of them are dead ends.
00:53:20:16 - 00:53:38:00
Bradley Reynolds
some of them aren't. And you don't want to invest a huge amount of money building essentially the run phase first before you essentially use a knowledge master, expert or subject matter expert to validate those early phases and then do slow increments up.
00:53:38:03 - 00:53:39:00
Bradley Reynolds
Is that perfect?
00:53:39:02 - 00:53:39:19
Bryan Smith
Yep.
00:53:39:21 - 00:54:02:02
Bradley Reynolds
Or okay. another question. Does this solution include the ability to directly create Microsoft 365 documents or integrate directly into Microsoft 365 apps like Copilot? so, or do we need to license Copilot for Microsoft? 365 on our own?
00:54:02:04 - 00:54:26:08
Bryan Smith
Yeah. Do you let me take that or. Sure. So, you know, it has the ability to interact with your Microsoft documents so that you could, upload, you know, word files or PowerPoints or other things into it and question, get summaries, create, text from that, or take data and have it export. do work on it and then take a large table of data and export it, as, you know, like an Excel file or things like that.
00:54:26:08 - 00:55:07:06
Bryan Smith
You know, I use that on a pretty regular basis. it doesn't tie directly into inside the application. And copilot exists, as, you know, part of the software from Microsoft. So that's, currently unique, to them, you know, there are, talks of opportunities to, potentially have other ways that people could interact with that hook that's tying into copilot, you know, so it really depends on if you need the access inside the application itself versus the way that we look at this is if you're having lots of different, AI tools from Salesforce, from, Microsoft, from different places, I think that personally, it's like, you know, leaving tools all around the house,
00:55:07:08 - 00:55:30:15
Bryan Smith
versus like all the tools in my house are either in the garage or in the basement. And, you know, that's really how I think of this tool. It's like bringing those different AI tools into one place so that when I'm looking to leverage something and if no matter if it's image generation or if it's tech space or if it's, computational or if it's developing code, all of those things, I can come to one interface and select different models for those different purposes.
00:55:30:18 - 00:55:54:00
Bryan Smith
is, you know, the benefit that we see from going that way. And in addition, it's radically lower cost because, you know, you have a flat cost for the organization and then a minimal, usage fee that, and that's only if you go beyond what's included in, in the regular consumption versus, as Brad talked about copilot, you're doing $30 for every employee, no matter if they're using it or not.
00:55:54:00 - 00:56:06:06
Bryan Smith
So makes a lot easier to sell, implemented in your business because it ties in your cost, really to, you know, what, the value you're getting out of it, you know? Yeah. Upfront.
00:56:06:09 - 00:56:28:08
Bradley Reynolds
Yeah, I know, I know why I can't find my tape measure. Yeah. It's not it's somewhere else. But I mean, in all honesty, that's that's a really good analogy that Brian brings up because, don't only think about this as the AI tools that you need access to, but also think about it as the data tools you may need to integrate to.
00:56:28:08 - 00:56:50:25
Bradley Reynolds
So when I look at Copilot in those applications, I look at them as silos to said, hey, I need my PowerPoint integrated into my Salesforce, okay? You know, it's never going to happen. But if you are thinking about, hey, I'm hooking into the secure AI gateway, the secure AI gateway can hook into AI endpoints and different data endpoints.
00:56:50:27 - 00:57:17:05
Bradley Reynolds
Now I have that that common tool storage spot that hooks AI into data. And that's kind of as you're thinking about platform versus siloed application. Something to think about in terms of maintainability, scalability let alone cost. Yeah. So I wanted to thank thanks, Brian, for being on and helping develop this product with me and the whole, team here at experience that's built it.
00:57:17:07 - 00:57:26:13
Bradley Reynolds
and thank you all for attending, and I'm excited to have conversations with you about how the secure AI gateway can help you meet your your business needs.