1 00:00:10,191 --> 00:00:14,931 Bret: Welcome to the first episode of my new podcast, a Agentic DevOps. 2 00:00:15,071 --> 00:00:16,061 this episode. 3 00:00:16,781 --> 00:00:22,481 Is kicking off what I think is going to be a big topic for my entire year, 4 00:00:22,661 --> 00:00:28,811 probably for the next few years around wrangling AI into some usable format. 5 00:00:29,201 --> 00:00:35,491 For DevOps, you probably heard of AI agents by now, or the MCP protocol. 6 00:00:35,611 --> 00:00:38,491 I guess I should just say MCP, since P stands for protocol. 7 00:00:38,821 --> 00:00:43,561 And these two things together are creating potentially something 8 00:00:43,561 --> 00:00:47,491 very useful for platform engineering, DevOps, and that stuff. 9 00:00:47,791 --> 00:00:50,281 it has so much potential that. 10 00:00:50,671 --> 00:00:55,411 In the first quarter of 2025, I kind of thought this was gonna be a big deal. 11 00:00:55,411 --> 00:00:58,861 This was gonna be, uh, if we can figure out how to keep these things 12 00:00:58,861 --> 00:01:01,501 from hallucinating and going crazy in our infrastructure, this could 13 00:01:01,501 --> 00:01:05,941 potentially be the AI shift for infrastructure that I was waiting for. 14 00:01:06,361 --> 00:01:08,401 So started this podcast. 15 00:01:08,461 --> 00:01:12,856 We recorded our first episode at KubeCon at the beginning of April, 16 00:01:12,856 --> 00:01:20,131 2025, and this is gonna be a series of very specific episodes around getting. 17 00:01:21,196 --> 00:01:25,786 Ais to do useful automation and work for DevOps, platform engineering, 18 00:01:25,966 --> 00:01:29,086 infrastructure management, cloud, you know, all those things 19 00:01:29,416 --> 00:01:31,726 beyond just writing YAML, right? 20 00:01:31,996 --> 00:01:35,686 So the, the intro for this podcast, there's a separate episode for intro. 21 00:01:35,686 --> 00:01:38,866 It kind of goes into my whole theory of why I think this is gonna be a thing. 22 00:01:39,166 --> 00:01:42,706 And this episode we really try to break down the basics and fundamentals for 23 00:01:42,706 --> 00:01:44,296 those of you that are catching up. 24 00:01:44,296 --> 00:01:45,631 Because it's a lot. 25 00:01:45,781 --> 00:01:47,101 There's a lot going on. 26 00:01:47,281 --> 00:01:52,031 It seems like We have announcements every day this year around AI agents or 27 00:01:52,031 --> 00:01:54,061 Agentic, ai, however you wanna call it. 28 00:01:54,311 --> 00:01:58,331 I am calling it Agentic DevOps, and hoping that name will stick. 29 00:01:58,781 --> 00:02:01,841 Now, this episode, since it's from the beginning of April. 30 00:02:02,546 --> 00:02:06,416 And it is technically now just getting released at the beginning of June. 31 00:02:06,926 --> 00:02:09,296 We're a little bit behind on launching this new podcast. 32 00:02:09,626 --> 00:02:11,816 Um, I think everything in it's still relevant. 33 00:02:11,816 --> 00:02:13,286 There's just been a lot more since. 34 00:02:13,496 --> 00:02:15,116 And I don't know the frequency yet. 35 00:02:15,116 --> 00:02:17,336 I don't know how often this podcast is gonna happen. 36 00:02:17,576 --> 00:02:19,676 It could be potentially every other week. 37 00:02:19,856 --> 00:02:20,726 It could be weekly. 38 00:02:20,726 --> 00:02:23,756 I just don't know yet because we are not gonna do the same thing 39 00:02:23,756 --> 00:02:25,826 here as on my usual podcast. 40 00:02:25,826 --> 00:02:28,616 If you're someone who knows that one DevOps and Docker talk that I've 41 00:02:28,616 --> 00:02:31,976 been doing the last seven years, that one is still gonna have AI in it. 42 00:02:31,976 --> 00:02:36,146 But this one is very specific and there might be a few episodes that have 43 00:02:36,146 --> 00:02:40,736 syndication or whatever you wanna call it, of the episodes on both podcasts. 44 00:02:41,291 --> 00:02:45,701 But most of the time we're gonna keep the focus of just everything, 45 00:02:45,701 --> 00:02:49,541 DevOps, everything, containers on the DevOps and Docker talk show. 46 00:02:49,601 --> 00:02:54,041 And this one is gonna be very specific around implementing useful 47 00:02:54,671 --> 00:03:00,851 AI related things for Agentic DevOps, or automating our DevOps with robots. 48 00:03:01,331 --> 00:03:04,841 So I hope you enjoyed this episode with Nirmal from KubeCon London. 49 00:03:10,564 --> 00:03:11,294 Hey, I'm Bret. 50 00:03:11,364 --> 00:03:12,444 And we're at Kon. 51 00:03:13,389 --> 00:03:15,079 We are Hi, Nirmal. 52 00:03:15,399 --> 00:03:16,179 Nirmal: I'm Nirmal Metha. 53 00:03:16,179 --> 00:03:20,079 I'm a principal specialist solution architect at AWS and these are 54 00:03:20,079 --> 00:03:24,579 my views and not of my employers, but this episode is all about 55 00:03:24,669 --> 00:03:25,149 Bret: AI 56 00:03:25,149 --> 00:03:25,929 Nirmal: agents 57 00:03:26,229 --> 00:03:28,869 Bret: for DevOps and platform engineering. 58 00:03:28,869 --> 00:03:28,899 Ooh. 59 00:03:28,959 --> 00:03:32,439 So let's just start off real quick with what is an AI agent? 60 00:03:32,469 --> 00:03:32,649 Okay. 61 00:03:32,649 --> 00:03:37,029 So we've heard of ai, we know ai, gen, AI chat, GPT. 62 00:03:37,239 --> 00:03:38,109 We've talked about. 63 00:03:38,449 --> 00:03:41,779 running LLMs, running inference on platforms. 64 00:03:41,809 --> 00:03:41,989 Yep. 65 00:03:42,049 --> 00:03:47,029 And that we are managing the workloads that provide other people services. 66 00:03:47,089 --> 00:03:47,539 Absolutely. 67 00:03:47,809 --> 00:03:50,749 So how is AI agents different than that? 68 00:03:51,309 --> 00:03:53,849 Nirmal: This is a air in terms of bleeding edge. 69 00:03:54,059 --> 00:03:54,299 Yeah. 70 00:03:54,779 --> 00:03:55,919 This is it, right? 71 00:03:55,919 --> 00:03:55,979 Yeah. 72 00:03:55,979 --> 00:03:57,239 Like we're a year ago. 73 00:03:57,239 --> 00:03:57,599 No one 74 00:03:57,599 --> 00:03:57,899 Bret: had this 75 00:03:57,899 --> 00:03:58,319 Nirmal: term 76 00:03:58,379 --> 00:03:59,009 Bret: six months ago. 77 00:03:59,009 --> 00:03:59,519 I don't think anybody's 78 00:03:59,519 --> 00:03:59,939 Nirmal: talking about it. 79 00:03:59,939 --> 00:04:00,524 I'm very few people. 80 00:04:00,594 --> 00:04:01,564 Yeah, very few people. 81 00:04:02,064 --> 00:04:06,084 and we've seen it in the news a lot of vendors and big companies 82 00:04:06,134 --> 00:04:10,744 announcing Agentic ai, that's another term's ai, so AI agents, Agentic 83 00:04:11,154 --> 00:04:19,044 It's giving your LLM, like your chat, GPT or your Claude or local LM Lama. 84 00:04:19,104 --> 00:04:19,404 Yeah. 85 00:04:20,304 --> 00:04:24,264 Access to run commands. 86 00:04:24,364 --> 00:04:25,444 On your behalf. 87 00:04:25,804 --> 00:04:26,974 Or on its behalf. 88 00:04:27,544 --> 00:04:27,724 Bret: Yeah. 89 00:04:27,724 --> 00:04:30,154 And we call those tools like that, if you hear that word. 90 00:04:30,214 --> 00:04:30,334 Tools. 91 00:04:30,334 --> 00:04:30,514 Yeah. 92 00:04:30,544 --> 00:04:33,244 That's like the generic tool, like I guess a shell. 93 00:04:33,709 --> 00:04:34,639 Could be a tool. 94 00:04:34,669 --> 00:04:35,149 Correct. 95 00:04:35,199 --> 00:04:37,659 Reading a file could be a tool. 96 00:04:37,809 --> 00:04:41,679 Accessing a remote, API of a web service is a tool. 97 00:04:41,709 --> 00:04:41,979 Yep. 98 00:04:42,009 --> 00:04:43,509 Searching could be a tool. 99 00:04:43,509 --> 00:04:47,429 And so these tools what what makes that different than what we've 100 00:04:47,429 --> 00:04:50,069 been seeing in our code editors? 101 00:04:50,099 --> 00:04:50,429 Yeah. 102 00:04:50,579 --> 00:04:51,309 How is that different? 103 00:04:51,692 --> 00:04:54,742 Nirmal: I'm a platform engineer and I want to build out an 104 00:04:54,742 --> 00:04:56,422 EKS cluster using Terraform. 105 00:04:56,422 --> 00:04:57,202 That's what we use. 106 00:04:57,502 --> 00:05:00,922 So I'll ask let's say Claude or chat GBT. 107 00:05:01,042 --> 00:05:01,222 Yeah. 108 00:05:01,282 --> 00:05:05,902 I'm a platform engineer and I want to build a production ready EKS cluster. 109 00:05:05,932 --> 00:05:06,742 Please create. 110 00:05:07,042 --> 00:05:11,272 The assets I need, and it will spit out some Terraform. 111 00:05:11,277 --> 00:05:11,542 Yaml, right? 112 00:05:11,542 --> 00:05:11,752 Yeah. 113 00:05:12,112 --> 00:05:13,337 Bret: And it's writing text. 114 00:05:13,337 --> 00:05:14,267 Nirmal: It's writing text. 115 00:05:14,267 --> 00:05:15,887 And I can, I'll double you a little button. 116 00:05:15,887 --> 00:05:16,727 I copy that. 117 00:05:16,727 --> 00:05:19,457 Put it in, or there'll be, if you're using Cursor, all these other tools, 118 00:05:19,457 --> 00:05:21,997 you can put it into some TF file. 119 00:05:21,997 --> 00:05:22,057 Yeah. 120 00:05:22,597 --> 00:05:26,107 I can then take that and I can ask the LM what's the command that I 121 00:05:26,107 --> 00:05:29,227 need to run to apply this Terraform? 122 00:05:29,685 --> 00:05:33,982 To actually stand up the, what it's, what's described in this terraform. 123 00:05:34,192 --> 00:05:36,892 It'll spit out, okay, you wanna do Terraform plan and then 124 00:05:36,892 --> 00:05:38,122 Terraform apply and all that. 125 00:05:38,122 --> 00:05:41,837 Terraform in it or whatever, and I'll just copy those commands and 126 00:05:41,867 --> 00:05:43,727 check 'em and write them myself. 127 00:05:44,777 --> 00:05:49,277 So the LLM is not executing anything on my behalf. 128 00:05:49,277 --> 00:05:49,772 On, on your behalf. 129 00:05:50,138 --> 00:05:54,158 Agent would be defining a tool set. 130 00:05:54,218 --> 00:05:59,138 So I could give, I could define a tool called Terraform or a tool 131 00:05:59,138 --> 00:06:04,988 called Shell I could describe what that tool does in natural language. 132 00:06:05,288 --> 00:06:05,528 Bret: Okay. 133 00:06:05,828 --> 00:06:10,688 Nirmal: And then I can give the LLM system a list of these 134 00:06:10,688 --> 00:06:12,218 tools and their descriptions. 135 00:06:12,518 --> 00:06:13,958 And tell it. 136 00:06:14,198 --> 00:06:14,858 Okay? 137 00:06:15,788 --> 00:06:16,838 Back to the same scenario. 138 00:06:16,838 --> 00:06:20,258 I'm a platform engineer and I want to create an EKS production cluster 139 00:06:20,258 --> 00:06:25,306 using Terraform, and I want you to create it right for me because 140 00:06:25,306 --> 00:06:27,256 it has the access to those tools. 141 00:06:27,256 --> 00:06:33,196 Now it internal reasons, okay, I need to create some Terraform. 142 00:06:33,496 --> 00:06:36,196 I need to validate it in some kind of way, and then I need. 143 00:06:36,746 --> 00:06:38,306 I need to execute this Terraform. 144 00:06:38,876 --> 00:06:40,736 Is there any tools that I have in my toolbox 145 00:06:40,736 --> 00:06:42,866 Bret: In this case, sorry the i is the, you're referring 146 00:06:42,866 --> 00:06:43,916 to yourself as the ai, right? 147 00:06:44,276 --> 00:06:44,426 Yeah. 148 00:06:44,426 --> 00:06:44,606 Sorry. 149 00:06:45,451 --> 00:06:46,916 It's no longer the human doing this, right? 150 00:06:46,916 --> 00:06:47,000 No. 151 00:06:47,000 --> 00:06:48,410 We gave it instructions and we sit back 152 00:06:48,950 --> 00:06:51,384 Nirmal: from the perspective, from the perspective of the, LLM the 153 00:06:51,384 --> 00:06:56,484 Gen AI tool itself, the LLM system that's the I in this scenario. 154 00:06:56,484 --> 00:06:56,485 Yeah. 155 00:06:57,384 --> 00:06:59,244 I, the LLM is deciding. 156 00:07:00,114 --> 00:07:07,374 The Gen NI tool is looking at its list of available tools and matching what it needs 157 00:07:07,374 --> 00:07:13,434 to it, figure it, it's reasoning about what the end goal is and it looks and 158 00:07:13,434 --> 00:07:17,544 says, there's this tool called Terraform that allows me to use infrastructure as 159 00:07:17,544 --> 00:07:20,274 code to deploy resources on the cloud. 160 00:07:20,784 --> 00:07:21,954 That sounds like what I need. 161 00:07:22,104 --> 00:07:22,614 Maybe. 162 00:07:22,754 --> 00:07:24,284 And it. 163 00:07:24,974 --> 00:07:27,674 Generates the terraform just like it did the first time around. 164 00:07:28,244 --> 00:07:31,124 It knows what command to run. 165 00:07:31,124 --> 00:07:36,174 It generates the command and then the magic here, a little box will 166 00:07:36,174 --> 00:07:40,014 show up and says, do you want me to execute this on your behalf? 167 00:07:40,224 --> 00:07:44,004 You click the button, you click the button, and then it executes that 168 00:07:44,034 --> 00:07:53,214 Terraform apply Uhhuh and it sounds very simple, but it's a very different paradigm 169 00:07:53,214 --> 00:07:58,624 in terms of thinking about how we interact with infrastructure or systems in general. 170 00:07:58,624 --> 00:08:00,334 Like broadly systems in general. 171 00:08:00,364 --> 00:08:04,354 Because we are no, like in this way of looking at it or thinking 172 00:08:04,354 --> 00:08:11,164 about it, I, as the human, are no longer executing those commands. 173 00:08:11,289 --> 00:08:11,379 I am. 174 00:08:11,409 --> 00:08:16,269 Trusting to a certain extent that the LLM can figure out what it needs 175 00:08:16,269 --> 00:08:22,859 to do and giving it a guardrail set of tools to use and execute. 176 00:08:23,159 --> 00:08:23,609 Bret: Yeah. 177 00:08:23,669 --> 00:08:26,319 And so we're giving the, we're giving the Chaos monkey XI 178 00:08:26,319 --> 00:08:27,279 mean, it's automation, right? 179 00:08:27,279 --> 00:08:29,829 We could actually classify this as just automation. 180 00:08:29,829 --> 00:08:30,729 It just happens to be. 181 00:08:31,464 --> 00:08:33,654 Figuring out what to automate in real time. 182 00:08:33,894 --> 00:08:37,764 Rather than the traditional automation where we have a very deterministic plan 183 00:08:37,764 --> 00:08:41,701 of, steps that are repeated over and over again by a GitHub action runner 184 00:08:41,731 --> 00:08:43,651 or a CI CD platform or something. 185 00:08:43,861 --> 00:08:43,951 Yeah. 186 00:08:43,951 --> 00:08:49,651 Nirmal: And the agent part is the piece of software that enables. 187 00:08:50,356 --> 00:08:51,466 The LLM to execute. 188 00:08:51,766 --> 00:08:52,036 Bret: Yeah. 189 00:08:52,086 --> 00:08:56,516 Nirmal: and pull, pulls this all together and one, so back to what I was 190 00:08:56,516 --> 00:09:00,056 talking about with the infrastructure and there was a part where I said, 191 00:09:00,086 --> 00:09:07,046 okay, how do we define what tools are available for the agent system to use? 192 00:09:07,046 --> 00:09:11,868 and how do I want the agent to call those tools? 193 00:09:13,053 --> 00:09:15,933 And reason about them, and there's a protocol called 194 00:09:15,933 --> 00:09:18,003 MCP Model Context Protocol. 195 00:09:18,103 --> 00:09:23,413 Just outlining a standard way of defining the tools, the system prompt 196 00:09:23,903 --> 00:09:25,283 for that tool and a description. 197 00:09:25,583 --> 00:09:27,943 Bret: And this is like an API where you like define the spec of an API. 198 00:09:27,943 --> 00:09:33,188 Nirmal: It's a defined spec of an API and the adoption of that API is 199 00:09:33,188 --> 00:09:34,208 Bret: just exploding right now, 200 00:09:34,328 --> 00:09:34,838 Nirmal: essentially. 201 00:09:34,928 --> 00:09:34,988 Bret: Yeah. 202 00:09:35,138 --> 00:09:38,078 So we're to, to under if you're not, okay sorry, lemme back up a second. 203 00:09:38,318 --> 00:09:41,558 That's a very valid point because that's the reason I wanted to record This's 204 00:09:41,558 --> 00:09:43,408 a I don't wanna be a hype machine. 205 00:09:43,738 --> 00:09:43,888 Correct. 206 00:09:43,888 --> 00:09:45,898 But I'm super excited right now. 207 00:09:46,058 --> 00:09:50,498 if you can see inside my, in my enthusiastic brain, I've 208 00:09:50,498 --> 00:09:53,498 only been paying attention to this for a little over a month. 209 00:09:53,813 --> 00:09:56,153 If you asked me two months ago what an AI agent was, I'd say, 210 00:09:56,393 --> 00:09:58,373 I don't know a robot that's ai. 211 00:09:58,403 --> 00:09:58,898 I don't know. 212 00:09:59,543 --> 00:10:01,823 I now think I've got a much better handle on this. 213 00:10:01,898 --> 00:10:05,663 I've been spending so much of my life right now, deep diving into this, to 214 00:10:05,663 --> 00:10:08,693 the point that you and I are talking about changing some of the focus 215 00:10:08,693 --> 00:10:10,133 this year on, on all these topics. 216 00:10:10,133 --> 00:10:10,493 Absolutely. 217 00:10:10,493 --> 00:10:13,163 Because I think this is gonna dominate the conversation. 218 00:10:13,463 --> 00:10:15,953 This is, these are, there's gonna be a lot of predictions in this and we're 219 00:10:15,953 --> 00:10:18,983 not gonna talk forever 'cause it's gonna need to be multiple episodes to 220 00:10:18,983 --> 00:10:20,663 really break down what's going on here. 221 00:10:20,663 --> 00:10:22,013 But we now have the definitions. 222 00:10:22,263 --> 00:10:24,273 AI agents, what are tools? 223 00:10:24,693 --> 00:10:27,303 The protocol behind it is essentially MCP right now. 224 00:10:27,303 --> 00:10:30,513 Although that's not necessarily gonna be the only thing, it's just the thing right 225 00:10:30,513 --> 00:10:33,003 now that we're agreeing on by one company. 226 00:10:33,063 --> 00:10:33,573 Exactly. 227 00:10:33,873 --> 00:10:39,733 Nirmal: We have to caveat this with, this is like this is early like Docker days. 228 00:10:39,763 --> 00:10:40,273 This is like 229 00:10:40,573 --> 00:10:43,123 Bret: Docker in day 60, right? 230 00:10:43,123 --> 00:10:43,213 Yes. 231 00:10:43,213 --> 00:10:46,833 Like we were like right after Python in 2013 when we gave that 232 00:10:46,833 --> 00:10:48,243 de, when he gave that demo, Solomon. 233 00:10:48,723 --> 00:10:53,073 Like we all saw it and didn't understand it fully, but it 234 00:10:53,073 --> 00:10:54,423 felt like something right. 235 00:10:54,423 --> 00:10:57,993 And like you and I both, that's why we were early docker captains, is 236 00:10:57,993 --> 00:11:00,693 we saw that as a platform shift. 237 00:11:00,911 --> 00:11:04,401 we've seen these waves before over, over our careers of many decades 238 00:11:04,591 --> 00:11:09,171 that we earned with this gray beard status with effort and toil. 239 00:11:09,471 --> 00:11:12,921 And I feel like this is maybe the moment. 240 00:11:13,731 --> 00:11:17,901 That was the moment of 2013 and that, and yeah, I'm not alone in that feeling. 241 00:11:17,961 --> 00:11:18,241 yes. 242 00:11:18,241 --> 00:11:21,781 Nirmal: And there's just to be clear, there's massive differences between 243 00:11:21,901 --> 00:11:26,131 like paradigm shifts in terms of like virtualization, cloud containers. 244 00:11:26,211 --> 00:11:31,111 And the tooling of software development and systems development 245 00:11:31,111 --> 00:11:35,071 and right systems operations, it's still in that same vein, but. 246 00:11:35,431 --> 00:11:36,261 Yeah, we're not replacing, 247 00:11:36,261 --> 00:11:39,051 Bret: this is not replacing infrastructure or containers or anything like that. 248 00:11:39,101 --> 00:11:41,021 This is just gonna change the way we work. 249 00:11:41,321 --> 00:11:41,741 Nirmal: Correct. 250 00:11:41,741 --> 00:11:44,831 And also it's broader than just like IT infrastructure. 251 00:11:44,961 --> 00:11:49,731 Like this has implications with software design or application, 252 00:11:49,731 --> 00:11:51,261 like what an application does. 253 00:11:51,361 --> 00:11:54,451 And I want to think of this as a teaser trailer. 254 00:11:54,766 --> 00:11:56,751 To subsequent new series, episode. 255 00:11:56,751 --> 00:11:56,873 A new series. 256 00:11:56,878 --> 00:11:57,556 Yeah, absolutely. 257 00:11:57,586 --> 00:11:57,886 We're gonna have to 258 00:11:57,886 --> 00:11:58,696 Bret: come up with a name. 259 00:11:58,696 --> 00:12:01,846 I'm toying around with the idea of Agentic DevOps, and just classifying 260 00:12:01,846 --> 00:12:05,816 that as the absolutely as the theme of certain levels of podcast episodes. 261 00:12:05,816 --> 00:12:06,867 You've heard it here first. 262 00:12:06,867 --> 00:12:07,011 Heard it here first. 263 00:12:07,046 --> 00:12:07,246 This 264 00:12:07,276 --> 00:12:08,546 Nirmal: is Agentic DevOps. 265 00:12:08,616 --> 00:12:11,106 Another term we're seeing is AI four ops. 266 00:12:11,406 --> 00:12:12,666 Again, this is early days. 267 00:12:12,696 --> 00:12:13,656 None of this is like 268 00:12:13,716 --> 00:12:13,896 Bret: Yeah. 269 00:12:13,896 --> 00:12:14,676 Set in stone at all. 270 00:12:14,676 --> 00:12:17,946 Yeah, and if you're at KU Con today with us, if you were here at this 271 00:12:17,946 --> 00:12:21,126 conference all week, AI was a constant topic, but it wasn't about this. 272 00:12:21,501 --> 00:12:26,271 It actually, there was only one talk in an entire week that even touched on the 273 00:12:26,271 --> 00:12:32,901 idea of using AI to do the job of an DevOps or operator or platform engineer. 274 00:12:32,961 --> 00:12:35,571 Like people are, what we're talking about at KU Con for the last three 275 00:12:35,571 --> 00:12:39,771 years has been how to run the inference and build the LLM models. 276 00:12:40,161 --> 00:12:44,271 And so we are just still using human effort to do that work. 277 00:12:44,466 --> 00:12:48,621 But this, I feel like I'm gonna draw the line in the sand and say, this is the. 278 00:12:49,081 --> 00:12:53,911 month or the definitely the year, that kicks off. 279 00:12:54,381 --> 00:12:59,321 What will be a multi-year effort of figuring out how we use automated 280 00:12:59,321 --> 00:13:02,981 LLMs Essentially with access to all the tools we want to give it with 281 00:13:03,011 --> 00:13:05,391 the proper permissions and only the permissions we want to give 282 00:13:05,391 --> 00:13:08,181 it right to do our work for us. 283 00:13:08,586 --> 00:13:11,916 In a less chaos mon monkey way, right? 284 00:13:11,916 --> 00:13:13,116 Like less chaotic way. 285 00:13:13,121 --> 00:13:13,311 Potentially. 286 00:13:13,386 --> 00:13:13,746 Potentially. 287 00:13:13,836 --> 00:13:15,816 It could, this thing can easily go off the rails. 288 00:13:15,906 --> 00:13:16,296 Absolutely. 289 00:13:16,296 --> 00:13:21,066 I will probably reference in the show notes Solomon Hike's recent talks about 290 00:13:21,076 --> 00:13:25,436 how they're now using Dagger, which is primarily A-C-I-C-D pipeline tool. 291 00:13:25,486 --> 00:13:30,286 So he's talking, and a lot of my language is actually from him iterating 292 00:13:30,286 --> 00:13:34,006 on his idea of what this might look like when we're throwing a bunch 293 00:13:34,006 --> 00:13:39,706 of crazy hallucinating AI into what we consider a deterministic world. 294 00:13:40,076 --> 00:13:40,326 Correct. 295 00:13:40,326 --> 00:13:46,506 Nirmal: I think with containers and cloud and on the infrastructure APIs we have. 296 00:13:47,001 --> 00:13:51,801 We were chipping away and really aiming at deterministic behavior 297 00:13:51,801 --> 00:13:53,673 with respect to infrastructure. 298 00:13:55,541 --> 00:13:57,753 Ironically, maybe not ironically, I don't know. 299 00:13:58,113 --> 00:14:02,883 Now we're introducing a paradigm shift that reintroduces a lot 300 00:14:02,883 --> 00:14:05,078 of non-determinism right into. 301 00:14:05,763 --> 00:14:09,483 A place that we have been fighting to non-determinism for a long time. 302 00:14:10,173 --> 00:14:12,483 Bret: We have been working to get rid of all that. 303 00:14:12,483 --> 00:14:15,333 And now we're, that's why I keep saying Chaos monkey, because we're 304 00:14:15,333 --> 00:14:16,863 throwing a wrench into the system. 305 00:14:16,873 --> 00:14:21,163 That is in some ways feels like we're going back to a world of, I don't 306 00:14:21,163 --> 00:14:22,813 know, what's the status of the system? 307 00:14:22,813 --> 00:14:23,143 I don't know. 308 00:14:23,813 --> 00:14:27,003 and this will probably be another episode, I feel like this Agentic 309 00:14:27,003 --> 00:14:30,003 approach where we're actually can have the potential to pit. 310 00:14:30,378 --> 00:14:32,448 The LLMs against each other, right? 311 00:14:32,448 --> 00:14:34,338 And have different personas of these agents. 312 00:14:34,488 --> 00:14:37,008 One is the validator, one is the tester. 313 00:14:37,158 --> 00:14:39,228 One is one is the builder. 314 00:14:39,288 --> 00:14:40,878 And they can fight amongst each other. 315 00:14:40,968 --> 00:14:41,808 And it all works out. 316 00:14:41,808 --> 00:14:44,118 It actually ha happens to actually work out better. 317 00:14:44,358 --> 00:14:48,198 And so if you're like me and for the last three years of 318 00:14:48,258 --> 00:14:50,148 understanding, ever since GPT. 319 00:14:50,658 --> 00:14:52,068 3.5 or whatever came out. 320 00:14:52,068 --> 00:14:56,778 We all saw chat GPT as a product, and then we started with GoodHub copilot and 321 00:14:56,828 --> 00:15:01,678 we started down this road As a DevOps person, I haven't had a lot to talk about 322 00:15:01,918 --> 00:15:05,638 because I'm not interested in which model is the fastest or the most accurate. 323 00:15:05,638 --> 00:15:06,298 'cause you know what? 324 00:15:06,428 --> 00:15:10,608 they all hallucinate and still even today, years later. 325 00:15:11,598 --> 00:15:14,898 Code agents and we and you can see this on YouTube, you watch basically 326 00:15:14,898 --> 00:15:18,528 thousands of videos on YouTube of people trying to use these models to 327 00:15:18,528 --> 00:15:21,018 write perfect code and they just don't. 328 00:15:21,508 --> 00:15:25,258 And so we in ops, but we look at that, I think, and the people I 329 00:15:25,258 --> 00:15:29,308 talk to even for years now are like, we're never gonna use that for ops. 330 00:15:29,348 --> 00:15:32,168 But now my opinion has changed. 331 00:15:32,228 --> 00:15:32,378 Yeah. 332 00:15:32,728 --> 00:15:33,088 Nirmal: yeah. 333 00:15:33,088 --> 00:15:37,088 And I. If you're listening to this and your gut reaction is, wait we 334 00:15:37,088 --> 00:15:39,398 have like APIs that are deterministic. 335 00:15:39,398 --> 00:15:39,998 Like you just 336 00:15:40,328 --> 00:15:40,568 Bret: Yeah. 337 00:15:40,868 --> 00:15:41,978 Nirmal: We can just call an API. 338 00:15:41,978 --> 00:15:46,538 We can have an automation tool call an API to stand up infrastructure and like, 339 00:15:46,538 --> 00:15:52,148 why do we need to recreate like another layer that makes it non-deterministic. 340 00:15:52,148 --> 00:15:56,858 And looks like an API but isn't an API and you don't really know what it might 341 00:15:56,858 --> 00:15:59,588 do or which direction it might go. 342 00:15:59,948 --> 00:16:00,128 Yeah. 343 00:16:00,158 --> 00:16:01,368 And you're feeling I don't know. 344 00:16:01,368 --> 00:16:04,398 That doesn't seem like it would solve any problems for me. 345 00:16:04,398 --> 00:16:06,408 And it seems like it might introduce a lot of problems. 346 00:16:06,608 --> 00:16:08,918 You're in the right place because that's exactly what we're gonna explore. 347 00:16:09,218 --> 00:16:09,638 Bret: Yeah. 348 00:16:09,988 --> 00:16:13,288 Nirmal: one thing for sure though is it's here, right? 349 00:16:13,723 --> 00:16:20,313 I and so I feel like as good engineers, as good system admins and operators 350 00:16:20,793 --> 00:16:22,408 Bret: are we enjoy, we love our crafts. 351 00:16:22,408 --> 00:16:23,338 We, we look at this as an. 352 00:16:23,653 --> 00:16:26,553 Art form of brain power and Right. 353 00:16:26,653 --> 00:16:29,873 Reaching for perfectionism in our YAML and in our infrastructure 354 00:16:29,873 --> 00:16:31,823 optimization and our security. 355 00:16:32,043 --> 00:16:37,053 Nirmal: And we have a healthy sense of skepticism on new tools, 356 00:16:37,053 --> 00:16:38,853 new processes, new mechanisms. 357 00:16:38,853 --> 00:16:39,063 Yeah. 358 00:16:39,123 --> 00:16:43,053 When you, when availability of your services is paramount and reliability, 359 00:16:43,413 --> 00:16:45,903 you want to introduce new things in a. 360 00:16:46,293 --> 00:16:47,583 In a prudent manner. 361 00:16:47,683 --> 00:16:51,553 And so we're gonna take that approach, but we're not going 362 00:16:51,553 --> 00:16:54,043 to dismiss that this exists. 363 00:16:54,098 --> 00:16:59,538 Clearly there's a lot of interest, energy integration happening, 364 00:16:59,868 --> 00:17:04,888 experimentation happening and some people are already starting to see value. 365 00:17:05,128 --> 00:17:05,308 Yeah. 366 00:17:05,358 --> 00:17:08,281 and we're gonna explore with you where that, goes. 367 00:17:08,311 --> 00:17:08,521 Bret (2): Yeah. 368 00:17:08,521 --> 00:17:15,691 This, just to be clear, this is KubeCon April, 2025 and almost 369 00:17:15,691 --> 00:17:17,671 no one is talking about this yet. 370 00:17:17,951 --> 00:17:21,941 It feels like it's right under the surface of a lot of conversations and 371 00:17:21,941 --> 00:17:25,181 a lot of people maybe are thinking about it, but I'm not even sure that 372 00:17:25,181 --> 00:17:27,986 we're honest with ourselves around. 373 00:17:28,751 --> 00:17:31,241 That this is coming, whether we like it or not. 374 00:17:31,451 --> 00:17:38,561 And only because, yeah, not only, but one of the large reasons is business. 375 00:17:39,161 --> 00:17:39,491 Okay. 376 00:17:39,851 --> 00:17:40,361 Lemme back up. 377 00:17:40,361 --> 00:17:44,381 You know how in a lot of organizations, Kubernetes became a mandate, right? 378 00:17:44,381 --> 00:17:47,601 So there's lots of stories that came out over the course of Kubernetes 379 00:17:47,601 --> 00:17:51,531 lifetime of teams being told that they need to implement Kubernetes. 380 00:17:51,581 --> 00:17:56,651 It didn't come from a systems engineering approach of solving a known problem. 381 00:17:56,651 --> 00:17:57,701 It came down. 382 00:17:57,986 --> 00:18:02,496 Because an executive decided that they read a CIO magazine article 383 00:18:02,496 --> 00:18:04,686 that said Kubernetes was a cool new thing and they did it right. 384 00:18:04,836 --> 00:18:06,126 I hear this all the time. 385 00:18:06,126 --> 00:18:10,386 I confirm this multiple times this week with other people, and I now feel 386 00:18:10,386 --> 00:18:12,876 like we're not talking about it yet. 387 00:18:13,296 --> 00:18:20,406 But I did hear multiple analysts say their organizations that they're working with 388 00:18:20,436 --> 00:18:26,586 expect that we are going to reduce the number of personnel in infrastructure. 389 00:18:26,886 --> 00:18:27,936 Because of ai. 390 00:18:28,061 --> 00:18:31,421 the only way that's possible is if we use agents to our 391 00:18:31,421 --> 00:18:33,401 advantage, because we can't, yeah. 392 00:18:33,401 --> 00:18:35,531 I still don't believe we're replacing ourselves. 393 00:18:35,861 --> 00:18:38,841 I don't think the agents will ever in, in the near term. 394 00:18:38,841 --> 00:18:42,061 And as far as we can see out, let's say five years they will, they 395 00:18:42,061 --> 00:18:45,361 won't be running all infrastructure in the world by themselves. 396 00:18:45,481 --> 00:18:46,921 They can't turn on servers. 397 00:18:47,101 --> 00:18:51,201 They maybe you can actually pixie boot and do a power on a POE or whatever, but. 398 00:18:51,951 --> 00:18:56,511 Like we still need someone to give them orders and rules and guidelines to go 399 00:18:56,511 --> 00:19:00,801 do the work, but to me, I'm starting to wonder if very quickly, especially 400 00:19:00,801 --> 00:19:04,581 for those bleeding organizations that are looking to squeeze out every cost 401 00:19:04,581 --> 00:19:09,531 optimization they can of their staff, that they're going to be mandated to 402 00:19:09,711 --> 00:19:15,891 not just take AI as a code gen for yaml, but to start using these agents to. 403 00:19:16,386 --> 00:19:20,482 Increase the velocity of their work . And my, one of my stories is 404 00:19:20,482 --> 00:19:24,292 over the last 30 years I do this in talks is every major shift has been 405 00:19:24,292 --> 00:19:26,782 about speed, cost reduction in speed. 406 00:19:27,022 --> 00:19:28,942 Sometimes we get 'em both at the same time. 407 00:19:29,002 --> 00:19:30,382 Sometimes they're one or the other. 408 00:19:30,382 --> 00:19:32,542 We get a cost reduction, but we don't go any faster, which is 409 00:19:32,542 --> 00:19:35,962 fine, or we're going faster, but it's not necessarily cheaper yet. 410 00:19:35,962 --> 00:19:36,322 Nirmal: Right. 411 00:19:36,622 --> 00:19:36,862 Bret: And. 412 00:19:38,167 --> 00:19:42,127 I feel like this is maybe the next one where We're gonna be feeling the 413 00:19:42,127 --> 00:19:46,117 pressure because all the devs are gonna be writing code with ai, which 414 00:19:46,117 --> 00:19:50,257 in theory is going to improve their performance, which means they're writing 415 00:19:50,257 --> 00:19:53,497 more code, shipping more, or need, or wanting to ship more code, potentially. 416 00:19:53,497 --> 00:19:56,407 And if we're not using AI ourselves. 417 00:19:56,707 --> 00:20:01,357 To automate more of these platform designs, platform build outs, 418 00:20:01,417 --> 00:20:04,537 troubleshooting when we're in production and things are problematic and we 419 00:20:04,537 --> 00:20:07,157 don't wanna spend three hours trying to find the source of the problem. 420 00:20:07,397 --> 00:20:12,617 If we're not starting to use agents to, to automate a lot of that and reduce the 421 00:20:12,617 --> 00:20:17,657 time to market, so to speak, for a certain feature or platform feature then I don't 422 00:20:17,657 --> 00:20:21,967 think these teams are gonna hire more of us to help enable the devs to deploy. 423 00:20:22,297 --> 00:20:26,287 What it could end up happening is we end up more with more shadow ops, where 424 00:20:26,287 --> 00:20:30,127 the developers are so fed up with us not speeding up to the, if they're 425 00:20:30,127 --> 00:20:32,317 gonna go 10 x we have to go 10 x. Yeah. 426 00:20:32,367 --> 00:20:35,097 If they're gonna go three x or whatever the number ends up being in the reports. 427 00:20:35,097 --> 00:20:38,362 And Gartner puts out like the AI makes it efficient, more efficient 428 00:20:38,362 --> 00:20:40,342 for developers to, to code with ai. 429 00:20:40,342 --> 00:20:42,832 And the models get better and the way they use it is better. 430 00:20:43,222 --> 00:20:46,682 And so they're shipping code faster and they can do the same speed with three 431 00:20:46,682 --> 00:20:48,490 times less developers, or they can just. 432 00:20:48,932 --> 00:20:52,100 Produce three times more work, which I think is more likely, because if it's 433 00:20:52,100 --> 00:20:55,010 the common denominator and everyone has it, then that means every company 434 00:20:55,010 --> 00:20:58,430 can execute faster and they're gonna, they're gonna want to do that because 435 00:20:58,430 --> 00:20:59,540 their competitors are doing that. 436 00:20:59,540 --> 00:21:02,300 So that's a's, that's a very loaded and long prediction. 437 00:21:03,170 --> 00:21:04,280 Nirmal: That's a hypothesis. 438 00:21:04,770 --> 00:21:05,860 It's, I think there's a lot of predict here. 439 00:21:05,860 --> 00:21:09,580 It's gonna take some time for us to even chip away at that hypothesis, 440 00:21:09,580 --> 00:21:11,140 but it's a good starting point. 441 00:21:11,240 --> 00:21:16,700 If we're, but assuming that is like the hypothesis that organizations 442 00:21:16,700 --> 00:21:20,210 are looking at to adopt these tools that's a great starting point 443 00:21:20,210 --> 00:21:22,880 for us to help you figure out. 444 00:21:23,390 --> 00:21:26,240 what they are, why they are, what they do. 445 00:21:26,270 --> 00:21:26,420 Yeah. 446 00:21:26,420 --> 00:21:27,170 And how to use them. 447 00:21:27,470 --> 00:21:30,200 Bret: This is this, by the way, a lot a little bit of that opinion 448 00:21:30,200 --> 00:21:32,660 of mine, and there's more to come 'cause I've got a lot more written 449 00:21:32,660 --> 00:21:33,530 down than we're never gonna get to. 450 00:21:33,950 --> 00:21:38,550 But a significant portion of that is actually coming from what I've learned 451 00:21:38,550 --> 00:21:43,650 this week from analyst whose job it is to figure this stuff out for their 452 00:21:43,650 --> 00:21:45,120 organization and their customers. 453 00:21:45,120 --> 00:21:45,480 Interesting. 454 00:21:45,660 --> 00:21:49,800 And so I, I am a little weighted by their. 455 00:21:50,805 --> 00:21:54,615 Almost unrealistic expectations of how fast we can do this. 456 00:21:54,615 --> 00:21:55,785 'cause we are still humans. 457 00:21:55,945 --> 00:21:59,185 An organization can't adopt AI until the humans learn how to adopt AI and 458 00:21:59,185 --> 00:22:00,835 the humans have to go at human speed. 459 00:22:00,915 --> 00:22:04,035 So we can't just flip a switch and suddenly AI is here and 460 00:22:04,035 --> 00:22:04,995 running everything for us. 461 00:22:04,995 --> 00:22:08,085 At least not until we have Iron Man's Jarvis. 462 00:22:08,115 --> 00:22:08,595 Or whatever. 463 00:22:08,595 --> 00:22:12,015 Like until we have that, we still have to learn these tools and still have 464 00:22:12,015 --> 00:22:13,635 to adapt our platforms to use them. 465 00:22:13,635 --> 00:22:13,636 Yes. 466 00:22:13,641 --> 00:22:15,195 And adapt our learning to use them. 467 00:22:15,555 --> 00:22:16,845 And that's gonna take some time 468 00:22:16,905 --> 00:22:17,415 Nirmal: and. 469 00:22:17,770 --> 00:22:20,740 I'd like to, like the parting thought for this is Okay. 470 00:22:20,740 --> 00:22:25,300 And here, like you said, there's an under the surface kind of thing happening. 471 00:22:25,330 --> 00:22:25,690 Yeah. 472 00:22:26,050 --> 00:22:26,890 So whispers, 473 00:22:26,890 --> 00:22:28,090 Bret: it's almost like murmurs and under 474 00:22:28,090 --> 00:22:28,690 Nirmal: the surface. 475 00:22:28,695 --> 00:22:28,895 Yeah. 476 00:22:29,195 --> 00:22:31,900 AI agent, AI agents, mag 477 00:22:32,200 --> 00:22:32,530 Bret: DevOps. 478 00:22:32,560 --> 00:22:32,950 Ooh. 479 00:22:33,490 --> 00:22:34,690 This is our ASMR podcast. 480 00:22:34,690 --> 00:22:35,440 Moment of the podcast. 481 00:22:37,395 --> 00:22:38,625 Nirmal: Like MCP protocol. 482 00:22:38,925 --> 00:22:39,255 Bret: Yeah. 483 00:22:39,595 --> 00:22:43,555 Nirmal: you mentioned HA proxy on the previous podcast, about load 484 00:22:43,555 --> 00:22:47,515 balancing and figuring out the street, like token utilization of 485 00:22:47,515 --> 00:22:50,065 GPUs and tokens and all that stuff. 486 00:22:50,315 --> 00:22:54,425 and we had a conversation at the solo booth and they were talking about having. 487 00:22:54,885 --> 00:22:59,115 A proxy for an MCP gateway, one of the things that we're seeing the early 488 00:22:59,115 --> 00:23:01,795 signs of is these new workloads, right? 489 00:23:01,795 --> 00:23:08,375 This agentic kind of thinking Around even just executing the agentic platform, 490 00:23:08,375 --> 00:23:12,785 if you will, And everything from looking at the tokens and optimizing 491 00:23:12,785 --> 00:23:19,375 load balancing to inference endpoints or MCP is, doesn't behave the same 492 00:23:19,375 --> 00:23:21,505 way as like just an http connection. 493 00:23:21,605 --> 00:23:22,265 Necessarily. 494 00:23:22,505 --> 00:23:23,075 And solo. 495 00:23:23,075 --> 00:23:25,475 We were talking to them and they have an MCP gateway. 496 00:23:25,755 --> 00:23:28,305 We're seeing a little bit more of a trend on AI gateways. 497 00:23:28,305 --> 00:23:33,705 Is DO the project has an AI gateway and so this is not just another workload 498 00:23:33,885 --> 00:23:35,475 and looks like just a web server. 499 00:23:35,523 --> 00:23:38,223 And the networking and everything is gonna be different. 500 00:23:38,303 --> 00:23:41,603 Not dramatically different, but We'll, but drift different 501 00:23:41,603 --> 00:23:43,283 enough that we need to be aware. 502 00:23:43,883 --> 00:23:47,483 'cause even if you're not using any of these tools, someone in your 503 00:23:47,483 --> 00:23:50,183 organization is probably gonna say, oh, we need to integrate this stuff 504 00:23:50,213 --> 00:23:52,673 into our software, to our right. 505 00:23:52,673 --> 00:23:54,053 Whatever we're delivering. 506 00:23:54,533 --> 00:23:56,513 And we'll need to know it even at that layer. 507 00:23:56,573 --> 00:23:59,993 So we're gonna also cover that component as it relates to. 508 00:24:00,533 --> 00:24:02,003 The Kubernetes ecosystem, right? 509 00:24:02,018 --> 00:24:02,648 And cloud native. 510 00:24:02,948 --> 00:24:03,308 Bret: Yeah. 511 00:24:03,428 --> 00:24:07,278 I think this, if we had to do like an elevator pitch for this podcast, it would 512 00:24:07,278 --> 00:24:15,358 be we now have a industry idea around these terms agent, and then it uses an API 513 00:24:15,358 --> 00:24:19,675 called MCP to allow us to give more work. 514 00:24:20,005 --> 00:24:24,325 To these crazy robot texting things that we have to talk to in human 515 00:24:24,325 --> 00:24:26,005 language and not with code, right? 516 00:24:26,005 --> 00:24:27,985 It's running code, but we're not talking to it with code. 517 00:24:28,255 --> 00:24:33,345 And that it can now understand all the tools we need to use and we can just give 518 00:24:33,345 --> 00:24:34,665 it a list of everything I wanted to use. 519 00:24:34,807 --> 00:24:37,927 here's my Kubernetes API, here's all my other things that I, you have 520 00:24:37,927 --> 00:24:40,177 access to, and here's my problem. 521 00:24:40,507 --> 00:24:41,527 Go solve it. 522 00:24:41,677 --> 00:24:44,437 And that paradigm. 523 00:24:44,812 --> 00:24:48,232 Three months ago, two months ago for me, I didn't know existed. 524 00:24:48,802 --> 00:24:51,802 And that's why I've been sitting on the sidelines with ai. 525 00:24:51,802 --> 00:24:55,882 Like it's cool for writing programs that mostly work in a demo. 526 00:24:56,132 --> 00:24:59,342 It's cool for adding a feature to something I already have, but it's 527 00:24:59,342 --> 00:25:03,242 not doing my job as a platform engineer or DevOps engineer. 528 00:25:03,292 --> 00:25:05,182 It's just helping me write text faster 529 00:25:05,275 --> 00:25:06,565 Then I can type into my keyboard. 530 00:25:06,615 --> 00:25:08,235 And that was not that interesting. 531 00:25:08,235 --> 00:25:10,605 That's why you didn't see a lot of me talking about that on this show, 532 00:25:10,785 --> 00:25:11,715 was it just wasn't that interesting. 533 00:25:11,865 --> 00:25:17,475 This is an interesting topic for ops and for absolutely engineers on the platform. 534 00:25:17,775 --> 00:25:17,955 Nirmal: Yep. 535 00:25:18,270 --> 00:25:18,570 Bret: So 536 00:25:19,050 --> 00:25:19,650 Nirmal: stay tuned. 537 00:25:19,740 --> 00:25:20,280 Yeah. 538 00:25:20,280 --> 00:25:23,610 And I, I love crazy texting robots. 539 00:25:23,640 --> 00:25:24,000 Crazy 540 00:25:24,000 --> 00:25:24,930 Bret: texting robots. 541 00:25:25,060 --> 00:25:26,650 Maybe that's the title. 542 00:25:26,650 --> 00:25:27,130 TBD. 543 00:25:29,350 --> 00:25:29,890 Alright. 544 00:25:29,950 --> 00:25:30,460 Alright. 545 00:25:30,460 --> 00:25:31,210 See you soon, man. 546 00:25:31,810 --> 00:25:32,020 See 547 00:25:32,020 --> 00:25:32,200 Nirmal: you. 548 00:25:32,200 --> 00:25:32,202 See you. 549 00:25:32,860 --> 00:25:33,430 Bye. 550 00:25:33,430 --> 00:25:33,490 Bye.