Lori MacVittie (00:04.07) Welcome back to Pop Goes the Stack, where AI promises acceleration and delivers a fresh set of failure modes at machine speed. I'm Lori MacVittie and today we're gonna see what's actually intelligent out here. So... Did Joel laugh? Joel laughed. I, Joel laughed. I can't, I can't go on. Joel Moses (00:25.591) Ah, the search for intelligence, it goes on and on and on, doesn't it, Lori? Lori MacVittie (00:28.952) I know, right? Is it out there? Is it in here? Where is it? We don't know. But today we're gonna take a different look at intelligence and tackle a massive operational challenge that is top of mind for every enterprise accelerating with AI. Or at least it should be, right? AI code quality management. I know, most exciting thing you've heard since Hey, it's tax season. So, I mean, the issue here is that developers increasingly are using copilots and AI to write thousands and thousands of lines of code in seconds. And we're experiencing this unprecedented productivity boom. Yay, go us. But the velocity is introducing a critical risk. How do we ensure that the flood of AI-generated code is secure, compliant, and aligned with enterprise architecture standards? Assuming that you have standards. You should, but, you know, we're gonna assume you do. So in this episode we've got Elijah today. Welcome back, Elijah. Elijah Zupancic (01:31.505) Hey. Lori MacVittie (01:33.66) Good to have you. I know you have opinions, so let's just cut through the noise and start, you know, discussing how organizations can build things like automated guardrails, validation pipelines, governance frameworks, all the the cool words to keep AI written code from turning into Joel Moses Yeah. Lori MacVittie an operational and security nightmare. And in our pre-discussion, Joel had some great points and questions. So I'm gonna, hey, I'm trying Joel Moses (01:59.061) Sure. Lori MacVittie to be nice here, Joel. Joel Moses Absolutely. Lori MacVittie Like you kick us off. All right. Joel Moses Well, you know, at the risk of being a little controversial, I'm just going to be very direct. Elijah, is it akin to malpractice to fail to use things like frontier models to ensure the quality of your code? These things can be very, very expensive when run in sequence on a code base that's been generated--you have to run them multiple times and you have to continuously redirect the output and check for quality--but is the return that they deliver such that failure to do so is akin to malpractice? I mean, where are we right now? Elijah Zupancic (02:40.338) I mean, I think I'm gonna take your hot take and flip it back on itself. Joel Moses (02:45.792) Excellent. Lori MacVittie Ooo. Elijah Zupancic Remember back in the old times before we were talking about AI? Was it malpractice to ship a product that didn't meet quality standards then? Joel Moses (03:02.54) That's a that's an exce- Lori MacVittie (03:03.749) It's, Joel Moses That's a good way to turn it around. Lori MacVittie Ha ha ha ha, it really is. Joel Moses I Elijah Zupancic (03:06.118) Bec- Joel Moses I mean, of course coming out of the security industry as I do, yeah, I would say yes, that failure to turn out a piece of software with sufficient quality is akin to malpractice, especially if you're simply bypassing the tools that were available to you in order for expediency to be obtained. Lori MacVittie (03:27.537) Aren't we making it the you must have a WAF part of PCI DSS, right? Well, you must have run it through this. Doesn't say anything about addressing any concerns that the model may have found, just you have to have run it through it. Like Joel Moses Mm-hmm. Lori MacVittie is it, that's making it kind of just a checkbox thing, isn't it? Elijah Zupancic (03:46.707) Yeah, I mean there is a checkbox thing, and that's been a part of the industry for gosh knows how Lori MacVittie (03:52.4) Yeah. Elijah Zupancic long, right? But I do agree with you, Joel. Like this releasing software with no thought of quality to use by enterprises in critical infrastructure is software malpractice. And the sad truth is this has been a long-standing industry problem way before AI came along. And just like everything else with AI, Elijah Zupancic (04:16.388) it's just accelerated. It made all the existing patterns just go way, way faster. Now I can create a year's worth of technical debt in about four hours. Joel Moses Mm-hmm Elijah Zupancic I can create exploits faster than anyone could in the past. And I mean exploitable code I'm writing. Joel Moses (04:38.093) Right. Lori MacVittie (04:38.495) Ha ha ha. Joel Moses (04:41.72) So tell me, are these systems do they have sufficient quality or do they still need enough human oversight that the investment in these frontier models to help you maintain and manage your code quality is it worth, is the juice worth the squeeze, so to speak? Elijah Zupancic (04:59.536) I would say if you're gonna drink the AI pill and take it down, you need frontier models. And getting in the definition of what are frontier models is its own problem because it seems like that term has become a little bit ambiguous, but you need the best models you have available to you for the initial codegen and architecture steps as well as the multiple revision steps you do. If you choose to use self-hosted, open weight models, you probably need a whole set of second opinions from other models if you want to do this all via AI. Human review steps are like gold often if you actually can get people to do it, because you catch many things and can set the AI off in a useful direction where before you might have spent a huge amount of tokens of it going down a wrong turn. There's a term I like to use that I borrow from the slow food movement. I call it the slow AI movement. And that you should Joel Moses/Lori MacVittie Ha ha ha. Elijah Zupancic Yeah, let that sink in. Lori MacVittie (06:10.853) Yeah, I Joel Moses (06:11.362) I'm getting there. I'm getting there. Elijah Zupancic (06:13.67) But hear me out. Hear me out. Okay, so you should just be happy with a 7x to 15x productivity increase. You don't need to turn the dial to 100x all the time. You can sit there, rather than run everything in a Ralph loop, like, you know, you typed in one sentence, I want to generate this, and then let it off for 10 days of execution. Don't do that. Actually sit with the model and go through steps where you're architecting the application as a partner with the model and then looping back with various QA and security checks and architecture checks iteratively. You go so much faster than you ever were in the past. And you should be happy with that because you get a much higher quality product at the end. And you also have a SME that actually knows how it works. That's my hot take, Joel. Joel Moses (07:16.278) I see. In human terms they know how it works, right? Lori MacVittie (07:16.433) The Joel Moses In human terms Lori MacVittie It kinda sounds like Joel Moses they know how it works, right? Lori MacVittie Yeah. Elijah Zupancic (07:20.476) Yeah, I mean the chance that somebody understands it is way better than the Ralph Loop approach. Lori MacVittie (07:27.483) And that sounds like the "cleanup as you go," right? If you take the time to clean up, right, your mess every day, it goes a lot faster than if you wait until, well you know, you're a teenager and you need a bulldozer, right, to get it cleaned up. It's actually faster and more efficient to do something like code reviewing and checks and everything as you're building it out rather than waiting till the end when now have this huge corpus of code that you have to try and get through. And like you said, understanding that can be problematic. I mean we often use AI to help us understand big corpuses of code because it's too unwieldy. So, you know, incorporate it into your daily, I guess, you know, routine rather than making it a stop. So agile rather than waterfall. That's what I'm hearing. Waterfall Elijah Zupancic Yep. Lori MacVittie AI is bad. Okay. Elijah Zupancic (08:25.308) That's an interesting analogy. I would take it even one step further that you should have hooks in your Codegen configuration that call out to algorithmic approaches and perhaps AI approaches, but steps that do things like look at the code quality metrics like the classic one, cyclomatic complexity, Halstead F ratio, maintainability index, etc., and then feed back into the model saying, Hey, we detected the code metric exceeded this. Right? This is a code smell. Hey, go analyze it and fix it. Joel Moses Mmm. Elijah Zupancic And so at key junctures, it could be done in CI, or it could be actually Claude Code, for example, and OpenCode support this notion of hooks, so when it writes files, it will trigger these secondary processes that do inspections. Semgrep now has, I think, a beta product that will do this as well. So whenever those hooks are triggered, it will trigger security review of what was generated by the agent. And then if there are problems, it'll hand it back to the agent and be like, Yo, you got to fix this before you move on to the next step. And I think this is a promising approach for how you can take some of the toil off the person using the tool and also ensure that people are doing these cycles of checks. Joel Moses (09:56.504) Well you've said a couple of things here that kind of triggered me a little bit. You said that, you know, you arm these these people with very expensive tools in order to make them go faster but still retain some sort of sense of ownership and knowledge of the code that's generated from these particular systems. However, increasingly, you know, a lot of the projects that are being refactored in enterprise space are heavily based on open source projects. And open source projects have flaws just like every other software project on the planet does. However, open source projects don't necessarily have a dedicated, reliable staff of people, nor do they have necessarily access to the same expensive tools that commercial software developers do. What do you make of this kind of enterprise asymmetry to volunteer open source? And what do we need to do to fix that? Elijah Zupancic (10:57.85) It is a DDoS attack. Full stop. I mean, Joel Moses Hmm. Elijah Zupancic we have a small set of volunteers that are not well compensated for the many, many hundreds of thousands of hours or more they put into these projects. And the automated AI bug filings, security exploits found, et cetera, et cetera. So many people are relying on AI for their interactions with open source projects that these OSS projects are just suffering under the weight of information overload. Where we're now seeing projects that require when you file a bug that you click a checkbox and say, I did not use AI to author this bug report. I testify Lori MacVittie (11:53.862) Ha ha ha Elijah Zupancic (11:54.843) that I understand all of the problems in this bug report and when questioned about it can answer as a human being intelligently. Joel Moses Hmm. Lori MacVittie Wow. Joel Moses (12:04.717) So that's just a step that one takes in the pipeline to prevent the kind of the problem from being thrown over the fence by a system that doesn't quite know how to describe what it found, right? But I mean, it's still a problem. It's still a software quality Elijah Zupancic (12:18.908) That's right. Joel Moses escape, potentially, right? Elijah Zupancic Yep. I mean we're seeing some of the AI labs offering help. Anthropic just gave all Apache committers free access to Claude, which I think was a huge positive step. And you know, many OSS, you know, committers are using AI tools now just to keep up. But it's an asymmetry and it's not getting better. Joel Moses (12:57.845) Let Lori MacVittie (12:57.98) But it's just Joel Moses talk about the other half of this asymmetry, which is the economy of this. Lori MacVittie (13:03.292) Yeah. Joel Moses Obviously the people who are using this Lori MacVittie It's money. Joel Moses on behalf of open source, either they have something that's been donated to them or Joel Moses (13:08.957) in a lot of cases, they're simply using or spending money of their own to contribute back to this open source project. Now, the maintainers of many open source projects are, you know, they volunteer for this. So they don't spend a ton of money on it, especially not relative to the commercial entities that use the open source that they produce. Talk for a little bit about the economy of this. I mean, obviously there's an economic deficit of, you know, whether you use these tools to their fullest extent or whether it's okay just to use it to get a 15X speed up instead of a 100. But what about the other side of this, which is people who are using this as a volunteer and paying their own way to contribute back to the quality of open source projects. What should we be doing to help out with that? Elijah Zupancic (14:00.708) I mean, this is my opinion, but the industry overall, the tech industry and so many other industries are on the foundation of open source. They have made tremendous revenues, tremendous revenues in the last thirty years on the foundation of open source, with very little of that revenue going and being reinvested into the foundations that are running the modern world, truly. If you look at things like even Linux, which gets a lot of funding, the amount of money made that is coming out of the use of Linux is completely non-proportional to the amount invested. Not to speak of things like OpenSSL or TreeSitter or gosh knows, you know, any number of network libraries that everything is running on and running through. You know, and it's really a tragedy in my mind. We have to, as an industry, come up with a way to fund this critical infrastructure, because the tsunami of AI-driven bug finding and vulnerabilities found is coming. And these projects are not going to be able to sustain themselves. I think it actually almost comes down to a national security problem, in the end. Joel Moses (15:33.344) Interesting. Lori MacVittie (15:33.907) Interesting, yeah. It's interesting. And I think it gets worse. I know about... the last time I went and checked about open source projects and what the most popular ones were, one of the things that cropped up is actually, and you mentioned it, kind of the network side: network tools, administrative tools, right, a lot of IT tools that we generally don't consider as open source or as code. But we are seeing everyone's using AI to generate code, whether that's a script, you know, that a SecOps, NetOps, NetDevSec ops, the ops people, right, are writing or generating or using tools to do it and then they're also pulling in as well. And that's another area of your infrastructure that needs to be looked at in terms of code quality. Because these things are taking software that is just been AI generated and now it's trying to keep it available and, you know, provide communications, what's, you know, what's going on. So I think in general, the enterprise has to be more aware of it on the open source side in every area of its infrastructure where it might be using this kind of code and it needs to do something more overreaching. You know, maybe it's time we stop separating these folks out and saying, Well, they're not developers. They're writing code. Isn't that the definition? So, we overlook that and that could be another area where, you know, we're gonna get bit eventually. Elijah Zupancic (17:10.802) I mean we're building way more code than I think we have the capability to maintain. And this was the trend we Lori MacVittie Yeah. Elijah Zupancic were seeing before the AI boom. Joel Moses (17:22.316) Yeah. Elijah Zupancic Now it is just, you know, a thousand X perhaps. Joel Moses (17:27.699) It it does strike me that, you know, as you were talking, Lori, I was thinking about all the titles that have somehow had ops grafted onto the back of them. I think we're at the very beginning of having dev grafted onto a lot of different titles. It's kind of like in the old days, Elijah, when the joke was Lori MacVittie Nice. Joel Moses that any document display engine would eventually grow to the point where it could read mail. I think Elijah Zupancic (17:50.791) Yep. Lori MacVittie Ha ha ha. Joel Moses every title in technology has over the past decade grown to include ops as a portion of it because of, you know, the tools becoming better in operations. Joel Moses (18:02.991) And now the development tools are getting better, so now everybody's gonna be having dev at the end of their title. Elijah Zupancic (18:08.24) I mean it's Joel Moses Marketing dev... Lori MacVittie Dev... Elijah Zupancic (18:08.24) kinda like engineer. Lori MacVittie (18:09.106) Yeah. Elijah Zupancic Somebody who drives a train and picks up your trash and, you know, Joel Moses (18:17.581) That's a good point. Elijah Zupancic deals with trash code. That's Lori MacVittie (18:20.21) Oh, I see what you did there. I like that. Joel Moses (18:23.287) Very nice. Lori MacVittie I was Joel Moses Very nice. Lori MacVittie We're starting to get to the end here. So, you know, I'd like to be able to like, I don't know, leave the listeners slash watchers with some kind of thoughts going forward, how they can get ahead of this or at least get a hold of it, right, start doing something to address it. Joel Moses (18:43.051) Yeah, you know, as I think about what we've talked about today, I kind of find myself thinking about where the so-called digital water in our industry comes from. Think of open source software as a massive public fountain, a mountain spring that feeds the entire tech ecosystem. And commercial enterprises typically build these massive high-tech bottling plants at the bottom of the mountain, and they use automated tools and lots and lots of people to produce products out of it. But up at the top of the mountain, you've got a handful of these volunteer park rangers that are feverishly trying to keep that spring clean and replacing the filters on it. And, you know, all the while they're being pelted by an avalanche of synthetic debris to keep out of the water. If that spring gets contaminated or if it dries up, you know, out of sheer exhaustion, the multi-billion dollar bottling plants at the base of that mountain, they stop working too. And so if you're consuming open source at machine speed, you might think that it's time to start funding the park rangers keeping it clean. Lori MacVittie (19:49.042) Ooo, that's lovely. Elijah Zupancic (19:49.363) Completely agree, Lori MacVittie I'm just, yeah. Elijah Zupancic Joel. And actually I would take it one step further is that those rangers are now targets of geopolitical issues. And AI is just accelerating that and it actually comes down to ensuring that all of us have clean drinking water. Perhaps not even as a metaphor. Lori MacVittie (20:13.867) Ha ha ha. Joel Moses (20:15.149) That's a good point. Elijah Zupancic (20:15.96) And then, you know, you might be asked, Okay, what can I do about it? Well, one, if you work at a big company or a corporation, press on your leadership to support the open source projects you depend on. Two, contribute. Get involved in open source yourself. And this is where I'll drop a pitch. I have an open source project that's related to this talk called Big Code Analysis, maybe we can drop in the notes, but you can hook it up into your AI codegen pipelines into your hooks, and it will assess your code quality and then feedback into your AI gen loop. Joel Moses (20:59.937) That's pretty cool. Lori MacVittie (21:01.084) That is. That is very cool. Joel Moses And definitely needed. Lori MacVittie Yes, absolutely. And I like one of the things I took away and I think everyone should be taking away is there's hooks. Just like web hooks and, right, GitHub hooks, all of these hooks to run things. So take advantage of that and run things more often, right? The test early, test often mantra is even more true in a world where we're generating this stuff at machine speed. So we need to machine speed do more testing, have more eyes on the code, and make sure somebody knows what it does--not just comments but actually understands what all the pieces are. So, you know, if you want to stay safe, pay attention, start testing a lot more often to make sure that you're producing the cleanest water you can, right, for the data set. Ahh, there we go, even I got it in. All right, Joel Moses (21:56.461) There you go. Lori MacVittie (21:57.23) I'm gonna close it out now before I make another bad analogy or something like that. And you know, because it just it can't be done. So that's a wrap for this episode of Pop Goes the Stack. Please subscribe because your next agent you deploy will eventually ask for permissions and you'll want witnesses.