Subscribe
Share
Share
Embed
What happens if AI finally writes secure code by default? In this episode of Pop Goes the Stack, F5's Lori MacVittie, Joel Moses, and Ken Arora take that question seriously, even if it feels like a punchline today. The premise is simple: if AI starts producing mechanically correct, vulnerability-free code at scale, the security industry doesn’t disappear. It gets forced up the stack.
They outline the near-term reality first: AI can already find and fix large classes of common issues in legacy code, and we’re likely heading into a chaotic cleanup phase as tools like Mythos-style systems accelerate remediation. But longer term, the conversation shifts to the uncomfortable tradeoff: code quality may improve faster than complexity shrinks. And complexity is where authority problems, logic flaws, and operational failures thrive.
Ken uses a practical analogy: a “perfectly safe driver” can still follow a GPS off a cliff. Likewise, perfect code can implement flawed or exploitable business logic flawlessly. Ticket scalping, abuse of workflows, and manipulation of incentives aren’t bugs; they’re behavior and design gaps. The bigger risk becomes how systems are orchestrated, who has permission to do what, and how autonomous components interpret intent.
They also flag the adversarial side: attackers won’t stop; they’ll shift from injecting obvious vulnerabilities to subtly influencing specs, workflows, and machine-generated architectures. As specs in plain language become the new “source code,” more people will be able to build powerful systems without the instinct to think like an attacker, widening the logic-attack surface.
The takeaway is a shift in mindset: security becomes less about chasing broken code and more about governing boundaries and monitoring behavior. AI will increasingly do exactly what you ask, which makes the new security imperative painfully clear: be precise about what you ask it to do, and enforce what it’s allowed to do when it tries to go beyond that.
Creators and Guests
Host
Host
Lori MacVittie
Distinguished Engineer and Chief Evangelist at F5, Lori has more than 25 years of industry experience spanning application development, IT architecture, and network and systems' operation. She co-authored the CADD profile for ANSI NCITS 320-1998 and is a prolific author with books spanning security, cloud, and enterprise architecture.
Guest
Ken Arora
Ken Arora is a Distinguished Engineer in F5’s Office of the CTO, focusing on addressing real-world customer needs across a variety of cybersecurity solutions domains, from application to API to network. Some of the technologies Ken champions at F5 are the intelligent ingestion and analysis of data for identification and mitigation of advanced threats, the targeted use of hardware-acceleration to deliver solutions at higher efficacy and lower cost, and the design of user experiences based on intent and workflows. Ken is also a thought leader in the evolution of the zero trust mindset for security, and how that will be applied to increasingly distributed and even edge-native apps and services. Prior to F5, Mr. Arora co-founded a company that developed a solution for ASIC-accelerated pattern matching, which was then acquired by Cisco, where he was the technical architect for the Cisco ASA Product Family. In his more distant past, he was also the architect for several Intel microprocessors. His undergraduate degrees are in Astrophysics and Electrical Engineering, from Rice University.
What is Pop Goes the Stack?
Explore the evolving world of application delivery and security. Each episode will dive into technologies shaping the future of operations, analyze emerging trends, and discuss the impacts of innovations on the tech stack.
Lori MacVittie (00:03.015)
Welcome back to Pop Goes the Stack, where shiny demos meet production traffic and reality files a ticket. I'm Lori MacVittie and I'm standing by with incident notes. This week we're gonna answer the question what happens when AI finally writes secure code? No, really. Think about it for a minute. Try to stop laughing. Just try. I mean, because if AI actually starts consistently producing secure-by-default code, the security industry changes, right?
AppSec has to shift from catching basic mistakes to dealing with all the new ways autonomous systems can accidentally set production on fire. As Joel will no doubt explain in just a few minutes. The problem is that AI models still produce vulnerable code at hilariously consistent rates while sounding extremely confident about it. So this week we wanted to ask the question of Ken Arora, one of our security experts here. Hi Ken.
Ken Arora (01:07.49)
Hi all.
Lori MacVittie (01:09.009)
What happens if AI gets good at secure coding? What breaks? What evolves? And does secure code even matter when the real risk starts moving higher up the stack into orchestration, autonomy, and machine-made decisions? Right? Because that's a very different conversation. So let's have that. Go ahead, Joel. I know you've been waiting, you're chomping at the bit.
Joel Moses (01:32.741)
Well, look, I think it's there's two ends to this. One end is when AI generates more secure code, and I think that the systems that assist coders are getting much, much better at generating quality code. Then there's the flip side of it, which is AI helping fix vulnerabilities in code that perhaps humans have created, perhaps other AI coding assistants have created. You know, everybody's read the the the statements about Mythos and you know the elements that create multi-stage attack chains and find deeply hidden security flaws.
So I think that in the end analysis, I do believe that the bar is raising for all of us in terms of the code quality that's necessary to sustain within enterprise applications and certainly the code quality that's necessary to sustain a cybersecurity product line. And so, you know, obviously we've been taking this very seriously, for sure. But it doesn't skip, you know, it doesn't really address the fact that, you know, one of the deepest misconceptions in cybersecurity is that insecure systems exist because of bad programmers.
They don't necessarily. They can. But for the most part, many catastrophic failures come from flawed incentives or rushed deployments or excessive complexity or poor operational decisions, weak governance, bad supply chain trust, business pressure. Perfect syntax doesn't produce perfect systems all the time. And so I think the onus is shifting from security being a find and fix flaw problem to a figure out who has authority problem. I think it just shifts the bar. I don't think it necessarily moves it downward.
Lori MacVittie (03:25.286)
Ken. Rebuttal. You're good at rebuttals.
Ken Arora (03:26.242)
Well no, I generally agree with Joel. I mean, I think Joel mentioned this and I think it's worth repeating just to emphasize it a bit. There's what happens in the short term and what happens in the long term. In the short term, we have a lot of human written code that was honestly poorly written. Maybe you know, some of it is whether it was, you know, unchecked pointers, unvalid inputs, whatever, all these things. And AI, you know, Mythos, Glasswing, all these things going on are very good at that.
Ken Arora (03:56.279)
And I think we're gonna go through a period, and I don't know whether it's gonna be six months, a year, two years, where all that is gonna get cleaned up. It's gonna be pretty chaotic and pretty wild for a little while we do that. And then there's a, and then at that point, most of the code being written will either be cleaned up or be AI written and so it'll be mechanically correct. And I think we'll have to live very good code. And then we do the shift that Joel talks about. I agree with you Joel. It's going to happen.
It's I think in when we're, you know, talking about this a little bit earlier off the record, we were talking about cars, the analogy. And I can have somebody who is a perfectly safe driver. It's like I wanna be safe. I'm gonna, you know, do everything right. I'm gonna wear my seat belts. I'm not gonna go over the speed limit. I'm gonna make sure I keep following distance, all this stuff. And I faithfully follow my GPS off the cliff. Was that safe driving?
Joel Moses (04:46.693)
Yeah, no, that's true.
Lori MacVittie (04:46.844)
Well, he safely went off the cliff. I mean, you know.
Joel Moses
Yeah, safely. You know, perfect code can and perfect approaches can still implement terrible ideas flawlessly. I mean that's the bottom line. And so I think that once we get to the level where code quality comes up and perhaps the quality of components that applications are comprised of
Joel Moses (05:07.843)
it, that it comes up, the focus will naturally shift to the things that are not necessarily related to code. Things like logic issues in applications, things like authority or control violations. I'm gonna make a controversial statement here--I think it's controversial--that in the future enterprise security teams will look more like governance teams than they will operations or engineering teams.
I think they tend to look a little too much like engineering teams today. They get very involved in the day-to-day find and fix issues. but I think that that's gonna be set aside in favor of looking for things that are actual authority problems in applications. That's probably what's going to happen with this.
Lori MacVittie (05:54.981)
I hear some people crying right now, like because you compared security to governance. I don't disagree with either of you that eventually, right, this moves up the stack. This is a time tested technology cycle evolution where, right, we address things at lower layers and then we have to move up the stack. It is up the stack from code and bugs to logic,
Lori MacVittie (06:23.14)
risk, authority, things like that. My contention is that we are very, very, very, very far from that, given that this is a vicious cycle. The AI has been trained on bad code. It produces bad code, but it can also recognize it. But now it's supposed to fix it.
Joel Moses (06:43.377)
Mm-hmm.
Lori MacVittie
How? How does it know what good code looks like when it's been trained on, you know, 20 years of you know
Lori MacVittie (06:52.676)
Not good code.
Joel Moses (06:53.369)
Yeah. Well, I don't think it's a universal that all AI systems generate bad code. I think that over time as they're being trained with human assistance to recognize code quality issues, that they're getting better and better. I mean, we we've seen a remarkable advancement, especially in the reasoning systems related to doing multi-stage design and building ever more complex systems.
That kind of introduces another question though. If AI begins generating wonderful code
Lori MacVittie
Circle. Circle.
Joel Moses
and it generates it much faster, it's going to generate a large amount of it. Large amounts of code interoperate with each other. Does AI reduce software
Lori MacVittie (07:35.192)
In strange ways.
Joel Moses
risk faster than it reduces complexity?
Lori MacVittie
No.
Joel Moses (07:38.959)
Complexity is an authority or an operational problem, right? So I think again, in the great analogy of the toothpaste tube, with security issues, we're squeezing one end of the tube and the other end fills up, right? So once you eliminate bad code and you eliminate quality issues, then it becomes a matter of looking for authority or operational issues or logic flaws. So again, I don't think the security industry is going to disappear.
I don't think the need for security products will disappear, but they'll have to change focus for looking for authority problems and looking for behaviors that represent possible attacker traffic.
Ken Arora (08:18.105)
Yeah. I think that word, that last word, behavior is what matters. I mean it's not the dry sort of governance we sometimes think about with compliance. It's really thinking like an attacker. And it is not, attackers don't necessarily, there's a class of attackers or bad guys who will look at code and look for code vulnerabilities, but there's I think increasingly a class that says, "is the business logic of this application correct?"
You know, a really well known example of that is, right, the people who buy tickets--you know, the airline tickets, Taylor Swift tickets--and resell them. I mean that's a really simple example. The code's working perfectly, they just didn't think about the business angle. I think they're gonna talk about being an adversary. And that's gonna be the shift. I'm gonna ask you guys there's a premise here that seemed to be implicit in what we're saying, which is that this is all a friendly cooperating ecosystem
Joel Moses
Ha ha ha.
Ken Arora
that's all trying to make this better.
Lori MacVittie
Of course.
Ken Arora
What happens when bad guys are
Joel Moses (09:11.105)
Yeah.
Ken Arora
are using AI to
Ken Arora (09:17.131)
subtly inject vulnerabilities? Not trying to make the code better, they're trying to subtly inject things.
Joel Moses (09:22.843)
No, you
Ken Arora
How do you-, that gets harder and harder to detect.
Joel Moses
Yeah, no, you're absolutely
Lori MacVittie (09:25.946)
Hmmm.
Joel Moses
correct about that. You know, it doesn't skip past my notice that every era of computing that we've been through so far believes that it is close to eliminating software insecurity. Every single one
Lori MacVittie (09:38.428)
Ha ha ha.
Joel Moses
has had elements in time where, hey, static code analysis or dynamic code analysis, it's gonna reduce software insecurity to, you know, levels that are not even measurable. Well, honestly, what it does is it just creates, one it eliminates one class of
Joel Moses (09:54.903)
problems, but it unlocks a new class of problems. Things that we didn't imagine before. And so I think that that's largely what's gonna happen here.
Ken Arora (10:04.739)
Right.
Lori MacVittie (10:04.787)
I don't know if it eliminates
Joel Moses
And adversarial, well, adversarial use of AI, that is certainly a new class of problem.
Lori MacVittie (10:09.498)
Yeah, no, it is, it's new. I just I don't think that we've eliminated old classes of, not necessarily vulnerabilities, but attacks. Like if you consider I mean a firewall is a firewall. We've had them for 25, 30, a long time. We've had them a long time. They still get deployed and they still stop attacks because attackers don't stop trying to volumetrically overwhelm
Ken Arora (10:36.815)
Mm-hmm. Sure.
Lori MacVittie
your system and firewalls are really good at that. It didn't eliminate it. We learned how to deal with it and how to mitigate it. But then a new class of attacks came up, and those
Ken Arora
Mm-hmm.
Lori MacVittie
were like layer seven at applications. Like, oh no, API attacks and
Joel Moses
Yeah.
Lori MacVittie
HTML and manipulation, and we're figuring out how to solve those. And now we're getting another one, like, oh yeah, logic, orchestration, how these things interact, because that changes,
Lori MacVittie (11:04.602)
right, how the execution happens. And to Ken's point, if we don't think about how someone might abuse
Joel Moses (11:14.113)
No, that's true.
Lori MacVittie
logic, we're not gonna be able to stop it. And maybe AI can help there. Like, tell me, how would you abuse this? Maybe we turn it around, right? And start thinking like the attackers. Just a thought.
Ken Arora (11:26.573)
Yeah. No, that's fair. I mean and going back to Joel's point about complexity. What happened? We started at the network, you know, people wrote software with IP addresses and we said no, no, you don't need to do that. We had firewalls, people went up, we went up the stack, that enabled a whole class of applications. We had came up with REST and HTTP, and now you know, now we have now we're getting the point where AI is writing code and we're--it's spec to code is is how I think of it, right? You write a spec. The time to get code from that spec gets increasingly close to zero.
Okay, what's gonna happen in that world? People are gonna write lots and lots of specs. People are gonna be writing specs for things who have never written a line of you know C code in their life. And are they gonna think about that spec in a way that they're thinking about what's gonna happen if I wrote the spec incorrectly? You know, software engineers are are oft- or hopefully being trained to say, think about how something gets abused. Think about what either a not competent user or a malicious user might do. That's part of the training. People are gonna be running specs.
Joel Moses
Yeah.
Ken Arora
They're not gonna be trained that way. Ergo, to your point, Lori, yeah. There's gonna be a new attack surface and people are gonna go for that.
Lori MacVittie (12:45.372)
And that makes sense. That, I, we don't normally think like that, maybe because we don't want to think like, oh, people are gonna abuse this because then we have to recognize that there are people who enjoy abusing systems and right, doing all of these bad things. And you know, that's sometimes people don't want to face that. They don't think like that. It also requires theory of mind, which a lot of people don't actively develop because that's not a skill that we teach them early on.
So maybe not only are we talking about security having to change, but also how we teach like comp sci and engineering and start introducing that earlier so you get in the practice, right? You have to build that muscle to be able to say, "Okay, this is good. How will they abuse it so I can protect against it?"
Ken Arora (13:33.967)
Yeah.
Joel Moses (13:34.095)
Yeah, I think Ken is right. When people begin to write specs in English language and not computer language, you know, computer languages have things that allow you to focus on types and logic and passing logic from one side to another, whereas a spec that's written in human language doesn't necessarily have to have that. And so the AI tries to fill in those logic blanks.
And if people don't write with, you know, with clarity, if they write something that's ambiguous, again, these systems may produce perfect code, but they will embody bad ideas flawlessly. And that's gonna be an ongoing challenge. I, you know, I think I've said on one of these podcasts before: this is kind of the golden age of the liberal arts degree. People with journalism degrees are actually fairly well equipped to navigate how to write specifications for applications because you know that's what it trains you to do.
It trains you to write unambiguously. But not everybody's gonna have that training out of the box. And so yeah, again, there's still gonna be flaws in logic, but I foresee a time when the weakest link is no longer code. The weakest link is authority and logic and that security tools and security teams are gonna need to shift to address that.
Ken Arora (15:02.064)
So
Lori MacVittie (15:02.204)
Yeah. Okay. Right. I mean
Ken Arora (15:04.634)
So, so-
Lori MacVittie
you're saying boundaries and behavior, which is
Joel Moses (15:07.577)
That's right.
Lori MacVittie
I've been, that's my current, right, mantra.
Joel Moses
Boundaries and behavior. B and B.
Lori MacVittie
Boundaries and behavior. B and B. That's that has to be our new approach to how we secure things. What are the boundaries in which they can operate and what behavior indicates they might be trying to get outside of those boundaries? And you know, we need to move toward more of that because prompt is the new layer.
Lori MacVittie (15:31.748)
Right?
Joel Moses
Yeah.
Lori MacVittie
Free text. That's, you know, you're right. I mean people write prompts. I've seen some prompts. Wow. Okay. That's why this is dangerous. Yeah.
Ken Arora (15:47.109)
Yeah. So let's go forward a few years. Let's imagine a future five years from now where, you know, writing developing code, developing agents is democratized. Anybody can do it. They embody exactly what you told them to do.
Ken Arora
In, what are the safeguards? And I-
Joel Moses (16:06.129)
Okay Lori, don't worry, it's all right.
Lori MacVittie (16:07.612)
I can't!
Joel Moses
It's all right.
Ken Arora
Ha ha ha.
Lori MacVittie
I can't! I don't want to
Joel Moses
Prepare yourself.
Lori MacVittie
imagine this future. It's scaring me.
Joel Moses
Just prepare yourself. That's it, take a deep breath.
Lori MacVittie
Okay.
Joel Moses
Ha ha ha.
Ken Arora (16:15.632)
What do we, and the analogy and I'm just sort of thinking this through, the analogy is a little bit like dealing with other I'll say human agents, right? You have human agents. How do you-. Governance. Goes back to governance still, right?
Joel Moses
Mm-hmm.
Ken Arora
How, what is, how do you put guardrails around them? And I obser-
Lori MacVittie
Boundaries.
Ken Arora
I guess just one observation I have is the guardrails sometimes have to be a lot more formal than the general directions.
Joel Moses
Yeah.
Ken Arora
Right, laws are a lot more prescriptive. Guardrails around LLMs might be more deterministic things like rule-based systems.
Joel Moses (16:55.311)
Yeah. I think that's correct. Security isn't becoming obsolete, it's just getting promoted and changing form. It's evolving from, you know, the rule, the static rule, please escape your SQL queries, to please ensure your autonomous AI agents don't accidentally destabilize civilization. You know, that's all.
Lori MacVittie (17:13.124)
That's a simple rule. You've got that one written, don't you, Joel? You want to share that with us? How do you
Ken Arora (17:19.307)
Well,
Lori MacVittie
How do you codify that?
Ken Arora
Right. Until the AI says, "Oops, I destroyed civilization. My bad."
Lori MacVittie (17:25.316)
Yeah. Ha ha ha.
Joel Moses (17:25.893)
Ha ha ha.
Lori MacVittie
Who does it tell? Who does it tell?
Joel Moses
"Sor, sor, sorry about that."
Ken Arora (17:31.657)
Ha ha ha.
Lori MacVittie (17:31.837)
Ha ha. Oh. Oh, my goodness. Oh, my goodness. So we could, I'm sure we could continue this conversation and continue scaring me. But so I don't have a heart attack and we can wrap this up in a reasonable amount of time. You know, it's not here yet, but it is moving that direction. People are using a lot of AI today to write code, to fix code, to help augment these processes. So, you know, what can we take away from it's not there yet but it is moving there and we do want to leverage AI to help us get more secure?
Joel Moses (18:07.769)
Yeah. You know, my takeaway from this is the future of security is no longer about fixing, shielding, or protecting broken code. It's about governing systems or operating systems well that are finally powerful enough to work exactly as intended. That's my takeaway. We're shifting the game in security. And it's gotta be more about operational control and like you said, boundaries.
Ken Arora (18:38.523)
Yeah. Yeah, I, it's, my thought my take would be pretty similar. It is, it will do exa-, AI is getting closer and closer to the point where it will do exactly what you tell it to do. And so now the onus shifts; security becomes be very careful about what you ask it to do.
Lori MacVittie (18:58.684)
Ha ha. That
Joel Moses (19:00.689)
It's pretty simple.
Lori MacVittie
It is. It is pretty simple and I agree, right. We're not there yet. It is getting better every day, but we do have to start looking up the stack, right? That value and security, everything has to move up the stack. How do we
Joel Moses (19:15.557)
Yes.
Lori MacVittie
deal with the new challenges introduced by new ways of integrating, orchestrating, and operating? And which is of course, yeah, gonna add complexity because we just added a new layer and that's
Lori MacVittie (19:28.802)
what happens. And complexity is the enemy of security. Right? I've heard that somewhere. So, remember, AI isn't gonna do your job right now, but start using it to help you figure out what those attackers might be doing. That's a wrap for Pop Goes the Stack. So hit subscribe before the next wave of tech buzzwords hits your planning meeting.