Subscribe
Share
Share
Embed
Coding pipelines are evolving and AI agents are taking the wheel. In this episode of Pop Goes the Stack, F5's Joel Moses teams up with Buu Lam to dive into “vibe coding”—where tools like Claude Code and GitHub Copilot plan, build, and optimize apps faster than humans can debate lint rules.
But is faster better? While agentic AI unlocks game-changing efficiency, it also introduces new risks: API keys hardcoded into apps, runaway GitHub actions, and a stark need for guardrails like sandboxing, runtime tripwires, and logging. As we embrace smarter pipelines, how do we stay in control?
Join us as we explore the promises and pitfalls of shifting left with AI agents—and why the era of “self-improving code” is both exciting and terrifying. The machines are coding. Are you ready to debug?
Creators and Guests
Host
What is Pop Goes the Stack?
Explore the evolving world of application delivery and security. Each episode will dive into technologies shaping the future of operations, analyze emerging trends, and discuss the impacts of innovations on the tech stack.
00:00:05:06 - 00:00:21:20
Joel Moses
Hi everybody and welcome to Pop Goes the Stack, where we take emerging tech flip it upside down and see what falls out, besides our own fragile sense of control. I'm Joel Moses filling in as your lead host today for Lori who can't be here. I'm here to talk about what shift left means when the agent is writing code
00:00:21:20 - 00:00:48:16
Joel Moses
faster than we can argue about Linting rules. We're diving into the new reality where guardrails start at intent and policies kick in before a single token hits the repo, and your delivery pipeline might have to babysit an overenthusiastic vibe coding model with control of your keyboard. So buckle up. This one gets spicy. With me today is Buu Lam, Director of Community Evangelism, or as I like to call him sometimes, Mr. DevCentral.
00:00:48:18 - 00:00:49:20
Joel Moses
Welcome, Buu.
00:00:49:23 - 00:00:58:21
Buu Lam
Thanks for having me. Excited to be here and excited to listen to one of your intros live. It is always a treat to listen to those. So I feel honored.
00:00:58:24 - 00:01:16:00
Joel Moses
Oh, it's good to have you here. You know, I thought we would begin by talking about a little bit about your experiences with using AI directed coding vibe coding as it's come to be called. Can you describe for us some of the things that you've done for the community work that you've been doing?
00:01:16:02 - 00:01:39:29
Buu Lam
Yeah. Vibe coding for me has been a bit of a journey. I would say, like, well, I don't know, maybe other folks have gone into things like Lovable, you know, type in some sort of prompt and be wowed that some sort of app just appeared based on a couple sentences. So I've gone in between that, of just, hey, what can I do from kind of a one shot
00:01:39:29 - 00:02:02:07
Buu Lam
and here's an application. I also went through, and I recommend this for a lot of people, me not having a development background, I actually went through a crash course or a really quick course on Python. And through that I was like, oh, now I know everything about programing, and so started to play a little bit more with vibe coding.
00:02:02:07 - 00:02:27:02
Buu Lam
And I went from, hey, I'm just going to put in a couple prompts and just create something to actually peeling it back and being like thinking about, okay, now that I know how these pieces come together, can I be a little bit more thoughtful about how an application comes together and actually vibe code pieces of an application? And so, we're fortunate to have GitHub copilot at F5
00:02:27:07 - 00:03:01:00
Joel Moses
Yeah.
Buu Lam
and so was playing with that. And more recently I got on the Claude code train and I don't know if I'm getting off because having AI from the CLI has been fantastic. So again, I've gone from really testing how Claude code, how far can I get with a single prompt to actually starting to have a conversation with Claude to plan an application first, and weighing out all the decisions and having it ask me questions about what I want to do with this.
00:03:01:03 - 00:03:29:22
Buu Lam
And then coming up with a few spec documents. And when I actually go to start building up the application, we plan out a feature list and we actually go through and vibe code a feature list one by one. And it's really allowed me to narrow down certain use cases for applications. And I still don't think I'm necessarily the best vibe coder at this point, but I actually start to see like, there's a range of vibe coding out there.
00:03:29:24 - 00:03:39:04
Joel Moses
Yeah, there is. Now, how would you rank vibe coding as an asset for you? Tell me about what it was able to help you accomplish, for example.
00:03:39:07 - 00:04:06:19
Buu Lam
Yeah. So I have an example of, you know, a lot of the things that people see from me out in public on social media is these videos that I do. And sometimes people will ask me, "hey, how did you be at an event and have that video that you recorded come out that evening or the next morning?" And the answer to that up until a couple weeks ago was, oh, I actually just stayed up all night to do that by hand.
00:04:06:22 - 00:04:29:11
Buu Lam
Now, you know, using Claude, I actually went through and took all of the processes that I have that have actually been processes that are as fast as possible that I can do. First, principles: let's get rid of all the things that we don't need and do that process manually if, quote unquote, manually, if you will.
00:04:29:13 - 00:04:50:18
Buu Lam
And so knowing all of that, I actually worked with Claude to say, "Okay, this is what I'm trying to do here. These are all the things that I have to think about. What could an application look like?" And we went through and made some adjustments and I started to build that. And so that pipeline of media: taking the content in, doing the transcriptions of the content.
00:04:50:21 - 00:05:11:11
Buu Lam
Those are to me are really important because I do deal with a lot of international folks and I want, although it's English, I want the English to be right so that auto translation can work off of a caption file. So I do transcripts for folks and I have OpenAI Whisper integrated into this application to do that.
00:05:11:13 - 00:05:36:23
Buu Lam
OpenAI Whisper has it's, understand it has different size models and different variations within those sizes as well. But you can, you can take the approach of fine tuning of Whisper or you can take the approach of actually just adding a vocabulary file to Whisper as well. So in our industry we've got a lot of acronyms, we've got a lot of terms,
00:05:36:26 - 00:05:58:12
Buu Lam
and we've got as I travel internationally a lot of accents as well. So I can actually build a vocabulary file. Something that I can export, import as well, so it's transportable between projects. So I can do those transcriptions super easily. This app monitors a folder for MP4 files and then I can cut my social copy.
00:05:58:15 - 00:06:17:02
Buu Lam
I can actually do edits on the fly within there. So it'll allow me to just break apart a video and be able to crop out different parts and just sends it off to FFmpeg and spits that out back to me. I have stats tracking in there as well. Another really great part of this is LinkedIn
00:06:17:03 - 00:06:41:10
Buu Lam
is not very open with its API for folks. It protects the social network. But you can build, you can have Claude build extensions in Chrome that will do things that don't need the API to get the information that you need, at least in much fewer clicks as well. And so I built that into it.
Joel Moses
Interesting.
Buu Lam
You know, little add ons to make everything go faster.
00:06:41:11 - 00:07:02:25
Buu Lam
So, at the net of it is I took a business process that I see value in, collapsed that business process into an application, kind of proved that that saves me time. I think it'll save my team time as well. And then we can look at, okay, what does that look like if it's actually a production system and not just the little script that I'm running on my laptop.
00:07:03:01 - 00:07:04:09
Joel Moses
Right.
00:07:04:11 - 00:07:26:04
Joel Moses
Well, it sounds good and, effectively, the net value to you is that you get to sleep at night while
Buu Lam
That's helpful.
Joel Moses
all of this post-processing occurs, which is, which is wonderful. Now, a lot of this is essentially describe, what you're describing to me is a managed pipeline. So, in one side goes the raw video and out the other side comes a finished package,
00:07:26:04 - 00:07:44:24
Joel Moses
and all the way down to applying it to the various social networks that you need to apply it to. So, you're managing a pipeline and you talked about moving it to production. So let's roll back a little bit and talk about what you've done. What you've effectively done is you've created a structure that vibe codes,
00:07:44:24 - 00:08:14:08
Joel Moses
and you have, you use directed input to an AI system to help you plan out what elements that you could bring to bear for that pipeline. So, your experiments, for example, with Claude code, you're developing effectively a development plan, a product plan so to speak, and then you're executing on that. Now, there of course is a lot of interest in having that whole solution managed as part of an ongoing code maintenance pipeline.
00:08:14:08 - 00:08:39:20
Joel Moses
So you have the AI in the background continuously working to improve the pipeline that you've helped that you've developed. And so a lot of this is beginning to tie itself to agentic AI. So not just using vibe coding as a tool to assist the human in coding, but also applying vibe coding structures to automatically create enhancements to the pipeline when it notices problems.
00:08:39:21 - 00:09:03:07
Joel Moses
Now, it's the I guess, you know, there's there was a lot of talk in the development circles about shifting left, about doing things with coding API first. And developing a shift left approach to your use of agentic AI in your pipeline is also becoming a popular thing. Now, what are some of the risks that you can see in that?
00:09:03:07 - 00:09:16:18
Joel Moses
Like describe for me some things where maybe the vibe coding system that you used maybe took a wrong turn. Did you have any of those occur?
00:09:16:20 - 00:09:48:11
Buu Lam
I've seen it take a wrong turn in terms of creating the thing that I asked it to create, and not in that it did something wrong, but I wasn't descriptive enough. And I think that's kind of where the sharp edges are, because I could see that it's not asking me about package versions. So there, you know, you would probably need a Claude skill to go out and do a scan to see if hey, the packages that I'm pulling down is there anything that I should be concerned about here?
00:09:48:13 - 00:10:21:16
Buu Lam
There's the tests that Claude will do as it's built, as it's testing the features that I asked it to build. And I can see it correcting itself, but within there there's no security validations as part of it as well. So you probably need to build
Joel Moses
That's a good point.
Buu Lam
skills around that, as well. It's also just up in building the Docker, in my case, building up a Docker container after it's done doing local testing and packaging it for me, because I want to be able to just spin it up with a Docker compose.
00:10:21:18 - 00:10:43:05
Buu Lam
And so from that respect it's also pulling in from the requirements file to see, you know, what other things do I need to build up for this environment. But all of those things along the way, those are all the things I think we talk about shifting left. And those are things that, you know, if you haven't adopted all of that already, you probably need to think about that just in your own little environment there.
00:10:43:07 - 00:11:09:21
Joel Moses
Yeah. Now, what other things do you think you should think about if you, say you want to enhance the pipeline that you created and you want to use agentic AI to its fullest extent and have the code effectively self improve, to notice new library versions, as you mentioned, and incorporate those automatically without having to go back and run through your prompt sets again in order to generate new code.
00:11:09:23 - 00:11:14:18
Joel Moses
What are some of the things that you think you might want to watch for if you do that?
00:11:14:21 - 00:11:50:28
Buu Lam
Yeah, I've seen the wonderful folks at JFrog actually gave me a demo of this before, but having global or at least corporate wide policies on the artifacts they're brought in and there's, I think there is security implications to that, but there's also compatibility implications to that as well. You know if you, you might be creating something that is more client server and you're creating the client side of things and you're creating something that could potentially pose impose something on the server side or vice versa,
00:11:51:00 - 00:12:23:07
Buu Lam
I think there's implications to that as well. But Claude can just go out and pull anything it wants at the end of the day into your stuff. So, if you're not paying close attention to what actually comes in and then, in addition to that, like things like static code analysis that you would normally, hopefully, normally people would do as well or dynamic analysis or dynamic security testing on your application, that's not built into the into that pipeline as well.
00:12:23:07 - 00:12:45:09
Buu Lam
So, you know, translating, all of that and, you know, I had Claude build some GitHub actions for me to do a rebuild of a thing that I was doing as well. But within there also know it rebuilds once a day and within that there isn't--hey, this is not a production application by the way--there's
Joel Moses
Right.
00:12:45:11 - 00:13:05:10
Buu Lam
no security checks in those GitHub actions that it's building.
Joel Moses
Right.
Buu Lam
And I'll give you an example here. I was building something that needed an API key. That API key has constraints on it. But Claude asked me to put, hardcode the API key into the code. And like,
Joel Moses
Yeah.
Buu Lam
hold up Claude. I don't want to do that.
00:13:05:10 - 00:13:08:19
Buu Lam
And we worked together to, you know, do secrets management.
Joel Moses
To resolve that,
00:13:08:21 - 00:13:30:01
Joel Moses
yeah. So effectively what you're telling me is, first of all, you need a place to evaluate as part of the planning loop. So putting evals in your planning loop, it's a bit like having a personal trainer that slaps the donut out of your hand, before you commit to your repo, right. So having a place to evaluate what's about to be applied.
00:13:30:03 - 00:13:47:26
Joel Moses
I think also maybe using sandboxing by default. So, you know an agent, if you ask it to do things you want to have a box in which it can draw with crayons. It's like a toddler with a box of crayons. They'll still try to color the dog, but at least they'll ask permission first, right?
00:13:47:29 - 00:14:16:24
Joel Moses
So I think that's another mechanism that can be brought to bear there as well. Another thing is, you know, these services and you used a GitHub action as an example, never incorporate as part of their security model any secrets or keys. But even beyond that, there are elements where you might want to incorporate a check for, what I would call, over utilization.
00:14:16:25 - 00:14:41:26
Joel Moses
So instead of instantiating 50 different GitHub actions, try to figure out ways that you can reduce the instantiation to one, because that has a direct correlation to cost. So runtime tripwires, I guess. Maybe, maybe watching to see if the agent is intently spawning infinite threads or using four actions where one will suffice.
00:14:41:29 - 00:14:53:15
Joel Moses
So, these are all elements that you're going to have to catch or write into your prompts that guide the actions of your agent. Any other things you can think of?
00:14:53:18 - 00:15:21:19
Buu Lam
You know, just on that note too, that last point there is that my early attempts at vibe coding where I didn't really know what I was doing, I was passing full context back and forth, and I was using a GitHub plugin, or sorry a VS Code plugin, that was showing me the cost for each back and forth. And the number kept going up and up
Joel Moses
Up and up.
Buu Lam
and up because it kept adding more stuff and something broke and copilot was like, 'oh, I'll just add in some debugging.'
00:15:21:21 - 00:15:49:23
Buu Lam
Well, it added debugging everywhere. So that number just kept
Joel Moses
Going up and up and up.
Buu Lam
So I can see for GitHub actions, you're paying for usage on that one as well. So seeing that go up and up is yeah, that's a scary thought for letting things get away from you. I've also seen more recently Docker in your sandbox environment there as you're building, if you're using Docker they have, seems like, policy-driven
00:15:49:25 - 00:16:06:18
Buu Lam
MCP or they, policy-defined MCP servers, that they make available. And also MCP gateway for egress control or and MCP server as well. So that might be a scenario there where, you know, people incorporate that into their sandbox development; they can see what's going on.
00:16:06:20 - 00:16:24:11
Joel Moses
Yeah, it's a good idea. The other thing that I can think of is that, you know, if you're going to have a system that conducts these actions, every action should be logged. Make sure that you can trace it back. I mean, you talked about a GitHub action that needs to use an API key.
00:16:24:12 - 00:16:49:09
Joel Moses
Well, if you give an agent the access to use that key and then expect it to stay within some confines that you set, you need to know when it goes outside of those confines. And you can't do that unless you're logging the action sets that it takes. You know, if an agent spins up 300 GPU instances, you're probably going to want to know exactly what conditions that was done under.
00:16:49:11 - 00:16:57:06
Buu Lam
Yeah. Yeah. Hopefully you have quotas set on your cloud accounts and whatnot that'll hopefully trip off some of that stuff.
00:16:57:09 - 00:17:27:11
Joel Moses
Yeah. That's true. Now, let's reverse course. We've talked about the dangers in using agentic AI and some of the things that you need to use that you might need to think about if you're going to design an agent based coding system into your pipeline. Which, but there's a lot of interest in doing so because, you know, I have been in development for a long time and I've sat and watched two engineers debate a single lint rule for 45 minutes, whereas, you know, 45 minutes
00:17:27:11 - 00:17:53:00
Joel Moses
an agent can write, test, ship, document and benchmark a feature. So, yeah. I mean, in the same time that it would take to for the single lint rule to be debated an engineer shipped no features and an agent shipped one. So clearly there's some value to be had here. Any other benefits to incorporating agentic coding?
00:17:53:03 - 00:18:13:27
Buu Lam
Yeah, I would say, you know, all of those things--your wish list of things that I wish I had--if this just existed, I could do my job so much more better or faster or whatever it might be that benefit getting that proof of concept out the door and showing that value, and then that becomes a justification.
00:18:13:27 - 00:18:22:28
Buu Lam
But, you know, being able to validate something super quick and pursue business I think is super valuable for businesses right now.
00:18:23:01 - 00:18:37:09
Joel Moses
Yeah, I agree. We're ticking down to the close to time and so we always like to finish out Pop Goes the Stack with some lessons learned. What did you learn today about the possibility of an agent coding for you?
00:18:37:12 - 00:19:05:02
Buu Lam
I would say that even though you can go out to those single short platforms to build something, just taking a little bit of time to learn, and again I'm no expert, but taking a little bit of time to learn a little bit of software development principles is going to help out in a lot of ways. And actually, I would increase my knowledge now just because I know I can work with Claude better when I know more about that.
00:19:05:02 - 00:19:19:12
Buu Lam
So I think vibe coding is not a hands off situation. It's very much actually the more knowledge you have and can apply to vibe coding, just it's jet fuel to really smart people.
Joel Moses
Yeah.
Buu Lam
So, hope to become a really smart person.
00:19:19:15 - 00:19:40:13
Joel Moses
Yeah. I think I learned that when you're talking what you should inject into your pipeline, you can't forget to do the things that you would do with an inexperienced coder. You have to know who asked the system to do what and when. So making sure that you record transactions, that every action is logged, and that you can trace it back.
00:19:40:14 - 00:20:02:12
Joel Moses
That's important and as you embed agentic AI into these systems, you need to ask it to do that for you. Runtime trip wires, having some sort of outside system that mediates and looks at the elements of cost, I think that's important as well. Sandboxing by default, making sure that you are very explicit about what the agent can do and what it should never do.
00:20:02:14 - 00:20:09:00
Joel Moses
And then putting evaluations in your planning loop for your product plan. These are all important things to accomplish. But
00:20:09:02 - 00:20:13:15
Buu Lam
It sounds like you've created vibe coding regulations on the fly, Joel.
00:20:13:17 - 00:20:34:03
Joel Moses
Well, you know, I'm trying. I'm absolutely trying. Unfortunately, that's all the time we have for today. That's, if you enjoyed learning how to keep your agents from YOLO shipping code at 3 a.m., go ahead and hit like and subscribe. We'll be back with more ways to stay one step ahead of the machine and preferably upstream of their pull requests.
00:20:34:06 - 00:20:38:03
Joel Moses
For now, that's all. Thanks for listening to Pop Goes the Stack.