Subscribe
Share
Share
Embed
Learning to apply solid engineering principles to AI-assisted software development.
Hello and thanks for listening and watching. I'm Jonathan Hall. I am trying to learn how to do better coding with AI. And today I have with me Brian Finster. Hi, Brian.
Bryan Finster:Hey, how's it going?
Jonathan Hall:I'm doing well. Thanks for joining me. Before we started recording, I was telling you that I've kind of been a potentially a late adopter for this this whole AI hype. You know, when it first came out, was kind of overwhelming and I guess it still is. But I have started dabbling with and I actually use Cloud Code now pretty regularly.
Jonathan Hall:It's sort of the way I code these days, but I I bumping into it all the time and I'm still not happy with my workflow and the way it works. And so I'm trying to learn from people like yourself, hopefully, who have had some success with it so I can hopefully learn from your mistakes without having to make them all on my own again. That's the reason I'm doing this. And hopefully the listeners and viewers can can learn something too. So before we dive into Coding with AI, why don't you do a brief intro, who you are, what you do, and and how you, you know, maybe use AI?
Bryan Finster:So I've been a developer for thirty years, mostly poorly. You know, I've been, I managed to get a chance to fix that when we were given the opportunity to try to to figure out continuous delivery at at Walmart back in 2015. Took it seriously and learned a whole bunch of stuff along the way that's mostly a people problem.
Jonathan Hall:Mhmm.
Bryan Finster:And I've been applying that skill since then, applying it now to everything I do. I mean, everything I do filters through that that lens of how do I get to production today. Really clarifying question to ask. Well, how do I deliver today's work today? You you find a lot of problems by doing that.
Bryan Finster:When... I'll say that my background with AI is that when people started talking about coding with ChatGPT and replacing developers, I went out there to go try it, and it was garbage. Yeah. I... And then people kept harping on it, and I went to go write a blog post about how dumb they were.
Bryan Finster:But I had I had to go and gather evidence to do it. And between the time that I first tried it, it was garbage. To the time I tried to gather evidence, it was no longer. I mean, it wasn't awesome, but it wasn't... I could see.
Bryan Finster:I could see the potential.
Jonathan Hall:Uh-huh.
Bryan Finster:Right? And and then I I went to a hackathon that John Willis and Patrick Dubois posted in New York. In less than a day, I created a rag using a language I don't know to go and query minimumcd.org and respond back with open source models, respond back to questions that weren't just cut and paste answers. It wasn't a search. It was responding with meaningful answers to the questions I was asking.
Bryan Finster:And I thought, yeah, I might have to dig in. And so I've been on that journey ever since.
Jonathan Hall:Okay. That's fascinating. So it's it's not completely dissimilar to my journey. My initial sort of response when everybody started hyping about it was, it's a cute toy. It's kind of, you know, it's fun to make silly memes and stuff like that.
Jonathan Hall:You know, as LLMs in general, right?
Bryan Finster:Boss was using it for haikus for his wife. Was like, yeah, it's probably good for that.
Jonathan Hall:Yeah, exactly, exactly. But as a serious engineering tool, I didn't see it for quite a while. And it was probably... So I guess it was probably about a year ago, I first started having chat GPT write some code snippets for me, especially in languages I didn't know. So I would just have chat GPT, you know, make a function for me in a language I wasn't particularly familiar with or something.
Jonathan Hall:And the other kind of worked, didn't scale very well. But, you know, as a as a first blush, it would get me there. And it was probably about three months ago I first started using Claude code. I was skeptical that it would be stupid. But the first task I threw at it, I had a I had a function.
Jonathan Hall:I don't remember what it did anymore, but I had a function with a big to do comment like refactor this to do something else. I already had good tests around it. And I I said... So I said, Claude, just refactor this for me. It started and about twenty minutes later, it had a working refactor, which completely impressed me.
Jonathan Hall:I was expecting it to just fall over. Now, the the code was terrible. It was absolutely terrible code. Lots of duplication and the wrong abstraction. It was terrible, but I threw it away.
Jonathan Hall:But I was impressed that it actually did something that worked. So that sort of opened my eyes to maybe maybe if I approach this differently than just like go do a bunch of work for me, maybe I can make some progress. And since then, I've I've started learning to write some skills for plot and I have it using TDD, kind of. It's not very good at it, but with the right guidance, I can get it to do what I need most of time. That's kind of been my journey.
Jonathan Hall:But let's talk about CD because that's your sort of specialty, I guess. That's what you're harping about on LinkedIn all the time. And that's sort of, I think the lens that you see engineering, software engineering through.
Bryan Finster:The CD habits really unlocked my ability to use Cloud for development seriously. Because, yeah, Cloud Code came out... Well, first, even before Cloud Code came out, I was doing the whole chat window thing. Yeah. I was getting...
Bryan Finster:You know, like you said, think it's spotty results. But then I started going, okay. I was getting spotty results as a developer too on Teams when we were just kind of throwing shit against the wall and seeing if it's stuck, you know. But what really affected for us was diving into develop... To behavior driven development.
Bryan Finster:So let me see if that works. And I started generating feature files and saying, implement this feature file. And so now I had all these test scenarios. And then that was working, and then that was a lot of work. So I said, here's what I'm trying to accomplish.
Bryan Finster:Create a feature file for that. You know? And I go and review the feature file and and and go and make edits, for it. And, that worked really well too. I actually gave a talk in Dallas this past August about...
Bryan Finster:And it's called AIs Taking Our Jobs.
Jonathan Hall:Mhmm.
Bryan Finster:Where I when I talked about this this workflow I used, I even had QR codes to the GitHub commits that you could go and see the workflow that I did. It wasn't BS. And created a tool I needed to go and and download information for what what I now call PocketFinster. This is gonna go and scrape Medium for my blog posts. Mhmm.
Bryan Finster:Because their their official method of doing it is terrible. But it took me about four hours to generate this tool, and was doing things I didn't know how to do on front end. You know, like, don't know how to handle infinite scroll. I'm not a front I'm not a front end developer. Yeah.
Bryan Finster:Yep. I had a tool. My four hours were great. And then I explained why it wouldn't matter because the rest of your organization's information supply chain was garbage. You have product owners who can give you vague stuff and throw it over the wall, and you've got a terrible bureaucratic delivery process.
Bryan Finster:You're accelerating developers if you work this way, and all it does is just accelerate your risk. That was the that was the mic drop of my talk. But, no. The the the lessons I learned was, okay. Well, behavior driven development works.
Bryan Finster:The other issue is that, you know, at the time, context windows were a lot smaller than are now, but they're still pretty constrained. But, you know, I relate it back to the principles or it's cognitive load. How do I handle cognitive load? Right? A small batch size, frequent commits.
Bryan Finster:So we're gonna... Small pieces of work frequently. And I learned that if it didn't have enough information about the goals we're trying to accomplish, that it would just go off the rails. And so then I learned about client files or agent files, and how do I make sure they have context about what's the current goal, and then what's the next step towards that goal. Right?
Bryan Finster:And I start... I I just treated it exactly like I would treat a development team. Make sure that they have context to what our goals are that that... That's, you know, not deep, but broad, and then deep on the thing that we're trying to accomplish right now and then go. Right?
Bryan Finster:And then... So then how do I handle, you know, the the... There's there's problems with its... You know, it would go delete tests. I didn't want it to, or it would, you know, write terrible code.
Bryan Finster:And I was like, okay. Well, how do I handle that? Is the tool bad or do I need to add validations in my pipeline? And so when I talk about a continuous delivery pipeline, I'm not talking about everything that happens post commit. Okay?
Bryan Finster:I'm I'm talking about everything that happens from the time we come up with an idea to production and get feedback from the idea. That's my continuous pipeline. Right? So what can I add to my CD pipeline to make those things, you know, not happen? And so when you're working with continuous delivery, you're always looking at, okay, I have failure mode.
Bryan Finster:How do I detect that failure mode as early as possible, and either prevent it, fix it, or break the pipeline, so it doesn't go to production? And so I started looking around and then, you know, Claude released skills. Okay. I can use these skills for the... These things, I think.
Bryan Finster:Well, I didn't know how to write a skill. So I asked Claude, how do you write a skill? Paper wrote a skill for me. Apologize for the noise. And and and so it's just...
Bryan Finster:It was just the problem solving around how do I constrain... How to put the guardrails around this the same way I would on a development team. Know? Because... People talk about, oh, it's nondeterministic.
Bryan Finster:I'm like, have you looked in the mirror? It's much more deterministic than the average developer is because the average developer is tired. They're a bad mood. They're stressed by timelines. And Claud didn't have any of those problems.
Bryan Finster:Their agents just don't have those problems. They're non deterministic. Cool. But they're non deterministic in deterministic ways. So we can write fitness functions and guardrails around that to constrain them.
Bryan Finster:So then... So what's the next thing? Well, I can't just trust the coding agent to follow all the rules because it won't. But what I can do is go... The reason that we don't have specialists on the team is it creates silos.
Bryan Finster:Right? We want generalists on the team because that's how you eliminate silos. You want people to be t shaped, but I don't need t shaped agents because there's no silo there. They run really fast or automated. So I started generating specialist agents.
Bryan Finster:Okay. So you've got standard tools for things like linting and testing and performance, all that stuff. But the things that we would do code review for or the, you know, the the the things we wanna maintain code quality, that takes human judgment, except it doesn't. These are just patterns. They're well documented patterns.
Bryan Finster:So I created a domain driven design agent, a hexagonal architecture agent. I generated an agent that all it did was say, do we have meaningful names? And it threw those at it. Now, I'd say about a year ago, my opinion was, well, the code just doesn't matter. It just won't matter.
Bryan Finster:It could be garbage code as long as it runs and does and behaves the way we want it to at the end. But then, you know, I started saying, okay, well, we have to roll it out to an enterprise and we have enterprise pricing. We have token utilization. Even on the non enterprise thing, you can run out of tokens. Mhmm.
Bryan Finster:So I learned that tokenomics was a thing. And then and then I learned that if you want good tokenomics, that you have to have well structured code, better structured than most humans would would tolerate.
Jonathan Hall:Mhmm.
Bryan Finster:And it would would demand. Right? It's like we tolerate garbage code, but the cost is hidden because it just comes out of our salary. But the cost is not hidden when you run out of tokens.
Jonathan Hall:That's an interesting insight.
Bryan Finster:Yeah. And so I'm learning all these things by breaking things. And instead of going, oh, it's garbage, I'm like, okay. What's the engineering solution to this? Because that's what you have in CD is what's the engineering solution for this, not the process solution, the engineering solution.
Bryan Finster:You talked earlier about the learning. I've had to learn new ways to learn to keep up with what's going on. I... I'm I'm at the spot where I was when we were first doing continuous delivery, where I'm, like, absorbing as much as I can from anybody, if I need, you know, podcasts and RSS feeds and anything I can get. I'm I'm at that spot now again because it's changing all the time.
Bryan Finster:I saw a YouTube video today that just completely changed how I'm gonna go and maintain agents and skills because they keep updating the models, which means that the agents and skills that you have today are either obsolete tomorrow or you don't need them, or they don't even work as well or they work worse tomorrow. So what do they do? They created a plug in on Claude that would go and make sure that you... That it it, like, retrains your skills.
Jonathan Hall:Right? Okay.
Bryan Finster:Well... That's my pipeline for skills and agents, you know, so this... It's fun and it's incredibly frustrating because I I have actual work to get done. I don't wanna go and maintain the tool. Right.
Jonathan Hall:And that's why I've been such a late adopter of this, because I didn't wanna sink all my time into learning a tool. I wanted to get work done.
Bryan Finster:Well, this is the problem really is that things were actually moving relatively slowly until November. On the... And and then Gemini and Anthropic made changes that massively improve their ability to do real work with it. And they're they're... See, this is the thing.
Bryan Finster:I I I know somebody who was building a air gapped AI solution for the military.
Jonathan Hall:Mhmm.
Bryan Finster:Okay. He didn't know how. He started with ChatGPT and started building it and then used the tool he was building to build the tool. Anthropic and Gemini, and I'm assuming OpenAI are doing the same thing. And so the acceleration that you're seeing in change is because they're using the tool to build the tool.
Jonathan Hall:Mhmm.
Bryan Finster:And so things are changing really, really fast. And so I've been trying to figure out what the patterns are that are good patterns. Been documenting those out on an agentic CD subsection of minimumcd.org
Jonathan Hall:Oh, fancy. Uh-huh.
Bryan Finster:To help people. I'm just learning... I've been learning out loud too. Right? I've been...
Bryan Finster:As I learned, we have... Well, me, I update that that website to... This is the current thing I know. I I did a recording this morning where I told I told Adrian Stanick. I was like, as of March 2026, I believe this is true.
Bryan Finster:It may not be next month.
Jonathan Hall:So much for evergreen content. Right?
Bryan Finster:Oh, man. Man. But I've... You know, what I've done is I've created a whole set of tech writing agents and indexing agents for content, for helping to maintain that site so that it's not... And and just onerous toil to try to update things quickly.
Bryan Finster:That I can update things in minutes with cross linking and glossaries and all kinds of stuff, because I have those agents helping me out while I while I say, okay. We need to we need to talk about this.
Jonathan Hall:Yeah. With with that backdrop, I would like to understand what your tool set looks like today, understanding that it'll be different in a month. But just to give us a snapshot of of which agents do you use, which tool do you use, how... What does your tool set look like right now?
Bryan Finster:So we're at work. We're we're right now unplugged. We may be switching to Gemini just because of enterprise pricing. As far as I can tell today, March 2026, Gemini and Plug are about on par when it comes to software development. They'll have different edge cases where one's better than the other and other things, but I don't care about that.
Bryan Finster:I care software development. That's why it won't matter except I'll have to go refactor everything. My... Thing I'm working on right now is building out an agentic development team that has all the... You know, has coding specialties.
Bryan Finster:It's got documentation specialties, you know, like, again, domain driven design, executable architecture. I've also created... And so you can see what I'm working on at the... On GitHub. If you go to my...
Bryan Finster:To, you know, GitHub and go to bdfinst/agenticdevteam, There's dashes between those words. That the repository is current as of today. It's actually sucking in a plugin. That's another repository I have that's called CabKiller. This job is code review.
Bryan Finster:This is another problem. Right? This is another engineering problem. How do you do code review when, you're generating codes so fast that you can't possibly review it? Well, it's an engineering problem.
Bryan Finster:Why do we code review? Identify all the... And this is just like a CD pipeline. We have all these manual processes for delivery. Why do we have the manual process?
Bryan Finster:How can we identify that this thing is is worthy of production? Let's automate it. Same thing here. What are we code reviewing for? So it's just because we have opinions, we wanna inflect our opinions, codify the opinions as a skill.
Jonathan Hall:Mhmm.
Bryan Finster:And so I I didn't know. I I went through and did a hard look at all the things we should be code reviewing for, and I just created agents for the things since you don't have tools for it. And and and then what do you do now? Well, you spot check code. You don't you don't have code review as a blocking thing.
Bryan Finster:You have code review as something that you go and look at hotspots post deploy. You know? If it's if it's running, it's good. If it's not running, your tests are bad. Improve your tests.
Bryan Finster:And so I've got this AgenTex Scrum team, but the way it's it's it's really focused on making sure we have, number one, good context management. Context engineering becomes very important because there's a... The the thought behind that is that you have a context window within any session that you run. Okay? Mhmm.
Bryan Finster:If you exceed 50% of that context window, then results start degrading quickly. You get less accuracy. The the... That session will start having amnesia. It'll start just forgetting things because things will just drop off the context.
Bryan Finster:Now that happens less today as far as amnesia because things like Cloud Code will auto compact, but you're still just auto compact in the conversation and you're not clearing it, which goes on to the CI behaviors, right? Do a small task, we clear the context, we start the next task. But that's toil. So the purpose of this team is you have an orchestrator, its job is to understand the goal. It has a very thin context understanding of data, thin and broad, here's the goal.
Bryan Finster:It's a specialist in assigning work to other agents. So each one of these other agents has, depending on what it is, has different levels of model required to perform that task. Coding doesn't take a high level model. Other other tasks do take a high level model. And so it automatically selects the model based on the kind of task it is, assigns it to the kind of agent.
Bryan Finster:Those agents get the level of context they need to do their work as separate sessions with separate context windows.
Jonathan Hall:Mhmm.
Bryan Finster:And followed by, okay. We've completed this unit, this feature. Now let's let's go hit it with code review with different agents because we're doing an adversarial review of everything with different agents. We don't... Just like a team, you you know, I don't even trust my code.
Bryan Finster:You will definitely want to try our code. Right? Yeah. Same process, just automated and fast. This...
Bryan Finster:And I'm I'm getting pretty close on on stuff I like, but of course, keeps changing. So I'm always having to refact that things change. But that concept of, if you're on a development team, now you're on a team of teams, every developer runs a team. They need to have... Coding skill becomes far less important than technical leadership skill, product management skill, defining what good looks like skill.
Bryan Finster:If you don't have a clear vision where you're trying to go and how you know you got there, you should probably start there before you start trying to generate code with it.
Jonathan Hall:So let's talk about code a little bit. At what level do you even care about code at this point? Earlier you said a year ago you thought the code didn't matter. It sounds like you've changed your opinion at least somewhat, but it sounds like you're not reviewing the code. You're spot reviewing it maybe.
Jonathan Hall:Just, yeah, in general, what what is your relationship to code these days?
Bryan Finster:So number one, the code quality point of view, is it just efficient for tokenomics? That's that's what I care about code quality is it is it good for the bot? And I'll have... I have a tokenomics skill that all I'll do is is highlight areas. So we need to fix for code.
Bryan Finster:And that's, you know, it's just one of the code review agents is how's it look for tokenomics. I care about it this... The the... You know, the thing is is I have always been DevOps before DevOps existed. I was Mhmm.
Bryan Finster:Deeply involved with the problem we're trying to solve, coming up with a solution, and then being responsible for the operations at the end. It's almost the entire time I've had this job other than when Walmart got stupid and copied Microsoft. I care about ops. Okay. So is it functionally correct?
Bryan Finster:Is it performant? Is it reliable? Is it available? Is it secure? Is it compliant?
Bryan Finster:I can validate all those things before production and post production. And so if it passes my fitness functions for those, it's good. That's it. Why do I care about the details of which way you decided to run a loop, whether it's a four loop or while or what? I don't care.
Bryan Finster:I just don't care. What I care about is the business results. And so I write fitness functions for the business results, and then I write agents for making sure the code is efficient to change.
Jonathan Hall:So I've for a long time believed that the highest aspiration should be that code is efficient to change. Yeah, that's sort of what clean code is supposed to be about. Now, maybe it's a different agent doing that change. To what extent do you care about the structure of code? You know, should this be split into multiple modules or classes or is the dependency inversion correct here?
Jonathan Hall:That sort of stuff, the more structural stuff. Do you care or you'd let the agents worry about that?
Bryan Finster:I know, care. I care about the structure. That's why I have exact same architecture and domain driven design agents. I care about the structure because to optimize for tokens means that it it encourages modularity. Smaller files, smaller functions, good documentation, well structured for what the intent of this area is with no domain bleed.
Bryan Finster:You know, these things all matter even more if you're trying to be economic with your tokens. So I care, but I I can't do it myself. There's... You know, I'm moving... Someone someone who was talking about how people like me, we never give, like, hard numbers about improvement.
Bryan Finster:I'm like, well, I can't. I don't have time. But because to do that, I'd have to do do it without the tool and then do it with the tool. But in the way... And the time it will take me to do without the tool, I can do three things in parallel and another three things in parallel.
Bryan Finster:That's my day job. Yeah. I I don't have time to give you the... Well, prove it to me. Know?
Bryan Finster:No. You either do or do not, but I have work to do. And I've just... The thought, this is not my hobby. Yeah.
Bryan Finster:Right
Jonathan Hall:before I started... In fact, I was I was at a meetup where I was convinced to start trying Claude code more seriously. Had been dabbling with it before. I'd gone to a meetup that was all about AI and some folks were telling us that, know, kind of what you just said, like, I'm so much faster. Like, are you really though?
Jonathan Hall:Because like, isn't the bottleneck human processes not writing code? And to to an extent, I still stand by that. It's it's true, but with more nuance now than I realized at the time. So I started taking some of the things I learned at that meeting and started applying it and and I am now able to work on three or four things in parallel that I wasn't able to before. So that's the proof to me is that like I could I could do the same potentially quality of work, but now I can do that same quality of work on three things at once that I couldn't have before.
Jonathan Hall:And so that, to me, that's the sort of the proof.
Bryan Finster:Well, the, know, like the agentic CD area, I talk about the stages developers... And they don't have to go through all these stages, but these are the different stages I see developers at. Apparently, people have seen the same thing because there's other people who've independently come up with very similar structures. And the constraints they'll hit, you know, it starts with them using a chat window, you know, and then I start hitting stuff, and then I start using, you know, autocomplete and I hit stuff. It goes all the way to I have an adjunctive scrubbing team.
Bryan Finster:But there's there's other stuff that people just don't think about. So we're working right now again, you know, it's a it's a older company, it's a 50 year old company. They've got a lot of legacy code from, you know, stuff that's been around for a while or acquisitions or whatever. And we've got a lot of focus on modernizing things and doing the DevOps stuff for real. That means pipelines that are solid, tests that are solid, you know, everything.
Bryan Finster:Right? I've been there for a few months. I don't I don't have time. Anytime you're gonna try to do a change like that, you've got limited runway. You're limited by mean time to leadership change.
Bryan Finster:So you've got to run fast. And before I would say, okay, if we're going run fast, we need to hire a whole bunch of people who know how to do this DevOps thing and go and fix all this stuff. Can't. So yesterday, during a meeting, told the the the primary thing... The primary system I'm focused on, I told Claude to go out there and, you know, one, use my DDD agent, map...
Bryan Finster:Go go and pull the 207 repositories that make up this area of the company and go do a create a domain context diagram and call out any issues that's used for domain bleed or anti patterns.
Jonathan Hall:Mhmm.
Bryan Finster:Twenty minutes later, I had a markdown document that I could upload to Confluence that had mermaid diagrams that mapped out everything. It showed databases are being used inappropriately from a service boundary point of view. It called out high, medium, and low... Critical high, medium, and low risk things from a demand perspective. And then I just iterated on that.
Bryan Finster:It's like, hey, I need some more information. I need this stuff too, and I need this and this and this. And and, you know, after a couple hours, I had a really well formed, here's what we need to work on in this sequence. You know, like, these things over here don't matter because they haven't changed in four years, but these things here are super critical because they're changing all the time. So we can start prioritizing, finding out what to fix, just it needs fixing.
Bryan Finster:It's not enough to throw tests at something if you have the wrong architecture and stuff like It's it's not auto complete. I mean, yes, I'm gonna validate that with the domain experts in the area. I'm already sending it them, say, hey, tell me where this is wrong. Mhmm. But from the work that we've done so far, it looks very, very accurate.
Bryan Finster:And you can use this for everything throughout the SCLC, starting with, I've got requirements for the product owner. Let's go run cloud against it and then open defects against requirements before it waste our time trying to implement them. Like, they're vague. If the robot can't figure it out, why should I spend my time doing it? I'm far more expensive than robots.
Bryan Finster:I'll just open defects against requirements.
Jonathan Hall:I've been working on a project for a client, basically rewriting a subset of a project. It's an open source project. They want just a subset of what it does. It's too big for what they need. Rewriting that as a drop in replacement.
Jonathan Hall:And Claude has been invaluable for this because it can go into the running Docker containers and debug the Python code that's running and figure out which features, which code paths are actually used, which ones aren't, which ones do we need to implement, and which bugs do we need to be compatible with and so on and so forth. It would take me months to do that. It does it in twenty minutes.
Bryan Finster:Yeah. Well, no. And and, know, two things on that. So number one, I'm not a front end developer. Although the last role I held at Defense Unicorns was leading a web development team as manager and tech lead, which meant I had to do front end development and, like, dev overruns and CSS with beta.
Bryan Finster:When I figured out that Claude could look, like, not just look at the code, but look at the results Mhmm. If I just told it to. And I just... And it it it would just fix stuff. I'm like, you know, the pages are...
Bryan Finster:Go go use the Anthropic front end design plugin, you know, make it better.
Jonathan Hall:Mhmm.
Bryan Finster:Literally make it better. Or tell it, this is broken, fix it. Yeah. And we'll go look at the screen and go, oh, yeah. I see that.
Bryan Finster:You know? It wouldn't be... Because stuff you can't see in the code unless you render the screen or render the screen and see it. Or just... I've done stuff like someone asked for a new feature on something on the platform in...
Bryan Finster:Internally, and I cut and pasted their their request from Teams as an image into into cloud and said, do this.
Jonathan Hall:I've done this
Bryan Finster:something else while I did it. You know? Yeah. Yeah. But the other thing is, you know, talking about the rewrite, I had something that had been written in React.
Bryan Finster:I've been working on this, like, value stream mapping tool for people who don't know how to animation that. Mhmm. And I wrote it in React. Well, I had cloud generated in React because that's just its default. All the tools, like default to React.
Bryan Finster:This everyone knows best, but I hate React. So I said, hey. Rewrite this in Svelte. Because I had tests. You know, it all worked in me.
Bryan Finster:So rewrite it in Svelte and make sure that it's, you know, pixel perfect as far as the results on both sides. So I created a branch, rewrote it in Svelte, compared the branches, images next to each other, made them match, and then so it was done. It took twenty minutes. It was not a small... It wasn't a huge application.
Bryan Finster:It wasn't trivial.
Jonathan Hall:Yeah, I've had similar experience. I had a front end app written in JavaScript. I had it rewrite it in TypeScript for me. I've had it changed from one templating engine to another in half an hour, you know, similar sorts of things. So you've talked you talked about all these agents you're doing and skills you've used and so on.
Jonathan Hall:How does somebody get started with that? Because I've done a custom agent. I've done a few skills, but like, how do you get started in that journey? What do I do? What does somebody else do?
Bryan Finster:Use the tool to use the tool. I don't know how to either. And it keeps changing. Well, and it keeps changing what a good agent looks like. So even if you mastered how to build a skill or an agent, that keeps changing as they keep keep optimizing things.
Bryan Finster:You know, I need a DDD agent. Cool. Claude, I need a DDD agent. It it creates more and more skill. Right?
Bryan Finster:And the other thing is like, hey, Claude, when do I use an agent versus a skill versus an agent wrapped up in Python versus a rule versus a book? You know? And it tells me, and then I'm educated. Now I know what. And then I say, okay, I need this to do this and go build it.
Bryan Finster:And that's a good first pass, but it's code. You never trust code. I need a review of it to find out how to make it better. The thing I heard about today on YouTube was that Claude recent... I mean, Anthropic released a plugin called Skill Creator.
Bryan Finster:And his job is to make it easier for people to do that. So you just tell them, hey, I need a skill to do this. And Skill Creator goes and does it. Or if you have existing skills, you will upgrade them to whatever the latest and greatest thing for how to use skills efficiently and effectively. And so now that's my tool for auto upgrade, is, like, just run that against all my skills every once in while to make sure that they meet the latest and greatest.
Bryan Finster:Anytime I don't know do something... Yeah. Anytime I don't know how to do something, I just ask Claudia to do it.
Jonathan Hall:So so I've tried that and I've had mixed results. I I know things are improving. So I... My my first, attempt at that was I opened up a new Claude session. I said, hey, go look.
Jonathan Hall:Here's my dot claude directory. Look at all the conversations I've had there in the last two weeks and find patterns where I have had to correct you frequently. And let's let's attack these. And the first thing I said was, oh, let's let's create a cloud dot m d that says always write tests first and always, you know, all this and this and this and this. That was almost ineffective because as you were saying earlier, Cloud MD goes out of context very quickly and it and it forgets.
Jonathan Hall:Yeah. It remembers as soon as I say you forgot to write a test. Oh, yes, I was supposed to write a test first. Let me go, you know, whatever. A few iterations after that, it it got to a skill and it and it wrote a skill that was under crap.
Jonathan Hall:I had to tell it, search the Internet for what other people are doing that works well with Claude, and then it was able to come up with a reasonable answer. Is that similar to your experience?
Bryan Finster:Oh, sure. No. Like I said, never trust anything, but it's a good place to start. Right?
Jonathan Hall:Yeah.
Bryan Finster:And I'll I'll even ask Gemini. I'm... I I like cross track models. Hey, Gemini. How do I do this with Claude?
Bryan Finster:You know? No. I don't trust anything. I'm always refactoring stuff all the time to make stuff better, But, it's so much better than doing it by hand. I would rather...
Bryan Finster:But so the... But the other thing is, you know, that a genetic dev team, it's got baked into it when I give a correction, update the thing that needs to be corrected so it understands what good looks like. So there's another thing that's out there that, if you look for, plot evals, there's an an anthropic blog post about evaluation functions to evaluate skills and agents against, to really give them deep examples of good and bad they can point to, that you can then go and make it your good and bad versus just generic good and bad. Mhmm. So what do I do?
Bryan Finster:Well, I need something that'll go... I need that... I wanted an evaluation skill so I could run that against everything. There's a... Claude, go read this blog post and do what it said.
Bryan Finster:And did. And it found a whole bunch of violations on everything. I said, okay, go fix it. And it fixed it according to what that skill set needed to be fixed. The other thing I've done is, you know, I've got all this stuff, you know, I've got all this stuff documented that's good ideas.
Bryan Finster:I've been on the CD. I said, hey, go read this and build a pipeline that does that. Know, pipeline of agents and tools, go do that. You know, and it did because it's got context or somebody will, you know, I'll see a LinkedIn post or a blog post from someone and I'll say, hey, compare what they're doing to what we're doing, find gap and suggest improvements. And so stealing is good ideas from people, which that's what we do as developers.
Bryan Finster:We're always stealing good ideas, but the effort required to steal good ideas is that you need to understand what you're trying to accomplish. You need to understand what's a good idea versus a bad idea. And that's the skill you need. You don't have to go and do it yourself. You're to say, Okay, know what good and bad looks like.
Bryan Finster:I know what I'm trying to accomplish. Good do the thing. I'll evaluate the results. Cool. We're good.
Bryan Finster:Let's go to the next thing.
Jonathan Hall:What do your pipelines look like these days? The thing that everyone thinks it means pipeline, right? When people say CD, what they what they think.
Bryan Finster:Well, and we'll start with just the pre commit, right? Because we'll start with just the automation that you normally do, the pre commit. I actually went through an exercise of, know, this is where I was, again, Hey, Claude, let's brainstorm. What can we evaluate now? Or what can't we validate with standard tools?
Bryan Finster:And we had a conversation about that. I said, Okay, now if I were to use agents, which one of these things could I evaluate with? Can I make better? Can I do with agents? We can't do with tools.
Bryan Finster:And all of them. Cool. Cool. All right. So what we need to do is we need to go and add a page to minimum CD that lists all of the problems that you typically see in software, lists where those problems systematically occur, meaning it could be the requirement, the process you have for refining requirements is wrong, right?
Bryan Finster:Like all that. What tools exist today to detect those problems and how early in the pipeline we can detect with those tools? And then go and add where AI can help either improve, augment that tool, do something a tool can't do, or AI doesn't help in the spot at all. Don't don't try to apply AI here, only apply the tool. And it created a spreadsheet that did that.
Bryan Finster:I added it to cd.org. It's like, hey. Here's... If you're... Here's all of the, you know, the the things that you generally test for, all of the things, not just function, all of the things.
Bryan Finster:Here's the tool you wanna use. Here's how early you can find it. So... Or, you know, structure or, you know, architecture pipeline to find it here. And here's where you can add agents to the flow in your pipeline to detect these other things that you couldn't detect before.
Bryan Finster:Mhmm. And it's all documented on my own CD.
Jonathan Hall:I'm gonna go check that out. So you have inspired me to try harder to get Claude to help me make Claude better. Something I've done already, as I talked about, but I I kinda... I don't know. I try it and I let it sit and then I stopped tinkering because it's good enough.
Jonathan Hall:But I think I need to be more intentional about continuing to improve.
Bryan Finster:Well, problem I'm trying to solve is not how do I make it work better for me. My problem is how do I platform it so it works better for everybody else. Along with the training, I can't... I'm terrified of the average developer just diving in and generating code with Claude because it's a skill. Even if you have all the agents and skills, it's a skill to tell it to do the right thing.
Bryan Finster:I talked to Paul Hammond about this, about trying to use classic TDD with agents and how I didn't think that was effective at all. You shouldn't... You should avoid it because the context window is so small that it lacks context, and you get... Wind up with spaghetti code that requires more remediation. And I said what we should be doing is acceptance test driven development so that it has enough context about the goals we're trying to accomplish with this specific test scenario instead of this specific piece of code.
Bryan Finster:And that the results I'm seeing are it's much better organized than first pass, you know, much higher first pass accuracy rate when it understands the goals you're trying to accomplish versus the function you're trying to write. Yeah. Or he argued with me, but I I I explained to him my reasoning behind it. I I believe I may have changed his mind.
Jonathan Hall:Yeah. I found that trying to... On the one hand, I wish I had this experience fifteen years ago because it's sharpened my understanding of so many principles. You know, test, you know, I'll just use one example. What is a behavior when you're trying to write a test?
Jonathan Hall:Yeah. Because Clog gets it wrong all the time and will test implementation details that I don't care about and I have to correct it. So, you know, being forced to make that correction over and over again.
Bryan Finster:See, have a test review agent that specifically looks for that and then forces correction.
Jonathan Hall:Yeah. That that that's the next step. But in the meantime, I've been learning like, you know, it's almost a trope if you're in the TDD crowd, that's a trope that test behaviors, not implementations to the point that I think a lot of people don't even know what it means anymore. This has forced me to realize that I can now spot an implementation versus a behavior a mile away that before I'd have to like squint and like, let me think about what that is. I've learned to recognize those patterns much better.
Jonathan Hall:I wish I'd known that fifteen years ago.
Bryan Finster:You know, I think the advantage I have on that is that when we were starting continuous delivery, I hadn't had either... I mean, the languages I'd worked in primarily didn't even have testing frameworks.
Jonathan Hall:Mhmm.
Bryan Finster:Testing was a 100% manual. And to solve the test automation problem, had... Found behavior driven development, and it solved all the problems that we were having with communication and learning how to test, you know, because we had to teach the entire team how to test, including me. And so I've always come at it from the out... From the outside in.
Bryan Finster:I've always done black box testing at the behavior level. I very rarely have ever gone down to the classic TDD level on anything except for really complicated stuff. I actually got interviewed for a job by the the the CEO of Blanken of the company. Kent Beck was working there. I really wanted to work there because Kent was working there.
Bryan Finster:He's an adviser for him now, but he was he was working there. And the the CEO who started... I'll think of it later. Anyway, he's a hardcore TDD guy, he gave me a TDD interview, but I don't work at that level. I work at the behavioral level or, like, the scenario level.
Bryan Finster:And so I I was, like, completely flubbed it. Yeah. Fair enough. But, because of because of... I've always addressed it from outside in black box testing.
Bryan Finster:That's that's just my habit. Yeah. And I knew it was generating, imperative tests. And so I said, hey, we're gonna use a behavior driven development expert to go and validate your testing behavior and that implementation and then test... Fix all the tests and just auto repair.
Bryan Finster:I don't have to worry about it anymore.
Jonathan Hall:So how do you have your different agents? When when does your BDD agent come into the picture? Is it all automated? Do you say, team, go build this thing for me?
Bryan Finster:I mean, yeah. So it's evolved over time. I mean, I'll do... It depends on what I'm doing. I can trigger those agents independently, but the goal I have and the the thing I'm trying to rely...
Bryan Finster:To build this reliable, and I'm... I've still got real work to test it on to see how reliable it is. Right? So this is still work in progress.
Jonathan Hall:Sure.
Bryan Finster:Is that I just tell the orchestrator, I'm acting exactly like a product owner. Here's the goal. Here's how we know it's good. Go do it. And it goes and does all the things.
Jonathan Hall:Is there an interview there between you and Claude first? Like, what are the acceptance criteria? What... Should this be on the left or the right? Should...
Jonathan Hall:You know, what are the performance requirements? Stuff like that. Does that happen first?
Bryan Finster:It it happens if I know enough to tell it because it's not smart enough to see what are the performance criteria. Right? But this is stuff that when you're talking about what the goals are, you define all this. But there's a product I'm looking really hard at to to help and to bring into my company that's open source with a coming up freemium model called SpecKitty. And SpecKitty enforces...
Bryan Finster:This this drives me crazy, by the way, because behavior driven development's now been just rebranded as specification driven development, and it's just BDD. Yeah. Drives that crazy. Oh, wow. You discovered BDD.
Bryan Finster:Cool. But spec kitty enforces a spectrum of development workflow. Yeah. And and keeps track of tasks across teams. It does all kinds of cool stuff.
Bryan Finster:And I think it's a really... It's enterprise. You know, the goal is enterprise tool to make it so that you just can't generate code without having specifications. And it runs you through an interview process to make sure that you can't get to the next level of interview until it's happy with this level of interview. Right?
Bryan Finster:But you still... This is the thing is you can't replace software developers because we still need to understand the entire iceberg, not just the tippy top on functionality. Really love Clay Schafer's answer to I wrote a post on LinkedIn about WTF is a nonfunctional requirement, right? That's not a thing. There's no such thing as a nonfunctional requirement.
Bryan Finster:And Andrew said, a nonfunctional requirement is the thing that if it's not there, makes your application non functional. Ah. Yes. Software engineers understand the engineering trade offs with performance, reliability, how to make sure that's there, how to make sure things are secure. So, people just hack stuff where people are just vibe coding, have no development experience.
Bryan Finster:They don't know the things they don't know. And so, things go wrong quickly. But forcing people, well, not not forcing really, but showing people the... Giving a tool to people that develops a habit of we're developing with intent and not with code. I think it's a really important thing if you're trying to roll it out in an enterprise and you've got a lot of people to train really fast, that they have to modify their work habits.
Bryan Finster:When we learned continuous delivery, it was a mindset and work habit shift that was pretty dramatic. People underestimate that. I think it's pipelines, it's not. With agents, it's more dramatic. If you're not if you're not already in that mindset of, I know what it is.
Bryan Finster:I I need to know what it is I'm trying to do before I start doing it.
Jonathan Hall:What... You see any tension between spec driven development as they're now calling it and agility, to use an overloaded word. But, you know, the idea that we don't know what we don't know yet, so how can we design for it?
Bryan Finster:Well, you know what the next thing you... Well, so this... Yeah, I've had this... I've ranted about this a few times where people are like, Oh, it's just waterfall. Only in that waterfall says you should know what you're doing before you're supposed to do it.
Bryan Finster:But there's nothing in Agile that says you're not supposed to know what it is you're trying to accomplish before you do it. The difference between the two is waterfall, you have a single pass through a flow and you've got a lot of upfront stuff where you assume you know everything. With agile, we don't assume we know everything, but we do have a hypothesis about the next feature and the value it should deliver. Well, we can't get there unless knowing how it's supposed to work. So we have the hypothesis, we have the goal we're trying to achieve, that's the hypothesis.
Bryan Finster:We define the scenario, behaviors that hopefully will meet that hypothesis, and we define how we're going to measure that it did or did not achieve that hypothesis. And that's spectrum of development, that's behavior driven development. That's what that is, right? And the difference is that we're focusing on invalidating the hypothesis or celebrating when it's not invalidated. And we're focused on how fast we find out.
Bryan Finster:And so it's about batch size and it's about frequency and it always has been. That's what agile is supposed to be, small batches of fast feedback, but operating with clarity. Mhmm. You can't just code shit, you know, and then just hope that it meets a thing, you know? It's like we have intent.
Bryan Finster:And so with... Agents force you to to expose the facts you don't know and you don't have intent.
Jonathan Hall:Yeah. It sounds a lot like the the same sort of... Tautology isn't the right word, but the the... Or the misconception that the people have that, like, TDD only makes sense, when you know what you're writing first. But, like, if you don't know what you're writing first, why are you
Bryan Finster:writing anything? Yeah. Yeah. The only other reason to code anything is because I want to learn this API. I have an idea.
Bryan Finster:I'm not sure how good it is. I'm not sure where this is going. And you're doing POC stuff or learning stuff and all that code should be thrown away anyway.
Jonathan Hall:Yeah, exactly.
Bryan Finster:But if you don't know if it... You know, stop. This drives me crazy. You're a professional software engineer getting paid to deliver something. Why are you spending all your time playing on the keyboard before you know what you're supposed to deliver?
Jonathan Hall:Yeah. Yeah.
Bryan Finster:I mean, you're burning money. Yeah. When I worked at Walmart, my bonus was based off of total company performance. When I saw people wasting money, it was like, You're burning my bonus. Yeah.
Bryan Finster:Yeah. They're crazy.
Jonathan Hall:Awesome. Well, I've I've enjoyed chatting, Brian, as always. You're fun to talk to. We see eye to eye I always learn something from you. So thank you for taking the time to do this.
Jonathan Hall:Any closing thoughts before we wrap up?
Bryan Finster:Yeah. One big one. So I I know you're joining late for reasons that I would normally consider to be sane. But right now, don't think that's a... That's it's not a sane position right now.
Bryan Finster:If you're... Yeah. Everybody who's waiting is falling further and further behind everybody who's trying to figure it out, and not learning the lessons, that they need to learn that that are important from a principles and practices point of view, no matter what model we're using. But the thing is, is that this is super cool. We're all learning at the same time.
Bryan Finster:And so there's not like this huge gap where you have these people that are way out in front who are like the leaders in the industry that you need to look. No. We're all learning at this. It's like, you know, I'm watching Patrick Dubois learn in the open. I'm watching all...
Bryan Finster:Just all these people who are... Who I've looked up to for you. David... Dave Farley learning in the open. You know, it's like nobody knows, you know, and it's so cool.
Bryan Finster:Yeah. And so anything I say, I'm not an expert. Nobody's an expert. There's no experts. Even
Jonathan Hall:the people
Bryan Finster:who work at Anthropic and Gemini are not experts.
Jonathan Hall:What you're describing kinda reminds me, was... I'm a little bit too young to really have been involved in this. But the early days of like home computing and the like get together and hack on a Commodore 64 or TRS 80, It kind of feels like that.
Bryan Finster:Yeah. And I know it's a lot like that. Yeah. You know, in the early days of the internet trying to figure out this web stuff, you know, it's it's either the the hobby stuff. Yeah.
Bryan Finster:It's a lot like that.
Jonathan Hall:Cool.
Bryan Finster:But it's... If you're applying it to work without without the right mindset, it's really dangerous. But all powerful tools are dangerous.
Jonathan Hall:That's true. Great. Well, thank you so much, Brian. I hope we have a chance to chat again before too long. It's always fun to do that.
Bryan Finster:So Oh, for sure. Anytime, man. I I love hanging out with you.
Jonathan Hall:Awesome. Thanks so much. Talk to you later. Sure.
Bryan Finster:Later.