Mike Gehard (00:00):
Generating code faster does not fix that process. You either A, build the wrong thing faster or B, you build something that isn't the thing you wanted built faster and you have no feedback loops. Yes, we want to generate code faster, but is the rest of your process healthy enough to support a human not writing code?

Nicky Pike (00:22):
This is [Dev]olution, bringing development back to speed, back to focus, back to freedom. I'm Nicky Pike. Okay, so here's what everyone's talking about right now. The software dog factory, code that ships itself. Then you got agents that write it and they ship and there's no human ever opening up a pull request. You just show up at the end and sign off on the outcome. Sounds like a dream, right? Well, some teams have tried it. One team ran a lights out version for six months and they came back to a code base that one of them called a complete pile of poo. It was a total rebuild. Turns out taking the human out of the loop doesn't mean you get to skip everything else. No test or team norms. That just means you're automating a mess faster than what you used to do when you wrote it by hand.

(01:09):
So if you're building anything with AI right now, you already know how to close this hits home. That's exactly why we do this show every week. So if you haven't yet, hit subscribe and stick around. Here's a height requirement to ride this AI journey and most teams have no idea what it is. And today we plan on finding out. Joining me today is Mike Gehard who runs R&D at Rise8. He's building AI augmented software delivery for veterans and defense organizations. And this is the type of engineering where ship it and see just isn't an option. He spent two decades in software from living social to VMware to teaching Pivotal's agile playbook. And when he's not in a code base, he's out on a mountain bike trail in Chattanooga volunteering as a trail steward. The industry right now is racing towards full autonomy, where you see agents that write and ship without anyone reviewing the code.

(01:59):
Mike has ran that experiment inside an organization where getting it wrong has real stakes and he's got the story of what happened when it broke. Mike, welcome to The [Dev]olution.

Mike Gehard (02:09):
Great to be here, Nicky.

Nicky Pike (02:11):
Well, so let's go ahead with your opening, right? So you've gone from a startup like Living Social, you've been in VMware, you were teaching Pivotal's agile approach, and now you're doing R&D at Rise8. What changes about how you write software when the whole concept of move fast and break things stops being an option for you?

Mike Gehard (02:29):
It's all about risk mitigation. I mean, even startups don't like to move fast and break things. There is still a piece of your code that you need to know works. So Chad Fowler, who is my boss at Living Social, is working on a book that talks about regenerative software. So what does the world look like where you never change a line of code, you just regenerate the software like we do with servers now. We have immutable infrastructure and he has this idea, I don't know where he got it from, he talks about pace layers. So software in your stack changes at different rates. So a UI is a place where you can iterate really quickly because a break really doesn't destroy anything, but you might have a layer of your software where the business rules for cash transactions happen and you want that to change a little more slowly.

(03:19):
There's more risk if that stuff breaks. So if you think about when you're developing software with AI, you have to realize what the risk is and what the blast radius is for a bad change. And if the risk and the blast radius is really small, just sling, let the agent sling as fast as they can. But if it's a higher risk change or you're going to lose millions of dollars if a bug goes out into production, wasn't there a financial company that went bankrupt because their software lost them a bunch of money? You want to take your time there. So you might not want to ship code to production that an agent wrote overnight without a human looking at it. So I think you just have to think about if a problem happens, what is the ramifications and am I willing to accept those ramifications?

(04:04):
And that's how you can start to make decisions on whether or not you let an agent ship code to production. I have to have trust that the thing I'm about ready to ship isn't broken. So you brought up CD, which I love. I mean, we talk about, I joke about must be this tall to ride the ride. I feel like if you're not doing CD at this point and have the trust that you can do that in your processes, the rest of this stuff doesn't make any difference. I really love your refinement. It's not that you can't do it. It's do you trust that you can do it? Do you have the confidence? Are you willing to bet your company that the change you're ready to ship is either A, not going to break or it's okay if it breaks? And I'm willing to accept that risk because I can iterate quickly.

(04:49):
So I think that is one of the prerequisites that I think we need to talk about is how close to continuous delivery, like an actual continuous delivery. I ship to production on the regular six times a day, are we before we start talking about how fast I can generate code?

Nicky Pike (05:07):
Well, and so okay, so that moves on. And this is a great segue because right now the big thing is a lot of people are talking about software factories. How do we turn our development processes into something that's repeatable regardless of the code that we put in it? And there's this new concept of dark factories, especially with AI coming in of, do we really need humans in the loop? There's a change to humans on the loop and all these different little nuanced changes of nomenclature that we're talking about. But you had mentioned a team that ran a dark factory approach for like six months and they ended up with a code base that they had to rebuild. Walk us through exactly what it looked like when they went through that and what did it look like when it broke for them?

Mike Gehard (05:44):
So this wasn't a customer of ours. This was something I read on the internet, but it's from Dex Horthy who is running a company called HumanLayer now. And he was the guy that kind of came up with a context engineering idea and he's been studying this a lot. One of the places I look to inspiration for is the Toyota production system. One, because I drive a 24-year-old Toyota Tacoma that is still yet to break down. But they've been talking about this idea of automation in cars for decades and what you need to do is you need feedback loops. So the whole Toyota production system, they have this idea of an Andon cord. So anyone on the line in a Toyota factory can pull the Andon cord, the whole line stops, they figure out what the problem was, what was the off-spec part, and instead of reworking the part, they go back and fix the factory.

(06:37):
And so they're not making bad parts or they're not doing a bunch of rework. You can apply those same things to the software we've been building for 20 or 30 years. So it started with a description of the problem I'm solving. I then wrote acceptance criteria. I then hand coded some stuff. Maybe I'm doing TDD, so maybe I'm doing continuous refinement there. I then put it into CI. I let a CI machine run it. So I did the part that I just built, matched the specification that I was supposed to build the to. We then do acceptance, which is just a human sitting down and saying, yep, that thing I asked you to build actually did the thing that I though it was going to build. And then the last step is, did the customer use it and did it have the outcome that you expected for the customer?

(07:25):
That's a software factory. We've been running, I mean that's an agile process software factory. All we're doing now is the thing we've done is now code generation is not the bottleneck anymore.

(07:37):
So I'm not handcrafting cars or furniture anymore, but I got a machine building it. So great, I can do that. But they're kind of not great at building. I mean, they can build code really fast, but they do not write the code you want them to write very quickly. So I think everybody wants to generate code faster, but it goes back to that whole idea said about CD, that if your process sucks right now, like your acceptance criteria sucks or you're doing acceptance by hand, heaven forbid, like you're not writing end-to-end automated playwright tests for your website or you don't have access to your customer, so you can't really figure out what they want, you're making a bunch of guesses about what they want. Generating code faster does not fix that process. You either A, build the wrong thing faster or B, you build something that isn't the thing you wanted built faster and you have no feedback loops.

(08:36):
So I think this is where we have to start to talk about, yes, we want to generate code faster, but is the rest of your process healthy enough to support a human not writing code, I think is the question you have to ask yourself.

Nicky Pike (08:52):
If I apply that towards my software factory, I'm worried that if I give my software engineers the ability to do that, I've got to worry about things like time to market, being the first one to get a feature on. I've got budgets to worry. So I don't know that that really applies to us. Do you think that's a symptom of the problem or do you think that they've got a valid concern there?

Mike Gehard (09:09):
But isn't that what agile is? When we were doing agile with humans writing code, humans were still stopping the process along the line. So iteration planning meeting, what is an iteration planning meeting? It's simply reworking of a story because the engineers who are getting ready to build it didn't understand what the product owner put in the backlog because they weren't in the meetings or they didn't have the context they needed. But I will tell you, you have to take a cold, hard look and ask yourself in the mirror, is that the thing that is going to make or break my business? Because if you're not talking to your customers and you're building a bunch of stuff and your customers are not using the stuff you built right now, you don't have metrics that say, "Hey, when I put out a new feature, here's the adoption rate and here's how many humans use it and here's how quickly they're using it and here's the problems they're having." Building that thing faster is not going to make or break your business.

(10:04):
You have another problem downstream, your value stream map is off, you have another bottleneck. We don't want to generate code faster. We want to generate code more cheaply, which is what we've been doing. We've wanted that for decades. Why did we outsource stuff overseas is because we wanted to generate code more quickly or actually more cheaply. So I think that's the question you have to ask. Is it cheaper code that's causing your business to fail or is it other things? You've

Nicky Pike (10:31):
Been talking about agile quite a bit and about two years ago you started writing publicly about AI-assisted agile software development. What was the moment that made you start, I need to start documenting this instead of just doing it quietly?

Mike Gehard (10:44):
I mean, I've always been a doer. I mean, I do a lot of talking. Anybody that knows me knows I like to talk, but I also like to do. So I think, I mean this was December of 2024, so probably an eternity ago in what we're dealing with right now. I just needed something to do on the weekends. I was unemployed, so I had left VMware, had taken a sabbatical to do some nonprofit work in town. You talked about mountain biking and was getting back into software development and was kind of like, this AI thing looks interesting. I mean, I had written code in 18 months, so why should I start writing code now? And this was the Aider days. So this was very early when it was more of a better auto complete. And I just started running experiments like, okay, cool. Well, the first experiment was, I don't want these things running on my laptop.

(11:36):
That was the first line I drew in the sand is like, no way you're getting access to my laptop. So what did I do? I was like, oh, dev containers. Let me see if I can stand one of these things up in a dev container. And I published an article on that and then just started writing. I was like, oh, here's a little experiment. This might be interesting. Mostly to document my journey. So when I went back and was like, oh, how did I do that thing? Oh yeah, there's that article that I wrote. And I've been doing this a lot of my career. So yeah, just started doing, started writing, was like, I should share this. And then I went silent for a while, but most of it was just to get it out and get people talking about it.

Nicky Pike (12:14):
Well, and so I find that interesting. Didn't write code for 18 months, jumped in, you started using AI to write code. Love the fact that you brought up, I'm not giving my laptop, it's not getting access to my laptop as part of this. I think that's a very smart move, but you had written software before for a long time. What really excited you on the AI side? Where did you go, man, this is going to be different? Or did you have doubts when you first started jumping into it?

Mike Gehard (12:38):
No. So I have never been the best code typer under the sun. So this is one of the reasons I love pair programming. I mean, I pair programmed in real life with humans starting in 2010 and did it for a lot of my career. So I was never the human that knew the little JavaScript things in the back of their heads. I was always looking stuff up on the internet, but I was the person that was asking the product questions like, Hey, are we sure about this? This doesn't make any sense to me. If I were using this, what would I do here? So it was great because now I could be more of a product engineer and less of a like, oh yeah, what is that esoteric bash command that I can never remember right now or the syntax for something in code? That was attractive to me.

(13:24):
I didn't have to remember all that junk because it's just like there's a lot to remember. And then back then it was a better auto complete. So we didn't have agentic loops at that point. So it was just helping me knock the dust off the commands that used to roll off my fingers. But it was mostly that I could focus on the product and not on the computer trivia.

Nicky Pike (13:45):
That's been my tagline for years, Mike. I am not a developer. I'm able to use AI. I did it much the same way you did. Hey, this is a problem. Now I'm able to give people an idea of what I would like to see rather than just describing it. I think that's going to be a huge movement forward for a lot of companies. And I think that is one of the things where we're getting kind of that renewed interest in software factories, not only from the technical developers, but also the non-technical developers and how they would affect it. We've got a lot of outage out there right now with people within the company that are not developers, but can have real impact on the platform. And I think that's another place where the software factories are becoming sexy again, but you have made the statement that you don't think a lot of people are ever really going to get there, especially when we start talking about the dark factories.

(14:31):
What do you think is really stopping most engineering teams from getting anywhere close, even the ones that really think this is the way forward for them?

Mike Gehard (14:39):
They have crappy process right now.

(14:42):
I mean, so I talked about test driven development. At Pivotal, we paired and we did test driven development. So TDD first and foremost is a design exercise. And hopefully if Kent ever listens to this, he can correct me on my desk description, but my understanding is I write the test first because I'm now defining what the consumer of that API has to do. Whether or not you start outside in or inside out, you're writing a test first and you're designing your software. The upside of that is you now have a regression suite that tells you with pretty high certainty, because I write a red test, I only write enough tech code to make it green, and then I refactor, I now have a regression suite that tells me when something is broken. If you are not doing that and you do not have a measure as to whether or not your software works or not, AI is going to screw you royally because it will build stuff that will sometimes work, sometimes not work, and it will be very confident that it works.

(15:43):
So I think that's what's going to stop a lot of people. It's not going to stop them from using AI. What it's going to do is bury them and slop. So I think testing is one of them. I actually have gone on the record recently saying I believe that inside out test driven development is dead in the age of AI, but long live outside in test driven development. So I want to start from the outside. Does my software work? That is in my belief, the primary goal of any engineer is to say, does my software work and does it continue to work when I make changes? Full stop. So if you have not adopted, if you're doing manual testing, you can't tell if it works until six months later. So I think that is one process that you got to take a cold, hard look at.

(16:31):
And then if you look at TDD, it was a design exercise. So the second property of software it gave me was, can I make changes quickly to the software because I've well architected it? I've got a pattern in there where if I want to change my databases, I don't have to rewrite the software. I just have to write a different implementation of this interface. So I think that becomes less important now because if AI can change software quickly and Chad gets his wish of regenerative software, I don't care how fast I can change it because I can change it instantaneously, but I still have to make sure it works. If your company cannot tell you with certainty, in almost real time does the software work, you got to get there first because the rest of this is not even worth your time at that point.

Nicky Pike (17:22):
Yep. And we had Ted, the TDD guy, he did an episode with us. There's a lot of things about TDD, of course, coming from Pivotal, that was one of the things that we did. I wonder how this is going to change with AI because everybody's talking about AI DLC, the differences between the regular software development and life cycle. Personally, I think that we're going to get a hybrid. I do think that there's still aspects. To your point, TDD was more of a design exercise. Hey, it keeps me from writing a bunch of bloated code. Here's the code, here's the function I need to do. Does it work? End of story. We all know that if you give AI the chance, it will write bloated code. It will go in and write a bunch of stuff you want. So I think TDD as a design exercise can help with that.

(18:02):
But I'm also a big believer, and we'll see where you go on this, that spec-driven design is also a need because the more specific that you can be with AI, I think the greater the validity of the output that you're going to get from it. So I wonder if we're going to start seeing some hybrid methods of TDD and spec-driven development and all these different concepts start blending in as it comes part of AIDLC. What are your thoughts on that?

Mike Gehard (18:26):
So I think what we need to do is go back and ask ourselves, what were we getting from the practice? So spec driven development. I have my thoughts on spec driven development. That's about batch size. So one of the things in Agile is we did really small batches because of context window limitations between our ears. So I think we're going to have to start to ask ourself. I think all of the agile practices still fit, but we have to understand why we did them because the practice might look different, but the reasons we did it still exist. So spect driven development, I want to know what I'm building up front. What are my acceptance criteria? Do my acceptance criteria make sense? Do I understand the outcome that I'm hoping to have with this software? So I think that still becomes important. Spect driven development. I gave a talk in March at our annual conference with my friend Jason Frazier about story formats I believe are still important in the age of AI.

(19:25):
As a human, I want this thing to exist so I get this benefit. That is an outcome definition mechanism that I still think is very needed in software in the age of AI. Given when then, it's still acceptance criteria given this pre-condition, given the state of the world, when I do this thing, this is the new state of the world. I think very important for AI.

Nicky Pike (19:48):
Yep. Well, and you mentioned it before, you got this line, you must be this tall to ride. I love that line. I also love the fact that in AI, I think that line, which used to be very clear, right? If you didn't have testing, if you didn't have lengthy, if you didn't have real normals that you could hand your code based, then you had problems. Well, again, in the age of AI, you must be this tall to ride. The height requirement's going to depend on what you're doing. And I think the only way to get around that, at least in my opinion, is we've got to go back to outcome-based. And you've mentioned that a couple times. We've got to be outcome-based. Quit trying to think in terms of all the old patterns we had. Quit trying to think in terms of frameworks. What's the outcome you're wanting to achieve?

(20:31):
Now, how do we go out and find out a way to make AI give us that outcome in the most repeatable, consistent way that we can?

Mike Gehard (20:38):
Yeah. This is where I struggle with a lot of the articles coming out of places like Stripe and Ramp and people that have built these factories and are successfully using them to deliver code. I think the Stripe article quotes 1300 PRs to production a week or something like that. Something insane, but they never tell us what those PRs look like. They never tell us what the outcome of those PRs is. How is it moving the needle for their business? They're telling me about outputs. I like outputs just as much as the next person. This is why I love TDD. It's like a dopamine hit. Look, the test is green. Yay. So I think, yeah, we got to go back to outcomes. What outcome are we trying to have? We got to go back to value stream mapping. Where is my value stream blocked? What is the actual bottleneck that I'm having in my system?

(21:27):
But we've been doing this forever. I mean, as engineers, whenever we had a bottleneck problem in production, we always measured it first and then we fixed it. We never guessed to where the bottleneck was. We went and measured like, oh, the database query, that database table's missing an index. Okay. I now know that because I've measured the query plan. And that's the hard part right now, back to your question earlier, we don't know what the outcomes are. We don't know what the tools are. We're kind of making it up as we go and we're trying to be scientific about it.

Nicky Pike (21:56):
I think the measuring portion of that is also going back to the fact that we're changing, the technology's changing faster than we can think up. When we start looking at some of these door metrics, I think are pretty standard, but when we start looking at how many PRs, that's a very human way of thinking because that measured our output, that measured what we were doing and how much work we were pushing through, but PRs, they don't really matter. At least to me, they don't matter anymore. Okay, you're pushing 1300 a week. Well, if 800 of those were rework from the 1300 you pushed last week, then you're not buying anything. You're just repeating, you're reinventing the wheel over and over again. But if you're getting 1300 through and only two of those were rework, then that's a true measure.

(22:37):
But how do you make that correlation? And I think there's a lot of people right now, we don't know what to measure because this is a very different attitude than what we've seen before. So we're putting out the things that we know. Okay. Well, we've got to start rethinking this. What's the answer? Hell, I don't know, Mike. I don't know what the answer is. This is why I talk to people like you to get opinions and see if we can't start pushing towards a common goal because I do think that the metrics that we have now are not a good measure of what we're getting out of AI.

Mike Gehard (23:04):
Agreed. I mean, token spend, horrible metrics. So I think I'm going to put a call out to anybody working at a company with a dark factory. If you're listening to this podcast, I would love to start to see some analysis of those 1300 pull requests, how many of them were rework? How many of them were a text change? How many of them were a button change? Break them down for us. Help us begin to understand the value that you're seeing from your software factory so we're all not left guessing, well, is this worth it? Because tokens aren't getting any cheaper. I will tell you that. That's the other downside. If you're a leader, you want a dark software factory, even a light software factory, you better be prepared to write gigantic token checks to the foundation model providers because this stuff is not cheap.

Nicky Pike (23:53):
Absolutely. So I'm going to add to your call to action there and say not only tell us about the outcomes that you're getting from those PRs, but tell us about the shape of the PRs because the bottleneck's moving, right? Code generation is no longer an issue for us. Let's just be clear. We can talk about slop and we can talk about quality all you want, but code generation is no longer a problem. We can generate code.

Mike Gehard (24:13):
Humans have been generating slop for most of my career. Correct.

Nicky Pike (24:16):
Right? But I want to know, we've seen that bottleneck move, and this is going to lead me in my next question. I think almost universally, people are coming out and saying, "Yeah, okay, we're generating a lot of code, but I can't keep up with the PRs now." And so people are starting to implement steps like limiting the size of the PR, limiting what the AI is actually touching versus what a human's got to touch. And we're starting to see the human in the loop, human on the loop, human out of the loop, which is the dark factory concept that you're talking about. Give us some examples of how you guys are working through the PR backlog. Because one of the things I found, and this is a little bit of a step, but one of the things I found very interesting is that you're working on a project right now where problem definition is your bottleneck, not code generation.

(24:59):
We can generate the code all day long, but we don't know what to generate. We don't have a good problem definition. How many teams do you think are actually pouring their AI budget right now into the wrong bottleneck?

Mike Gehard (25:10):
I mean, if you're not measuring, it's hard to say. So the project I'm working on now with one of our customers is we're trying to figure out what to build. So code generation for us is not the bottleneck. It wasn't the bottleneck with humans. We have a team of six engineers. We had a meeting today. I was like, "How long would it have taken us to build this by hand?" And our product owner's like, "Two weeks, I'd be happy." I was like, "We could easily build this by hand, by humans at two weeks. We don't need to spend a bunch of money, but we don't know what to build." So I think it goes back to that idea of where is your bottleneck? If you are not measuring your company's bottleneck in your software development lifecycle, not Stripe software development lifecycle, not Ramp's software development lifecycle, your software development lifecycle, and you are not addressing that single bottleneck with AI.

(26:03):
I'm not saying don't use AI, you should identify the bottleneck and throw a bunch of AI at it, then you're just throwing money. You're spending money that you don't know how the upside is because you don't know if it's moving the needle. It's the same thing with a database query. I'm going to go add a bunch of indexes, but if it's not the index that is slowing my database down, I'm just wasting compute at this point. So I think that's the key is you have to know what you're trying to measure and then you have to... If it's story definition, how long does it take you to write a story? How much rework? Did you misspecify the story? So did you not collect all the acceptance criteria? I think there's all of that you have to be measuring and understanding. And I think the other downside of this whole AI thing we're seeing, especially on LinkedIn and my feeds and on X, most of it is software engineers building code for themselves.

(26:54):
So the feedback loop is this big. It's like, I have an idea. Oh look, there's a pull request. Did it meet what I wanted? The feedback loop is minuscule. If your feedback loop is not that fast, then who cares if you speed the middle of it up?

Nicky Pike (27:09):
So I'm going to tell you a story. So we did an event last Monday at Levi Stadium. We did it with Gene Kim, but we held this exact round table in San Francisco, and I read one of the tables and the table was the bottleneck has moved. I got a very interesting story out of about half the people that were at my table. One of their biggest complaints on bottleneck, Mike, get this, their biggest bottleneck was the customer. Okay, explain that to me because I hadn't heard that before. They're like, "We're writing code so fast now that the customer can't keep up with what we're outputting." And so what we end up doing is we put the code, they say, "Hey, we want X." We go give them X. Three weeks later, they come back and say, "That wasn't the X we wanted." So now we're redoing the code.

(27:53):
We're ahead of our customers at this point. That was something new to me, and I found that very interesting. And I think this goes back to what you've been talking about. When we're talking about AI, we've got to consider everything outside of AI and what we're doing. Those feedback loops, we have to reconsider how we're going to redo our feedback loops so that we don't run into these types of problems. Thoughts on that one?

Mike Gehard (28:16):
I mean, that's a perfect example. If you give a customer six features and they can't use all six features, where it takes them a week to use all six features, three of them are wrong because of course you under specified it because humans never know what they want built till they see it. I mean, it's a whole Henry Ford thing. If he'd have built what people wanted, they'd gotten a faster horse, not a car. But that's again, another feedback loop. So that's just the outermost slowest feedback loop of our system. So yeah, perfect example of outputs versus outcomes. They were outputting at record speed, but they could not recognize outcomes or impact any faster than the slowest bottleneck, which is getting a customer to sit down and tell them, "Yep, that's what I wanted. I'm willing to pay money for that." So I think this is where it will get interesting is for so long we focused on outputs and Rise, we actually just announced an outcome model, which has always been fixed firm bid.

(29:15):
We've always done fixed, everybody wants fixed price, but focus, that's the key measurement now is am I getting the outcome that I need and how fast can I validate the outcome and the impact that I need? Because if you can do that fast and the rest of it is fast, guess what? You just feed the transcript back into the factory and the factory will build it for you.

(29:38):
So you're lucky if that's your only bottleneck, but you're probably going to fix that one and then find something somewhere else. Yep.

Nicky Pike (29:44):
Well, and I think we're starting to see that in the industry. Some of the trends around citizen development, non-technical development within the enterprise, I think is a way of incorporating AI into your business without necessarily going and touching your software that you're shipping to customers. We're seeing new process Come out AI DLC and how we can start taking some of the risk out of binaries or dependencies that we're bringing in from open source and other things. So we're starting to see that. I think you brought up Kent Beck and Kent Beck has his model, the 3X model. It was explore, expand, and extract. Take his model and apply it to what we're seeing. Where should teams be letting AI run loose? And where do you think we should be keeping them on a short lease using some of those examples that I just gave you?

Mike Gehard (30:28):
Yeah. So I think Kent came up with this. I actually just pulled up the article. He wrote this in October 22nd of 2019, so almost seven years ago. And he talks about the explore phase, which is you just got to figure out what's happening. And I think he was applying this to companies. So if you're a startup, when I was at Living Social, we were a mid-size startup, but I was at a smaller startup called Wonder Capital. You were just trying to figure out product market fit. If you're trying to go really wide and figure out what the heck you're going to build, sling AI all day long. Who cares if it's vibe coded or slop because you're just trying to figure out what's going to land with your customer. So I think you're a little safer there. You get into the expand phase, which Kent says to overcome bottlenecks to scaling, ease the limitations of the next rate limiting resource.

(31:16):
So I think we're in the expand phase kind of now of AI where we've kind of gone through the explore phase. These LLMs are great, Opus 4.8, Fable. I think we've got the tools we need. We're starting to see some patterns emerge. Now we're trying to figure out how to scale it. So now you got to figure out where is the bottleneck and what is the next rate limiting resource? But if you're a company in this expand phase, AI can be really powerful for you, but not generating code.

(31:48):
It can go find your bottlenecks. Give it all of your transaction logs from your database and have it find out which ones of your database transactions are causing slowdowns. So again, if you're in this expand phase with your company, you've probably got a bunch of bottlenecks you're trying to hunt down. So go let AI do that. Just give it a bunch of logs. And then if you're in the extract phase, which Kent calls a sustained growth, continually increase profitability while you finish growing, which is big Fortune 500 companies, you can't screw up. You can't have your payment gateway go down. If your payment gateway goes down, you're bleeding money. So I think again, we got back to those pace layers, figure out where in your company you can apply it. So you talked about citizen developers. We wrote an internal platform called ShipIt. We have just had a designer who had never written a line of code in her life ship a mission critical app to production that we use to track outcomes by herself on this platform.

(32:51):
It is vibe coded. It breaks every once in a while. She's fixing them as they come up. This tool would have never existed. It's not perfect, but okay, it's good enough, but it's not on the critical path of our business. We could survive with that. So I think you got to find these places where you can let your humans play with AI. You can also start to build your software factory. So ShipIt, we have some of these ideas built into ShipIt. It's all running in commercial so we don't have to worry about government data. And Alexandra, she crushed it. She released it to the whole company yesterday and she's like, "Oh, we're going to have an outage. I need to make a fix." And she's making them by herself with no engineering support.

Nicky Pike (33:32):
See, I think that's awesome.

Mike Gehard (33:33):
That's pretty cool.

Nicky Pike (33:35):
Yeah, absolutely. Everybody right now, they're talking about AI, about how AI can write their code, but you've even said, "Don't ask that question. That's the wrong question." The real question people should be asking is whether it should and in what areas, right? So how do you explain to a team, how does a team even start answering that question, whether it should be writing code or whether we should be using it somewhere else if they've never even kind of stopped to consider it?

Mike Gehard (34:02):
Yeah. I think you should be using it to write code. I haven't written a line of code in two years, but I've looked at a ton of code. So I think code generation is a solved problem. We know the machines can generate code. So I think that's one place you can do it. I think you just need to be realistic that that was never your bottleneck. Now that the machines can write code, you're just going to solve all your business problems. I loved your prototyping example. We just had the exact same experience on this project I'm working on. So we had a designer using Claude Code, literally vibe code a prototype, a single page web app, HTML CSS JavaScript, put it in front of the customer and validate that that was what the customer wanted. We are now using it as engineers to re-implement the stories, but now we have a prototype we can point the agent at and says, "Hey, go use Playwright." And it actually runs the Playwright server, runs the app, and will go in and look at the prototype for us and then re-implement it in code and we're going through the whole SDLC on that thing.

(35:07):
So I think again, another place that this designer just said they wanted to do it and they figured it out. So I think that's the other piece. There are still places where you want to let your people explore.

(35:20):
You can't hold them all to token budgets, but you need to be very clear with them that their goal is to learn and to expand that practice within the company. So I think this is where another place where people get into trouble is they don't figure out how to share the learnings within the company. We're struggling with this now too, but I think prototyping, logs, triage. So we run all of our AI agents in a dev container because we're running in controlled government environments. So I manage all of that project. It's a very small project, but I actually have an agent that does all of my software version updates for me. Every Tuesday night, it runs a skill scheduled by GitHub Action that goes through and updates all of my versions. And I come in Wednesday morning to a pull request. I take a look at it.

(36:14):
I was like, yeah, it looks about right. Hit send another one. I have an agent that will do CVE triage for me. So when our trivy scanner pitches a CVE exception, I just have the agent go triage it for me and give me one of two answers. Can I upgrade a version or do I need to do a trivia ignore on this thing? Another great place where agents are great, off the critical path, pretty low blast radius on that thing. I never have to triage CVEs again. So I think again, you have to figure out what are the tasks you do and then you automate them with AI. And I love the we do, you do. So Pivotal, we used to say I do, we do, you do. I've just cut out the I do because I'm never not pairing with an agent. So it's always we do together.

(37:03):
And then my goal is always to extract some sort of skill that I can now get the agent to do on my behalf. And we just refine that for a little bit. So I'm like, yep, that looks great. I trust you. Go off and do your little thing over there in the corner. So identify those little tasks and do it a couple of times with an agent, export a skill. You can then share that skill across your company, wash, rinse and repeat. This is the other big mind shift that I've been finding with people is that for so long we identified, I'm an engineer, I'm a designer, I'm a product owner, balanced teams. We talked about those roles. I think it's time for us to break down the role definitions and start to talk about tasks that we do. And I tell all of my teams, as an engineer, my job is to no longer build software.

(37:53):
My job is to build software that allows my whole team to build software. So how do I get my product owners to be doing better stories? I just wrote a skill yesterday that allows my product owner to do acceptance and give the acceptance feedback back to the agent and have the agent fix what they found in acceptance so it's not a story rejection. It just fixes it and he submits a PR. We review the PR. We massaged it a little bit with our adversarial agents that we wrote the other day. Perfect example of like, I don't want to have to do story. I don't want to have to tackle a rejected story, so I'm just going to empower my product owner to do that work. So it's finding those pieces that you can not have to do ever again.

Nicky Pike (38:39):
And I love the skills and the different agents. We recently talked to Russ Miles. You know Russ Miles. We followed him up.

Mike Gehard (38:47):
I love Russ Miles.

Nicky Pike (38:48):
Oh, such a great guy to talk to and definitely want to put the link down for people to go see that because he brought up the accountability portion of this. We've got to bring accountability back into the AI era and who owns what, but he did along the same lines. We wrote it with skills and agents to go in and do things like, am I in the right frame of mind to go be making these decisions? So he's got his Abacus Diablo agent, he's got his, I can't remember the name off the top of my head, but these are the kind of things that I wish more people would pay attention to that's not writing software necessarily. They're using AI to make sure that we're in the right frame of mind, that we're actually following through a process well so that we can be the right or architect good software.

(39:32):
And I think that's a key point. There's a lot of different ways to use AI here.

Mike Gehard (39:36):
Yeah. Russ, his harness is an interesting one. He has the agents that his whole idea is about habitat. This is not machines writing code. This is humans and machines working together with different types of intelligence to write code better. And yeah, I mean, I will tell you, I get to times where the dopamine is going and I'm just like, Claude is going and I'm like, I probably should step away from the computer. And Russ has built this into the harness that he has. So accountability is a huge thing. I think this is the one piece that we haven't solved yet. So when my agents and I submit a pull request, who's responsible for that pull request? Who's responsible if nobody reviews that pull request? Now, we want to do blameless postmortems and stuff like that, but at some point, someone has to be held responsible for that.

(40:29):
And I don't know what that looks like in the age of AI because I mean, my agents sling code under my GitHub credentials and I sometimes review the pull requests, sometimes I don't. I think this is the piece we're going to have to. And I think, I can't remember if it was Chad Fowler or Charity Majors from Honeycomb, but I heard it somewhere where accountability doesn't go away, to your point, it moves.

(40:56):
So we have to either... I talked about trust. Well, how do I trust something? How do I trust a human? Well, there's accountability there and they've satisfied my accountability meter enough times where I trust them or I trust the process.This is where we start. How do we build that trust and accountability so a human doesn't have to hold the accountability, but we can get deterministic code to hold the accountability. So I think there's something here we're going to have to figure out, and I don't know what it looks like because I mean, it changes every day.

Nicky Pike (41:27):
Yeah. I mean, on the accountability endpoint, it feels like the easy answer is, hey, if you signed off on the PR, you're accountable for what the code did. But it's not that simple. When it's become the PRs have become the bottleneck, how do you start separating the we from the chaff on this one to say, to rank these PRs, here's things I really need to pay close attention to because if we try to put the human in the loop and we really try to do the same efficiency on PRs that we did when we were humans, we're now the bottleneck. And I won't even say that the PR is the bottleneck. I'm going to say that humans are the bottleneck because we can't review fast enough. So how do you split that out? That's an unanswered question. That's another one of those things that we want to have on this show is ideas around that.

(42:13):
How do you start looking at these bottlenecks because to you, to Gene Kim, to almost everybody's point out there, we'll solve one bottleneck, which everybody thought was co-generation.That's no longer a bottleneck. Now it's PR. I gave a talk just this last week where if we solve the PR bottleneck, then we're golden. No, you're not. It'll be another thing next to that. You got to start trying to think ahead and trying to figure out where your next bottleneck would be. What would be your advice to software teams that are going through this right now? How do they start trying to get ahead of this and figuring out where their next bottleneck is?

Mike Gehard (42:45):
Yeah. Take a nice hard look in the mirror and be a real honest with yourself. I mean, I think Russ, in his workshop he gave it at Prodacity a couple of weeks ago when we saw him speak, he talks about various levels and he did this workshop where he actually gives you the levels and you go through very honestly, where am I on this spectrum? And that's really hard because everybody wants to be level six in AI. So I think you have to sit down, be very honest with yourself that you are where you are. You need to figure out where you're at and then you have to ask yourself, where do I want to be? Honestly, where do I want to be? What would actually be feasible for me to get to? And you can just start ticking those things off and just being like, okay, cool.

(43:33):
Every day, where can I have a machine do something that I used to do? I guess the other thing, concrete thing, ask yourself what value do you get from pull requests? Everybody's like, well, I need to understand the code. I have never fully understood any code base that I've worked in. How could I, unless I were review... So I think you have to ask yourself, well, what's the bottleneck? What am I missing? If this is the bottleneck, what am I learning from this bottleneck? And can I learn this somewhere else? Continuous delivery. There are no pull requests in continuous delivery. Well, you do, but no human ever really looks at them unless they're high impact. So that's the other thing. You need to ask yourself, is this a high impact change? And come up with some sort of triage mechanism that says, is this change I'm getting ready to make?

(44:25):
What is the risk of this breaking something? I think back to your original question, just to sum that up, it's like honest with yourself. Where are we right now and where are our pain points? Honestly, what am I doing that I don't want to be doing anymore? Get a machine to do it. And then for leadership, stop pressuring your people to adopt AI. Tell them you want them to adopt AI and give them the incentives to measure the outcomes of what the AI is doing for your company, which is really hard. It's a non-trivial thing to do. And then sometimes you just got to believe it's working.

Nicky Pike (45:03):
I'm going to reiterate what you said. I 100% agree with you. Let your people experiment. Tell them you want them to adopt AI. Let them figure out what works best in their processes to do so. It's your responsibility, I think, as a leader to give them a safe and governed space in which to do that experimentation so that it doesn't come back and affect your company. But if you want to realize the true benefits of AI, we've got to quit dictating to people what AI is supposed to be used for and letting them figure out how it's going to work best for them.

Mike Gehard (45:31):
And hold them accountable for outcomes.

Nicky Pike (45:33):
100%.

Mike Gehard (45:34):
Be very clear with them. This is the outcome I want you to have. I want you to increase engagement on the website by 15%. Yep. Great. I can do that without AI too, but I can do it probably pretty quickly with AI. So I think as a leader, you want to give them outcome-based metrics. And this is the other thing, a lot of companies don't have outcome-based metrics. So everybody will like, oh, well we do KPIs, but are those outcome-based metrics? Maybe not. So I think as a leader, the onus is on you to define what outcomes your people need to have with AI. It's up to them to figure out how to get there.

Nicky Pike (46:11):
All right, my friend. Well, you heard it here. Let's go ahead and jump into the rapid fires. Mike, I don't want you to think too hard about these. I want you to answer off the top of your head short sentences to defend your answer. Ready?

Mike Gehard (46:22):
Yep. All

Nicky Pike (46:23):
Right. Test before code or code before tests?

Mike Gehard (46:26):
Does it work?

Nicky Pike (46:28):
Neither one?

Mike Gehard (46:31):
If the test fails, I don't care when they were written.

Nicky Pike (46:33):
Okay.

Mike Gehard (46:34):
As long as the test fails. If the test fails when the code is broken, it is a good test.

Nicky Pike (46:38):
Cool. The one AI governance role that every team should have and almost nobody does.

Mike Gehard (46:45):
Oh man. Make sure you're not shipping customer data. Understand the data retention policy of the model you're using. That is the one. Understand the data retention policy.

Nicky Pike (46:59):
Good advice. The worst piece of AI generated code that you have personally shipped?

Mike Gehard (47:05):
Oh man, I wrote some crap over the weekend. I had Claude write a type script file that generates new VMs for me. I'm starting to use AI for open source stuff, so it spins up a new VM and it was complete junk, whatever it works.

Nicky Pike (47:22):
I think a lot could be taken from that answer, and it's one of the reasons I love you so Mike. Ah man, I just wrote a great big pile of crap this weekend, but I imagine that you learned something from it.

Mike Gehard (47:32):
I did, and it works. It spins up VMs for me. I'm working on these two projects with it now. I have no idea what it does. No idea how it works.

Nicky Pike (47:40):
Mountain bike trail or terminal. Where do you figure out your hard problems?

Mike Gehard (47:45):
On the mountain bike.

Nicky Pike (47:46):
Mountain bike. Do you think that's just because you're taking yourself out of that?

Mike Gehard (47:51):
I'm thinking of other things. I'm on the mountain. I'm just out not thinking too hard and I'm probably breathing out my eyes at this point trying to get up a hill somewhere.

Nicky Pike (48:01):
One phrase from the conference stage that you want to see retired in 2027.

Mike Gehard (48:05):
Man.

Nicky Pike (48:07):
That's a hard one because there's probably five I can think of in

Mike Gehard (48:10):
My head. Yeah. You should.

Nicky Pike (48:15):
You should. Yeah. All right. Defend that.

Mike Gehard (48:17):
Don't be shoulding on people because I think the hard part of telling some they should do something, this is back to adopting other people's process. If you don't have the same problems or the same constraints, then it doesn't work. I think what I try to do, I do this with my partner all the time, is like, "Here's what I would do in this situation and here's the situation." Your mileage may vary, but I think when we start to tell people you should be doing this, we don't empower them to make their own decisions based on their constraints and we shame them into adopting the way we think the world should operate. So I think it's hard because everybody wants opinions, but I've been trying to eliminate should from my vocabulary.

Nicky Pike (49:03):
I am still in that one. Quit shoulding on people. I love that statement. That works in so many different ways. All right, my friend. Well, we end each episode, we go into some predictions and I'm going to get you to give us some predictions. It's 2030. What do you think a developer's workday looks like in 2030?

Mike Gehard (49:21):
I believe we have solved the AI quality problem. I believe that AI is generating code. I believe that AI is regenerating code if Chad Fowler gets his way. I think our jobs as engineers is to determine what gets built. We are not engineers anymore. We are humans that talk to humans to solve human problems, and we are helping all of the rest of our team who have other skills that we don't have and do the same thing. So we are building the systems that build software and we are no longer building software. And the quality is high and we've solved all of the problems that we have right now with Slop and we're like, "Yeah, let's ship that to production without looking at the code."

Nicky Pike (50:08):
Nice. Well, and on that same lines, this is going to be your spicy question. With all of the hype around dark factories and agentic loops and human out of the loop becoming something that more people are talking about, do you think this becomes a default as AI capabilities mature or do you think this is just going to stay a flex that only say a handful of companies can actually pull off?

Mike Gehard (50:32):
Oh, I'm going to get spicy on this one. I'm going to hypothesize that companies get replaced on their ability to adopt this technology. So I think companies will pop up that will become competitive enough with incumbents in certain areas where the ability to move fast and of adopting an AI first mentality will help them unseat people. You're probably not going to see Capital One disappear because they're a bank, but I think you will start to see incumbents fall because they can't adopt this technology fast enough because they have these calcified organizational structures. I mean, we tried to do organizational transformation for a while and it works to a certain degree, but I think people will fail to adopt these technologies and will fall behind those companies that can adopt these technologies and do the work to adopt these technologies.

Nicky Pike (51:28):
I don't disagree with you on that one. I think we're going to see a lot of change. All right, my friend, last one. I mean, you've worked at VMware, you've worked at Pivotal, you're working at Rise8 now within R&D of AI. You're a conference speaker and you've got a Substack which we will put in the show notes for people to go and read about how to do AI assisted agile development. You are someone that says you haven't written code in two years, but what does it mean for you to be a coder?

Mike Gehard (51:56):
I don't think I was ever a coder. I was always someone who used code to solve problems. I think this is where we're going to see the rub. A lot of people still want to think about the code that they build with their bare hands. I think we've entered the Ikea phase of code generation. You can go buy an Amish dresser and pay a thousand or $2,000, but I think what our customers and our economy is going to demand is the Ikea version of software. So I think those of us that always worried about the code and the coding are really going to struggle. Those of us that thought about the product and the problems I can solve with software, which is where I've always put myself, those are going to be the folks that I think are going to lead this revolution into the future.

(52:48):
Talking to Daniel Jones on a podcast and he came with this idea of how cool would it be for us to build outcome factories? So you go into a restaurant, the server is struggling with the crappy software that someone has built for them. They simply asked AI to change the UI for them in real time to solve their problems that they're having right now, and that just shows up in production 30 seconds later and the rest of their day is magical. How cool would it be to live in a world like that where I never have to worry about software that I build is janky because a human can't use it. Every time I walk into a store and I see someone working behind the counter trying to get through their day doing retail and they're just struggling with the software they're using, it just pains me.

(53:41):
I just put my head in my hand and apologize to them for being a software developer, but I want us to get to this place where that no longer happens because they can just ask for changes to be made to make it easier and it just shows up in production.

Nicky Pike (53:54):
Oh, I think some of the smarter people in the audience are looking at this and saying, "Okay, knowing what Mike does right now, working with defense, working with the federal government, let's extrapolate what he just said out to those places where the end user is an operator or a war fighter or a defense contractor." I think that's a topic for another day, my friend, but I do think it's going to be a topic for another day. Folks, you heard it. We talked dark factories, we talked software factories, we talked trail riding to get past the big problems. Mike, at this point, can we consider you to be a full-fledged member of the [Dev]olution?

Mike Gehard (54:28):
Sure, let's do it.

Nicky Pike (54:29):
Absolutely. Well, we'll look forward to it. Well, sir, thank you very much for joining the show. Everybody out there, if you got questions, we put several calls to action out there, leaving the comments. What are you doing with your PRs? What are you seeing with your PRs? What are some of the metrics you're using to help us get through or help your company get through to see whether AI is a good return on investment for you? We want to hear it, and I promise you, we'll get Mike involved if he asks some of those questions. I'm not going to let him go. Thank you very much. And Mike, we'll talk to you. We'll see you again soon, buddy.

Mike Gehard (54:57):
Thanks.

Nicky Pike (54:59):
Thank you for listening to [Dev]olution. If you've got something for us to decode, let me know. You can message me, Nicky Pike, on LinkedIn or join our Discord community and drop it there. And seriously, don't forget to subscribe. You do not want to miss what's next.