Subscribe
Share
Share
Embed
The development world is cluttered with buzzwords and distractions. Speed, focus, and freedom? Gone.
I’m Nicky Pike. And it’s time for a reset.
[Dev]olution is here to help you get back to what matters: creating, solving, and making an impact. No trend chasing, just asking better questions.
What do devs really want?
How can platform teams drive flow, not friction?
How does AI actually help?
Join me every two weeks for straight talk with the people shaping the future of dev.
This is the [Dev]olution.
Gene Kim (00:00:00):
Coding has always been the ball neck. That's why you have prioritization processes and budgeting processes and capital allocation processes all to protect coding. But when that becomes free, everything else on the bottleneck.
Nicky Pike (00:00:11):
This is [Dev]olution, bringing development back to speed, back to focus, back to freedom. I'm Nikki Pike. Okay, so everyone's talking about how AI makes developers faster, 10 times faster, 100 times faster. Pick your number and somebody out there has a slide for it. But here's the question that nobody's talking about. What happens when people doing all the building aren't developers at all? When a product owner who never wrote a blog of code, an ER doctor, or a 35-year police veteran, all of them shipping software now. So if your company can suddenly build and the rest of the company ready to catch what they throw at them, today we're going to find out. So our guest needs no introduction, but for the folks who maybe weren't paying attention during the TevOps days, this guy literally wrote the book. The Phoenix Project, the DevOps handbook at my old company, those were required reading the stuff a whole generation of us actually learned this craft from.
(00:01:07):
His newest one, Vibe Coding, is about exactly where this is all headed. He spent 30 plus years studying how the best technology organizations really work and right now he's burning through quad code tokens, co-building software with people who have never written a line of code in their lives. Gene Kim, welcome to the [Dev]olution, my friend.
Gene Kim (00:01:26):
Oh my gosh, Nikki, it's been a long time coming and so great to be on and so much looking forward to this hour.
Nicky Pike (00:01:33):
Oh, I am too. This is a great honor. Like I said, I came up a lot of people that I know came up reading your books. You taught us how to do this. So this is a great honor for me to have you on here, man. Thank you very much for being on.
Gene Kim (00:01:45):
Oh my gosh, absolutely. And by the way, if I can thank you for that kind introduction. And if I can just maybe share with you how this journey led me here. So I spent 27 years studying high performing technology organizations and we called those organizations that had the best project due to performance and development, the best operational and reliability and stability and ops and the best posture, security and compliance. And so I thought until recently, the biggest surprise was how it took me into the middle of the DevOps movement. And I felt like I had a lot of fun in my career. I got to work on the state of dev ops research with Dr. Nicole Forsgren and Judge Humble. That was a cross-population study that spanned 36,000 respondents over six years. That's where the door metrics came from. Then I meet Steve Yegge in June 2024, famous for his 20 years at Amazon and Google.
(00:02:28):
Famous for many, many things. And it turns out that one of our common areas of passion was how AI was changing coding out from underneath us. And so by six months later, actually in October 2024, we pair a program together and that changed my life. He asked, "What do you want to work on? " And I'm like, "Oh my gosh, I listen to a lot of YouTube videos and podcasts, and every time I hear something interesting, I take a screenshot. And it turns out I have something like 6,000 screenshots I've taken of things that were interesting. And about how many times I've actually gone back and actually looked something up, read, listened to it or something. It was like something like three times. And so I was like, oh my
Nicky Pike (00:03:02):
Gosh,
Gene Kim (00:03:03):
What I want is very simple. I want to take this pilot screenshots, a YouTube video, a transcript, and I want to be able to create video excerpts around each screenshot and post it to social media. And so I did it. I wanted to do this for over 10 years and we did it together in 47 minutes. And this was in the age of Opus 3.5. There was no clog code. There was no coding agents. It was just copy and pasting in and out of these copilot windows on the side and to do something in 47 minutes that you wanted to do for 10 years, I promise you it will change your life. Did mine. And when he said he wanted to write a book on what we are now calling Bobcoding, that term didn't even come out for another year and a half. I just found myself saying yes.
(00:03:44):
And nothing hold a candle to how much fun I'm having now. And like so many of us, I'm learning so much every day. Anyway, so yeah, Nikki, it's just been an adventure of a lifetime.
Nicky Pike (00:03:54):
Well, and you did it on Opus three, which nowadays Opus three is the old stuff, right? That's going back there. You can do so much more when we're looking at the new models that are coming out. And it only took you 47 minutes to do something that you've been wanting to write for 10 years.
Gene Kim (00:04:07):
Yep. No, exactly. In fact, many of us, we got a little taste of anthropic fable before it was disappeared by the federal government. But oh man, I miss it already. I mean, you're right. These models are getting more and more powerful. And it's just amazing what happened right as we went through Opus 3.5 to four to 4.5 to 4.7. They say that there were some definite step changes in terms of how powerful these models are. And I think we got a little glimpse of the future with Fable and I sure hope it's coming back soon.
Nicky Pike (00:04:34):
Reddit right now. So many people are complaining because it's been taken away. But yeah, it's almost like it's a version of a drug. Everybody wants that next little hit going on. And it's amazing to watch people that are so upset by that and I get the reasoning why. Now, Gene, before we go on, you kind of alluded to this. I do want to kind of bring people into the story, maybe those that don't know, which I would be surprised, but maybe those that don't. I mean, you've spent your whole career since what, like 1999, right? You study what separates the great technology orgs from the ones that kind of stall out. What actually pulled you into that study? I mean, what was the moment that you realized, Hey, man, this is where I want to spend the next couple years of my decades of my life on.
Gene Kim (00:05:13):
Oh boy. Yeah. So I think many technologists now, they might not remember the days when organizations only deployed once a year. And they weren't great experiences. They were very dramatic, very exciting in the worst possible way. Many things would go wrong. They would call chaos and disruption downstream. They might even cause multi-day outages. And so we got used to rehearsing for these deployments. And then in 2009, John Ospa and Paul Hammond, the VP of infrastructure and the VP of development at Flickr said they were doing 10 deploys a day every day. And all firsthand accounts to the people who were there said even in that community of Silicon Valley startups and so forth, they thought that sounded like ridiculous. Ridiculous, irresponsible, reckless, and immoral. Because if you're doing one deployment a year and that's not treating you so well, why would you do 10 deployments a day?
(00:06:06):
And
(00:06:08):
I think that's what really was one of the handful of things that sort of catalyzed the DevOps movement that said, look, if you want to have high reliability, you have to do smaller deployments more frequently. And so I mean, so many friends in the DevOps community, whether it's John Willis, Jez Humble, Patrick Dubois, I got to work with all of them in writing the DevOps handbook. It actually kind of gave, I was working on the The Phoenix Project at the time and it was just so clear, all right, this is a proof point we need that shows that there is a better way. And so one of the things that I'm most professionally proud of is working with Judge Humble and Dr. Nicole Forsquin in this study that showed over 36,000 respondents that the only way that you can get great reliability is to do smaller deployments much more frequently.
(00:06:51):
And so we measure that in terms of how quickly can you deploy, how frequently you deploy, how often does things go wrong, how quickly can you recover? And so in that journey, we also found that these high performers have better security because they're integrating security objectives into everyone's daily work. They exceed profitability, marketing and productivity goals in military services, not - for-profits, government agencies. It helps achieve the organizational mission and goals. Incidentally, in those organizations, people are twice as likely to recommend their organizations as a great place to work because if you're in an organization where you can't build, can't deploy, can't get an environment, can't get production telemetry, can't get license keys, can't get provisioned, then it's a terrible place to work. You will never recommend that organization to your colleagues and friends. And so I'm so glad that you're a coder these days because getting people productive quickly so that they can actually do productive work, so important.
(00:07:42):
So that's where I thought maybe I was thinking kind of like, what next? And then vibe coding comes along and I guess I got to sort of put this into perspective. I don't feel like I'm a technology hipster. I'm a technology late adopter laggard, but I think like you, Nikki, I wrote my last production line of code until recently around 2000 and now I have, I don't know, like scores of things running in production. I'm on call all the time because I rely on the services I build and I've never had as much fun as now. I just have done these vibe coding workshops for leaders and I have all these pictures of people who haven't coded in 10 years. They used to be C sharp MVPs. They used to be very, very technical people, but then they chose the leadership track and they got to a point where their teams were working in technology stacks that they had no experience in and they, like me, have rediscovered the joy of creating things and coding and it's amazing to see them come roaring back.
(00:08:37):
You
Nicky Pike (00:08:37):
Actually led into, because one of my favorite things that you talk about, especially here lately, is who you're actually doing this coding with and it's not engineers, right? So paint the picture for everybody that's out there. Who are you actually sitting next to when you're vibe coding and what are you guys building? Because I think this answer is fascinating based on what you've been seeing out there. Yeah.
Gene Kim (00:08:56):
So often I'm working with technology leaders, right? I mean, that's people that are within our community who have spent careers either in infrastructure and operations and dev leadership and that's amazing to see them come roaring back without a question. But what's also been very surprising to me is paired programming with people who have never coded before. So there's a guy named Dr. Mike Stone. He spent decades as an emergency doc at Harvard Emergency and he asked me probably six months ago, I was like, "Could you help me write something where an LLM can analyze my patient records and the lab results and so forth?" And I said, "Sure, just go get some API keys from your electronic medical record system." And part of me was like, "I'll never hear back from him again." But he came back on Super Bowl Sunday and he's like, "I got all the keys, API keys." And so against all my expectations we spent six hours writing this and a whole bunch of times I'm like, "Mike, I love you, but I'm not sure I'm your guy.
(00:09:55):
I don't want to be the guy who splatters all your patient records onto the internet." There's no way I'm going to leave this on, but we were so judicious about, he made sure that everything was HIPAA BA certified. We traded in his Google Workspace thing for a Google Cloud workspace. We made sure that we were using all the Google Auth proxies to make sure that nothing could be accessed on the public internet. And it's not my expectation, not only did we get it running where we used tool calling to have these LLMs rifle through these records, it was really amazing, but I felt comfortable leaving it on. And if you had told me six months ago that I'd be willing to do that, I'd be like, "Yeah, you're crazy. I would never be that stupid." And yet it was not only fun, but I felt like we were very judicious and it's just fine running in production.
(00:10:41):
And another one was, this was about two and a half months ago, I did a vibe coding workshop with the US Navy and service sort of like the Air Force that you spent a significant chunk of time in. It was cool seeing these young officers in and out of uniform building applications. But the coolest part was later that evening when Sigma Defense, people were explaining what had happened that day where we had a hundred people vibe coding who've never coded before and he got immediately angry because he started complaining about a pickleball app that he's been using for three years and he's this avid pickball player. His name is Matt Jones. And if you look up interviews of him and every one of them, he describes how much he loves pickleball and he's also an event organizer. And as an event organizer, he hates this application where basically for the RSVP process it sends people to Google Sheets and a lot of his players are, let's say retired, so they've never been in Google Sheets and he spends more time cleaning up the Google Sheets than playing pickleball and the matches are not fun nor fair, right?
(00:11:44):
You have first time players playing really, really good players. Anyway, so it's like, "Hey, okay, how about we get 90 minutes next week and we'll write it together." And then there was this government buying officer and he said, "Normally for something like that, we would put together a request for bid for 5,000 hours." Sit up my drink. I'm like, "Nothing takes 5,000 hours." So I typed my phone to the laptop, I interview him and by two hours later we have the basis of an application that went into production the following week. And so Sigma Defense is like about a $500 million a year company to work on many interesting things. And one interpretation is that, what does pickleball have to do with anything important? But trust me -
Nicky Pike (00:12:28):
I was fixing to say, you pulled a bait and switch there because you're talking about the Navy and Sigma Defense and we're all thinking we're going to hear this great vibe coding story about something that has to do with the national security and then you switched it up on pickleball.
Gene Kim (00:12:42):
And he said, "Trust me, this is not about pickleball. I mean, this is going to inform every project they work on. " And there's this wonderful picture of him. In fact, I'm looking at it right now. It's like Matt and I sitting together in front of a laptop and he has this expression on his face of like, "Aw, enjoy." He said, "If you're working on a program and you're not one of those two people, the person building it or the person who's articulating what needs to be built, what are you doing? What value are you adding?" And Kent Beck, inventor of unit programming, unit testing, extreme programming, one of the agile signatories, I mean, this is like the Kent Becking ideal. He says it's always been about two people, person with a problem, person who can fix it. And the closer you can get them together, the better off you are.
(00:13:25):
So just to wrap this up, I mean, it was amazing. I told him to get a GitHub account and so he did and he was a cryptographer. He was never a programmer and it blew me away to see him creating these perfectly formed issues, these feature requests with ask your art mock-ups and perfectly framed corner cases and all the edge cases explored and then answered. I was telling these stories to my son, I have three kids, 18, 16, and 16 and Grant he said, "Dad, as I'm driving to school, dad, isn't this all encroaching on your craft, which you spent 40 years getting good at?" I'm like, "No, no, this is the best thing ever." You said, "No, Gartner, 4X more developers?" No, no, no, no. We're going to have a hundred times more developers. And here's the math, 28 million developers on the planet now, you multiply that by a hundred and you get about 2.8 billion.
(00:14:19):
So that's about a third of the world population and I think that sounds about right. So over to you, Nikki, I mean, does that resonate with your own -
Nicky Pike (00:14:29):
It does. And this is one of the reasons that I love talking to you so much because I remember one of the first talks I gave when I came to Coder, Gene, was about how I did a comparison against how AI was like Home Depot, how it made DIY possible to those that didn't need to have a license. And people argued with me and of course we saw the huge blow up on LinkedIn about no vibe coders should never touch code. They don't know what they're doing, but here's what we're starting to see. And it's become like a top three priority on almost every enterprise that I'm talking to because they've got these people, the domain experts, they know everything that they want. They have these great ideas, but they don't have the technology skills to go out and implement them. And if you put them into the IT team and the software development team, they go on a backlog, people get frustrated and they end up giving up these big ideas.
(00:15:15):
But now we're getting the ability to say, "Hey, this is the new wireframe. Go out and build the application." We can use the IT team to polish it up, secure it if we need to go through. Because let's be honest, some of the stuff that you get from vibe coding is slop, right? We see it every day, people complain about it and I think you do have to have some knowledge there, but the ability to unlock this for people that have never touched code to give them the ability to envision and instantiate their ideas into something that works, this is free. This is something completely unseen before.
Gene Kim (00:15:46):
Absolutely. In fact, I've generated more code in the last two and a half weeks than I have in a really long time and it's been so amazing. So I finally found this article about the invention of the spreadsheet. So think VisiCalc, Lotus 123, Excel. It was transformative for exactly the same reason. People could actually solve their own problems without the finance department, the accounting department. And so that actually did destroy about 400,000 clerk accounting clerk jobs, but it created 600,000 accounting jobs including auditors and so forth. And so I mean, can you imagine how liberating it would be for a sales manager to not have to ask the accounting department of the... I meant our CFO at Tripwires, this was in the early 2000s. His first job out of college was working on a spreadsheet. So he's probably about early 70s. And so back then in the 1970s, the spreadsheet job was sitting in front of a big sheet of paper and going across and down.
(00:16:48):
So tabulate across and then down. And when I heard that, I just started laughing because I remember in, was it multiplan or Lotus 23, it was a configuration setting. It's like, "Do you go across and down or down and across?" And I'm just like, "Oh my God." That's what people used to do. So to redo your sales forecast meant having a team of people redo the spreadsheet
Nicky Pike (00:17:11):
Well, let's be honest, Gene. I mean, we both still know companies whose primary tool is still Excel. They still run a lot of their business on spreadsheets, but I also think AI is changing that because why do I need to put stuff in a spreadsheet when I can have AI create it for me, visualize it for me, do everything that we're using Excel for in a much easier fashion?
Gene Kim (00:17:29):
No. And there are so many problems, right? I mean, how many ERP systems rely on the 20 spreadsheets that change, insert general ledger things and Angie owns that spreadsheet and Andy quit. Where's Andy's spreadsheet? I mean,
Nicky Pike (00:17:42):
Oh
Gene Kim (00:17:43):
My gosh, they forgot to set the filter correctly and suddenly we're missing 20% of our revenue. These are bad problems that I was just on a call with the CFO of a company and that was our job was like, all right, we are getting rid of those spreadsheets so that we can actually audit these, reconcile these. I mean, spreadsheets are great, but applications are better.
Nicky Pike (00:18:05):
Okay. So stop and think about what Gene just said. Not 4X more developers, not 10X 100 times more. He did the math live, 2.8 billion people building software, a third of the planet. And the really interesting part, he compared AI development to the spreadsheet. When Vis account showed up, it wiped out 400,000 accounting clerk jobs, but it created 600,000 new ones. The work didn't disappear, it moved. It got bigger and that's the way that's coming for the rest of your company. So if coding is really free and instant, or like Gene says, the rides at Disneyland are now instant and all your money is stuck waiting in line, then the line is the whole game. Your line is the outer loop. Coming up, we get into where you actually start, the concrete guardrails you stand up on Monday morning, who owns it when a non-developer ships something that breaks and the money because somebody's going to be holding the bill at the end just like everybody in the building does.
(00:19:03):
Stay with us. You're going to want to keep watching. Yep. Well, and I think this leads us in and I do want to talk. So for this episode, this is going to be the challenge on the table for us. So looking at Gartner, Gartner has forecast that inside large enterprises that we're going to see citizen developers, the people who are not the traditional IT, the professional engineers, they're going to outnumber actual engineers four to one. So when we look at this gene, picture company, 5,000 people, don't have a few hundred developers anymore. We've literally got a potential of 5,000 developers that could be building things, but the pipes, the outer loop that we built to handle that kind of code from a small team, that's fixing to get hit by everybody. So when you say that vibe coding is going to be a hundred times what DevOps was, right?
(00:19:48):
Yeah. That's the part that really gets to me because when we look at the history of DevOps, that rewired how we think about dev, about QA, security, ops, product, everything. So if vibe coding and citizen development, if this is a hundred times bigger, what specifically are we going to break first when the building stops being something that developers do, right?
Gene Kim (00:20:08):
Okay. Yeah. In fact, to put a frame on this, imagine the world where we did one deployment a year and we go to 10 deployments a day. How did that change? It changed development, but it also changed testing, it changed infrastructure and operations, it changed information security. It basically broke all those functions, which is the reason why in the The Phoenix Project, John, the security guy thinks that he's on this moral crusade to fix all the sins of the development organization and the organization wasn't doing great before and now they want to do 10 deploys a day. And he thinks
Nicky Pike (00:20:40):
This
Gene Kim (00:20:40):
Is very immoral. But in his path to enlightenment, he realizes this is the best thing ever because he's making development fully responsible and accountable for the information security objectives and he moves into a coaching consulting function. And incidentally, that's what testing did too, right? Like Netflix in 2010, I remember marveling as Adrian Kochroft, the cloud architect who migrated everything into the Amazon cloud. I remember him telling me at Netflix, we don't have QA. I'm like, "What?" And he literally had a QA function, reported to him and he gave it away. He's like, "That has to live in development." And so I think these days we call that shifting left and so everything shifted left to the developer. And I think another way to say what you said is that we are now shifting left development out of development into these citizen developers and the job of developers is really to help make sure that they can do it safely and securely and reliably and so forth.
(00:21:34):
And if it's your vibe coded data and analyst thing so you don't have to go into Snowflake or Tableau, right? Perfect. It's like, you know your data, you're relying on it. We can help you with ETL pipelines, but ultimately it's your application. That's not my job. But there are going to be some things that are mission critical that you definitely want to pull in some professionals if it's revenue generating, if it's safety critical. And I think the role of development is definitely going to change. And so even in development function, I think we're going to see numbers being posted that will shock people. I was just at the GitLab Transcend conference last week and I met the person who was in charge of the GitLab orbit team. So basically creating a knowledge graph among all repos and they did in two months tens of thousands of commits, thousands of merger requests.
(00:22:26):
They touched almost every repo in the company and I think some people would say that will thrill people like it thrills me. It will scare some people. And then some people will say that's all slop. And yet it's probably one of the more strategic initiatives of the company as these infrastructure companies start gearing up for this new era of computing. And one of the things that we were sort of laughing at in the last call is name a infrastructure tool that's just not falling over right now. I think one time GitHub had 87% uptime as like the number of mergers and commits are going through the roof is Sachin analyst that they have one new GitHub account being created every second, most of them non-developers. It's like name an infrastructure tool around coding that's not breaking right now. Sorry, over to you, Nikki.
Nicky Pike (00:23:17):
No, I think that's a perfect segue because when we look at like what you were talking about, when we did the one deployment a year, we came up where we worked for the software companies where I worked for a company where it was every two years we released something so people had to wait for new features. Now we did patching, but even patches had their thing and then we came into cloud native, we came into microservices, we came into CICD, getting things out there much quicker and that improved the customer experience because we no longer had to wait. If something was broken, we got immediate ability to see that almost immediate. This is amplifying that even more because now all these other people that may not be software developers, may not be product owners, have these ideas in their head, our ability to prototype now and just try out a new function infinite, right?
(00:24:00):
We can do that in a day, what used to take weeks or months with meetings and everything else. So the citizen developer, and this is just in the enterprise, which is primarily what we're talking about, but the explosion of software that we're going to see from outside the enterprise, from all the people that you're talking to doing things like, "Hey, I want to create," and I know this came from Sigma, but a pickleball app, things that are going to help their families, that's amazing to me and I can't wait to see that and I agree, I think we're going to see this explosion, but we started arguing, okay, yeah, we've made developers more productive. We've increased what is defined as a developer now, but nobody's looking at the outer loop. How are we going to make sure that we test these things? How are we going to manage the merge request and the testing and the qualifications?
(00:24:44):
And honestly, the only way that we're really going to be able to do that is going to be through more agents because things come in so fast. We're going to have to use AI to help us with our AI. So we've got to start making decisions about where human in the loop are, where humans need to intervene and where we got to check. Because like you said, none of the citizen developers are creating off systems for a banking system right now. So leave that off the argument. They're creating things that are going to make their automations faster.
Gene Kim (00:25:08):
So I ran the Enterprise AI Summit in April and that was fantastic and your colleague gave an amazing talk with the folks at RISE eight about when you have a whole bunch of non-developers building code, the first step is provisioning their environment. And if you're asking them to get wired into a Kubernetes cluster or a CI/CD pipeline, I mean, that's probably the deal killer right there. And so just having these amazing platforms that enable people to be out of the box already, it's critical, absolutely critical. Two is, and one of my favorite talks was Kent Beck. Who doesn't love Kent Beck? But he said every kind of process and practice that we've generated over the last 30 years that we are now accepting as commonplace, those all came from a kind of an economic trade off that we're making. But in an age where essentially for medium term, long term planning processes, we can basically say coding has become instantaneous and free.
(00:26:06):
I mean, look at token bills, right? It's not free, but compared to software development efforts in the past, I mean, it's free. So how many assumptions and trade offs change when something becomes free? So you have to revisit everything. And I think the reason why technology leadership is so challenging right now is that someone said, "We are now going to enter the greatest leadership challenge of the last hundred years." Big claim, right? But in 15 minutes, this person from a Canadian private equity firm, he's convinced me, he said, "For the last 50 years as a technology leader, you would inherit a process and if something didn't work, you blame them and you make a small little change and it works. And if it works, we still take credit for it. " These days, everything is an obstacle. Coding has always been the ballneck.That's why you have prioritization processes and budgeting processes and capital application processes all to protect coding.
(00:26:59):
But when that becomes free, everything else is bottleneck. And so in the vibe coding book that I wrote with Stevie Yaki, we said, it's like going to Disneyland, the rides are now instantaneous and all your methods is waiting in line. So as a leader, your job is to make sure that no one's waiting in line. And trust me, in these vibe coding things, if you don't have the right permissions, if you don't have the right authorization, you have to have someone on call the entire call, basically getting added to the right groups and blah, blah, blah, blah. Otherwise, you are as stuck. You might as well not have vibe coding as a power because it's not yours to use.
Nicky Pike (00:27:35):
Right. Well, and you mentioned Kent Beck there because I think he said something that I repeat quite a bit. Ken Beck came out and said, when people are asking questions about where this technology's taken us, what it's going to do, his answer was nobody knows. Nobody knows. And I think you even claimed that anybody that says they do, they must know something that Kent didn't. So let's sit in that for a second because I do believe that's true. We're all kind of just guessing based on the speed of things changing, what's going on. We're all kind of just guessing. Nobody knows.What do you think the most important thing that we genuinely don't have an answer for yet? What is that?
Gene Kim (00:28:13):
Oh, oh my gosh. I mean, where do I start? Hang
Nicky Pike (00:28:16):
On.
Gene Kim (00:28:17):
Quick
Nicky Pike (00:28:17):
Question. We're going to take a while.
Gene Kim (00:28:20):
What should teams look like when you don't need teams of eight people to do things consequential? What do code review processes look like? How do you trust and secure the stuff that AI builds and how do you measure dead productivity these days? Dora seems very like an incomplete expression of what's needed. How do you upskill thousands of developers all at the same time? I mean, this is why we have conferences is to share experience reports. And so it's interesting. Nobody knows. It's a very pithy, wonderful quote, but it's not that we're all making guesses. What it means is that we don't have experience reports yet to inform great answers to these at this point in time. And so it's not that nobody will ever know. It's that until we try things we won't know. And so another way to say this is that if you want to be on the frontier, you have to be trying lots of things and you have to be taking very good notes so you can determine what works, what doesn't.
(00:29:12):
You want to then be able to have a theory of like, why? Why did X work, but why didn't? And I think I was talking to Amy Willard and she was another speaker. She's the global director at all the important things at John Deere, 175-year-old tractor company. And she said, in an age where nobody knows, then if there's no canon, there's no book that's written down yet, then the next best thing that you need to rely on is your community of fellow leaders. So if you're not hanging out with people who are also doing distributed experimentation, you are at such a massive disadvantage because if you have friends that are all also on the frontier trying things, then all of their lessons become your lessons and vice versa. So it's a very heady time to be in the game and it kind of reminds me of The scientific revolutions that happened I guess around the time of Newton, but I guess around the time of Einstein, there are all these people sort of...
(00:30:08):
It's interesting, I was reading the book, The Making of the Atomic Bomb, and I remember when I was taking college courses on quantum dynamics and so forth, it seems like the quantum mechanics, it was all so abrupt and sudden. It's just alien and strange, but it turns out there are hundreds of people looking at the failures of Newtonian mechanics and how did it not apply to the atom? Why don't electrons eventually fall into the nucleus? And it turns out the particle, wave, duality, et cetera, they're all looking for a better answer. And so when Einstein put out his paper, there were a whole bunch of people saying, "Oh, that explains so much." And so he was working within a very productive community of practice that we're publishing, corresponding, sharing notes. And I think that's what's happening now within AI and technology leadership is we are all seeking answers.
(00:30:59):
In five years it'll all be obvious, but right now it's all very strange and puzzling and exciting.
Nicky Pike (00:31:05):
Well, and two things you said there. One, you mentioned John Deere, and I don't think a lot of people realize how big of a software company John Deere is. A lot of people heard, "Well, what does John Deere have to do with this? " But they do. They create a lot of software. But the other part of that is that nobody knows. You're right, but there's also, we're kind of in the matrix red pill, green pill or blue pill scenario. You can either everything that I used to know, I have to let go of because everything's changing. So do I choose to hone on or do I choose to move forward? And I think that's the point you're making is you've got to let go because we are in a new phase. We are looking at new things coming out and we're not going to know how they're going to impact us.
(00:31:43):
Things like DORA metrics, are those going to change? The things that we know we used to do to evaluate our productivity? Well, some of that stuff's going to go out the window. AI is bringing in something new. So you got to be able to let go of what you think you know and be willing to experiment and find new stuff. Am I following you on that one right?
Gene Kim (00:32:02):
Yeah, absolutely. I think the DORA metrics are like a law of physics, but I mean, here's what it doesn't explain. It's like how is it that I could be... There's a code base that I was very afraid of. I spent two and a half hours while in London with one of my colleagues and we spent two and a half hours with Fable, which was amazing saying, "All right, explain to us how this application works, how the deployments exactly work.What's the difference between production and staging?" And two and a half hours later, I'm like, "Oh, I get it. " It's like, "Okay, this is why it's scary. All right, here's my magic wand wish. I want to take it away from this weird Firebase single page application, JavaScript, blah, blah, blah with a horrible build process. I want to put it into Google Cloud Run, have it all be server side rendered, minimal JavaScript.
(00:32:45):
And before I went to bed, I just transcribed that into CloudCode and say, I'm going to sleep now. When I wake up, I would like to see a prototype of this application running and use all the same CSS and blah, blah, blah, blah. So it looks as much, it'll be pixel perfect copy of this other application. And when I woke up six and a half hours later, it was working. So did we do small little merges and review everything at one at a time? No. For all appearances, it looked like a one shot when there was a lot of planning that went into it and a lot of specification. But I think someone who takes a very little reading of Dora would say, oh, you did everything wrong. And I was like, no, this is magic. We did everything right. And I think part of my quest right now is like, how do we sort of create a better model for software creation that can take into account these magical conjuring up a software that was basically two minutes of transcribing and two and a half hours of research getting ready for it and explaining that magic where I woke up saying, this is the most incredible thing.
(00:33:55):
I call up a friend and like, dude, within two weeks we're going to replace our video library application with this new better thing and literally five years of aspirations and longing and anguish and frustration will disappear.
Nicky Pike (00:34:11):
And I'm with you. And let's go back because I do think that there was a good point there. Let's talk about the pipeline and this is something you know about. You're talking about this magic. I think most people agree with you. But when we talk about the pipeline, we see studies. There was a study out there, I think it was like 1200 engineering teams, and they found that those that are leaning really hard in AI, they are merging 98% more pull requests. But at the same time, the review time for those are going up 91%. So when we look at things like regulated industries like banking and insurance and things of that, we're seeing that the code writing's getting way faster, but the code reviews and the validation up by an equal amount. So you lived the operations side of this for decades. When we see that output double, but we're seeing the review gates buckle, where do you think that pressure's going to land and how do people and enterprises go to look to fix that?
Gene Kim (00:35:02):
Yeah. I mean, I guess one interpretation is like, oh, code review is a bottleneck and so we have to hire more code reviewers. I just don't believe that. I don't think that's what history teaches us. I mean, I think what we are going to be leaning into as you were suggesting is, okay, clearly there's a cognitive advantage in using these AIs. And apparently reviewing code is just as important as generating the code and we should be using and leaning into AI just as much for that. And holy cow, have we learned a lot about code reviews? The whole sort of methos project last wing showing that we're finding vulnerabilities in 32-year-old operating systems. That is amazing. It shows that these things are actually very capable of doing phenomenal code reviews. And I think I've heard that some of the magic was that Methos was able to not just find the issue, but also create a reproduction of the issue and generate a fix, which I guess makes it novel.
(00:35:57):
And so I don't have any problem believing that we can use AIs for reviewing code.
Nicky Pike (00:36:04):
So again, we're going back and we're looking at this and you said something earlier about GitHub, 87% uptime, I think you said. We're seeing a lot of these SaaS services that we're talking about, one and a half nines of uptime. We're seeing them fell over. Every tool's getting kind of hammered down.
Gene Kim (00:36:23):
You try building something where the number of commits go by three orders of magnitude. You'll see
Nicky Pike (00:36:27):
How
Gene Kim (00:36:27):
Well your infrastructure does that. But yeah, 100%.
Nicky Pike (00:36:30):
Yeah, I mean, I agree. Again, we're seeing this influx. I mean, is this something that you think that it's just a temporary growing pain that we'll be able to patch our way out of it? Or is this a sign that we are going to have to completely rethink the whole outer loop because the world that we built it for, it just doesn't exist anymore?
Gene Kim (00:36:48):
Yeah, it's interesting. I mean, one of the criticisms that we got in the Vibe Coding book is that we use the term inner, middle and outer loop. And again, our treatment of it was inner loop means literally you think about every second, maybe worst case minutes. Middle loop is days and weeks and outer loop is now weeks and months. And someone said, "Well, there already is an inner and outer loop." And I was like, "Do we need to reconcile this? " And Steve in his lovely way said, "Yeah, it's different." And because classically inner loop means on my laptop, outer loop means in the CI/CD pipeline. And maybe we've outgrown what that is and maybe there's actually more relevant loops that we should be looking at, whether it's the inner middle and outer loops that we talked about. Yeah, I don't know, but I think it is amazing.
(00:37:42):
Regardless of who you're using for CI/CD, who you're using for version control and all that stuff, the number of participants in that process is skyrocketing. The activity and interactions with those tools are increasing. A friend of mine said, "I'm at a financial services firm and we actually suspended security code scanning because we're out of compliance with our license because they meter us by number of lines of code we scan. And they're asking us how many lines of code will we have in five years?" And he's like,
(00:38:15):
"I don't know. Is it 2X, 5X, 10X? And why are we being licensed by the number of lines of code? Does that even make sense anymore?" Anyway, so I just love that because it shows that security scanning is a mission critical function and that got shut down in an organization because of licensing issues. Funny, right?
Nicky Pike (00:38:33):
Well, I mean, you look at this again, we're going back to we've got these things with AI, but AI really seems to be the only thing out. Security is one of those things. When a vulnerability used to come out, you used to have time to go in, investigate, look at it. Now we're seeing, and I believe I talked with Deshaun Carter about this previously, but now we're seeing vulnerabilities that are getting within hours of them getting released or being implemented because now we've got the AI speed to go and do this. The only way to keep up with that is to have 100% constant patching. That causes fatigue in humans. How are we going to do that? AI seems to be the answer, but how do we use that?
Gene Kim (00:39:11):
Yeah. In fact, I was talking to a friend of mine, he's a SVP of infrastructure at a large bank and he was saying the number of production applications, not necessarily tier one, but tier two, tier three is going through the roof. And he's like, "As an infrastructure team, we can't support that. The keep the lights on is going to kill us." And so they're looking at things like OpenClaw and something. Is there a way that we can use these kind of long-lived agents that you sort of talked through Slack, et cetera, as a way to help maintain these applications? I love that. Although dude, I had this really weird experience managing my OpenClaw where I was trying to get it to do something and it refused to do it. I'm like, "Stop, stop. Stand down. D not upgrade this file." And I texted a friend, I was like, "I've never had an argument with my Kubernetes
Nicky Pike (00:40:07):
Cluster." All I got was this picture of you from Terminators. You just running in with the X and trying to chop the cable because Cloud won't stop.
Gene Kim (00:40:15):
Yeah, no it was really weird and frustrating.
Nicky Pike (00:40:19):
I can imagine. I do want to go back because we were talking about, yeah, we kind of already got the inner loop outer loop and how we think those things are going to change. Me personally, I'm a big fan of what AWS put out as part of their AI DLC. We're taking the traditional software development life cycle and we're condensing it down into inception, construction and operations. And AI is assisted in that in every way, but we've got very defined boundaries for what AI is doing in each one of those. Have you looked at this? Have you seen it? What is your opinions on that kind of methodology?
Gene Kim (00:40:50):
Yeah, I looked at it and I love that people are working on this. I mean, it's funny, you just never know how these things settle out. What is the terms we're going to settle on? And a lot of people, they love that sort of DevOps infinity loop. I don't know who made that and I don't know what made it stick with people. I don't know why certain kind of ways we expressed the SDLC, why they sort of caught on. And I think we're in that phase where everyone's sort of experimenting with ideas and terminology. But I do love the fact that it's like inception, construction and operations. They are kind of three separate domains. I mean, it's actually really great. And I love the fact that they have the mob in the middle. I think what's also interesting that people talk about spec driven development, but it was actually done in collaboration with the Cairo team and so forth.
(00:41:38):
It was interesting to...
(00:41:41):
A lot of my friends, they love spec driven development and I want to love it. And when I heard the Cairo guy talk, this is that Patrick Devaus conference in New York. It was interesting. It's like I had this epiphany. I was like, "Oh, these are formal methods people. These are people that's all very grounded in formal methods." And it kind of explained to me something, which is that I love the rigor, but often I hate using it because I feel like every decision I want to make is I'm arguing with the committee.That to me, there's a place for that where you want that rigor, but in the majority of it is more of a mob-like ideation, iterative construction. And there you don't want to be arguing with committees. So I just think that's another example where what does the SDLT look like? Where do you want rigor?
(00:42:31):
Where do you want more kind of winging it?
Nicky Pike (00:42:34):
And this is exactly why I like AIDLC the way they put out. It's not even for the AI aspect, Gene. It's actually the human aspect of it. It's what you just described right there is we are forcing now a problem that has existed since we started enterprises, which is siloing of information and committees. You're forcing, if we're going to do this, we're going to get value out of it. Everybody has to be in the same room at the same time talking about this because AI is going to start spitting stuff out. And I am a huge fan of spec-driven development. I think that works best, at least in my opinion with AI, because here's what I want you to build, but I can't do a spec from a developer and then have another spec come in completely unopposed by security. No, we all got to be in the same room talking at the same time.
(00:43:15):
It's the human aspect of this, bringing in a balanced team that I really enjoy about this methodology.
Gene Kim (00:43:20):
Yeah. And I know the power of standardization. In fact, almost every project, I mean they look the same for me. It's enclosure, it's the backend. I have very little JavaScript on the front end. I use this thing called DataStar that kind of does server side events. Everything's in Postgres. Everything I want to be event sourced. I want a pend only because I don't want to delete data ever. And so to be able to give that to an LLM and say, "All right, this is what I want the shape of the event, of the application, the data to look like, " I love. Because to me it means it's safe, it's predictable, there's one source of truth. I mean, to me, that's a pattern I've used for, I guess, evolved over the last two years and I love it. And it just means that all my apps look the same regardless whether it's supporting an emergency doc, private practice, whether it's a pickleball app, whether it's a whatever.
(00:44:10):
So constraints sometimes can create incredible freedom.
Nicky Pike (00:44:15):
I agree with you, but I also see the opposite side of that. So I'm also, I like standardization. It makes things easy. It makes for an easy path. But again, this methodology, because it requires for it to give value, everybody come in the same room. If you want to go off the rails and you want to try something completely out of the box and new, this is easier because you got everybody there with the journey with you instead of doing it on your own and then trying to force everybody else to come with you. Now you're taking the journey together. Another reason I love this so much.
Gene Kim (00:44:43):
I love it. And by the way, this reminds me of, it was a leader at HP in the early DevOps days. He said, "Our goal is to create..." This is Ralph Loura, then CIO. He said, "As a kind of a standard architecture function, our goal is to create buoys, not boundaries." And he said in a river channel to know that if you stay within the buoys, you know that there's a certain amount of clearance in the river. It's safe. We have SLAs around them, we have the vendor contracts. But if you need a stray outside the buoys, you can, but then you have to follow these guidelines so that you do things, everything correctly, inception, construction and operations so that you don't actually go off the rails. And he said, "You never know. That might be the next pattern where we're going to dredge out that part of the channel because it's very important and that's the part of innovation." So 100%, thank you for introducing me to that.
(00:45:35):
It's just a really wonderful way of coining those activities that we all do. All
Nicky Pike (00:45:42):
Right. This may be the shiny moment in my life when Gene Kim tells me, "Hey, thanks for introducing me to a concept." I do want to take this into the practical and kind of get into some solutioning here. So for those that are watching, everybody that's watching what we're talking about, some of them are already absolutely drowning in what we're seeing with AI right now. We have this old way of human reviews. Every pull request, that was something people took pride in. We all know that this is not going to scale anymore. So what really does replace this? I want this to come from the OG, the DevOps Godfather, Gene Kim. Not a philosophy answer. What specifically do you think needs to change? What's something that a team could come in and they could stand up on Monday or Tuesday morning and put that guardrail in for them?
Gene Kim (00:46:27):
Yeah, I think we can actually bring in lessons from the very beginning of DevOps. I think when the practices are not known more than ever, you need cross-functional teams. And so when you look at any place where you're trying to do something different, the mission cannot be achieved within the big system. You have to carve out a small problem, get a team that's fully accountable and responsible for that, and they have to be liberated so that they're not dependent on everybody else to achieve the mission. Two is you have to have the right talent, the right skills, whether it's UX and design, development, AI, operation, infrastructure security. I think what's different this time around is that you don't need a team of eight people to do that consequential thing to create the proof point. I mean, it could be two or one engineer working on a significant part of the system, but you still need those skills.
(00:47:18):
And I think what everyone should be getting ready for is I just think about the GitLab orbit team. They touched almost every repo in the company, tens of thousands of commits across two months and thousands of merge requests. There was an experience report, Dustin Warner from NRC Health. He said we didn't wait two weeks to do a demo. We had demos every day. And then that's wildly insufficient. I mean, I can speak from firsthand experience. I'm working with this guy on replacing the video library legacy code here.
(00:47:50):
Engineers started recording with their phone demos, little two-minute videos and posting it in Slack because that's how much feedback you need. So as you get more work done, the frequency of contact actually goes up. You actually have to have more meetings and collaboration. And so that's a whole communication structure changes. And as a leader, you need to make sure that the right people are talking to the right people maybe all the time. It might be the customer. So it's a really exciting time right now. Agreed.
(00:48:21):
And I think in general, you have to have senior people on those teams and people who have logged a lot of hours with AI, as you said, as we were talking about beforehand. It's not something that you want someone who's only used tens of hours of AI. You want someone who's used hundreds or better thousands of hours of AI. There's a school of thought that says it takes 10,000 hours to get good at anything. And so the people who you want to be a part of these teams, at least a bunch of them should have logged hundreds of thousands of hours just to know what's possible. And obviously there's room for junior people too, but it helps to know where the ceiling is or where the perceived ceilings really are.
Nicky Pike (00:49:06):
Okay. So let me challenge you on that answer, Lilia. By the way, I do agree with most of it, but when we look at what's going on in the enterprise, we're seeing companies are seeing, we're doing all of these, touched every repo. We did all these merges and all this code. On the other side, we're seeing a rash of data leaks, AI doing things that are making the news, putting people in the news. There doesn't feel like there's an in between there for a lot of companies right now. Either I do this and I got to risk this. There's no if, ifs or buts. What is your answer to that?
Gene Kim (00:49:35):
Yeah, I guess I reject that because those fears should not be a reason why you shouldn't do it. I think there's plenty of evidence that says right now that you can do mission-critical things using these technologies. There is a way to use it safely. The problem with AI right now and AI coding agents is they are kind of dangerous. You got to hold them the right way, you got to use them in the right way. And that's where right now you do need more senior people. But eventually we'll get there where your citizen developer should be able to do more things safely. However, the universe of things to work on that are reasonable and sane things to take on without risking the enterprise is plenty large. So yeah, I think deep down inside, and Nikki, I know you know that's to be true.
Nicky Pike (00:50:24):
I do. And by the way, how do we constrain this thing? I know a guy that could talk to you about how you constrain agents, but
(00:50:31):
We'll save that for later. Moving on there. The other thing that we talked about, the specialization that we saw that came with DevOps, I think that's getting disrupted. The DevOps had whole classes. We had our Terraform people, we had our chef people, we had our metrics people. Your prediction is that the next wave of specialists might not be people at all. They might be the models, they might be the agents. Walk me through your thoughts on that. What do you think the first job inside the pipeline that you would actually hand to an agent other than the development? What's the first job that you would hand from a DevOps perspective in the new AI era?
Gene Kim (00:51:05):
So I think what's really exciting about what's happening is to do things securely and to work with Kubernetes and Terraform, you have to have a specialist on the team. And I think what's amazing is that it turns out these LLMs have read every manual on the planet and they are perfectly capable of writing a Terraform script, an Ansible chef puppet. And it's amazing. They are equally conversant in AWS and Google Cloud and Hetchner. I mean, it's just amazing. And I think what that does is that it decouples developers even further from the InfoSec and operations infrastructure roles. It doesn't mean that they don't need them. It means that they don't have to talk directly to one, which is actually great for the infrastructure specialist. Is that if they're building great documentation, if they've paved the road for other people, these AIs are phenomenally good at knowing how to...
(00:52:07):
You just point them at the paved road, they'll get there and stay within it. And this is great because you don't... James Wickett, he always said it's a hundred developers to 10 operations people to one security person. There's just never enough infrastructure operations and security people. And so the more we can lean on these LLMs to help people do their work securely and safely and reliably, the better off we are. And again, it doesn't mean that those roles go away because when you get into trouble, you are going to need that InfoSec specialist. You're going to need that security specialist. So I would view that as exciting. And I think as the frequency and tempo of development goes up, as coding becomes pretty instantaneous, it means that we need feedback more than ever. Dustin Warner, he said as a director of engineering, he's just always, he's very often starved for feedback.
(00:52:59):
He's like, "We got it done in a day. Who do we show and who do we need to get feedback from to know what to do next?" And sometimes I've heard stories of the bottleneck being product management. They don't get back to them fast enough and sometimes they go directly to the customer or the sales team goes directly to engineering. And so for him, Justin Warner, he said the biggest red flag is when he's talking to someone and they hear, "I'll get back to you. I'll get back to you in three days." And he goes, "No, open up your laptop. Let's just transcribe the answer right now because I can't wait three days. The opportunity cost is killing me here." So I think it's just the ballneck is moving further into the business and that's just a great, great thing.
Nicky Pike (00:53:42):
Well, and something I found very interesting. So the AWS team, when they were talking about their implementation, AIDLC, they did this migration for Bedrock and they used the AIDLC methodology for this. And one of the things that I found extremely interesting was the fact that not only how much time and resources did they cut out of that project and it was substantial, but they ended up writing a hundred thousand lines of code. But Gene, they used Rust as part of this, not because they though it was the best technology, not because there was some inherent reason for doing that. They wrote it because the error handling in their eyes was going to be easier for AI to read and react to. That's why they chose that. I think we're also going to see not only from the pipeline standpoint, but some really interesting architectural decisions being made around new applications and new code because of how AI's going to interact with it on the backends rather than humans on the front end.
(00:54:36):
Does that make sense?
Gene Kim (00:54:37):
Oh my God, I love it. This is amazing. I mean, it is the best thing ever. I mean, it's like your infrastructure operations almost wonderful dream. Can I share something crazy I heard yesterday in the monthly AI form? Please.
Nicky Pike (00:54:50):
I love crazy stuff.
Gene Kim (00:54:52):
Oh my gosh. So we run the Enterprise AI Summit and we had one in April. The next one's going to be October seven and eight in Charlotte, North Carolina. And I also run this monthly AI form. The speaker was Dr. Eric Meyer, widely considered to be the greatest programming language designer of all time. And we're talking about in April about how with the Methos model, Project Last Wing, he made this claim. He's like, you've got to be crazy if you're running open source software. If you're coupled to these things that always have vulnerabilities in it. I'm like, what? So they were joking about it. He said, "If software is free, why wouldn't you use these things when you can rewrite it from scratch?" And a bunch of us were laughing. And I was like, "Oh, so are you saying that open source is only for poor people?
(00:55:34):
" We all kind of laughed about that. I asked him about this yesterday upon some reflection and he said, "Yeah, no, it's probably a really great..." In fact, he routinely does this himself for many Java libraries. He will actually vendor that in. And there's a term now called AI vendoring that I learned from my friend at Anthropic where for critical dependencies, you rewrite it from scratch and you decouple yourself from the open source version, which kind of gets you off of that hamster wheel of pain. I just think that is wild. I mean, for 25 years we've benefited from open source software and it was always this incredible advantage. And maybe we're into this era where it's actually a liability. I mean, who knows? Nobody knows as Ken Peck would say.
Nicky Pike (00:56:16):
I think, was it you and I that were talking about this? Because again, that's one of those things where do I need to take the red pill or the blue pill? Because I'm the same way. Open sales has done loads for the community, has done loads for the enterprise. But you got me thinking, it was like, well, yeah, okay. Now I don't have to rely on somebody else patching something. I'll just rewrite my dependencies from scratch each time, make that part of my build process. Is it really going to increase my time that much? And it's got me thinking, man, this could change the way that we see open source when you, again, software's free. We can rewrite this as we need to. And I'm sure that there are other enterprises out there now thinking about it. Why do I want to worry about things like Shy Halud and all those things coming through the package repositories?
(00:57:00):
I'll just rewrite what I need every time I build time.
Gene Kim (00:57:05):
Maybe like you, I found myself really conflicted about it. But I mean, there's an element of what Eric Meyer is saying that it's just indisputable. All
Nicky Pike (00:57:18):
Right. Well, all right. So let's finish this up. Let's talk about really getting into this because I want to go back to something. Yo found Coder kind of the hard way. Yo found us when you were about to walk into the Rise8 and you were doing this infrastructure move. I want you to walk us through that story because I want to know what it's like. I think this is the exact same kind of pain that we're going to see from citizen developers everywhere on different bespoke setups. Kind of walk us through the story where you found Coder and how all of a sudden you were like, "Holy crap, this went from a need to have to something that I absolutely need to have. "
Gene Kim (00:57:53):
Yeah, for sure. So Steve Yegge and I have done vibe coding workshop for leaders ever since September of last year. And they've been so much fun to do. I mean, this is where I'm getting all those great pictures of people who are being reintroduced to the joy of coding. And so I got invited there to give a talk and run a workshop for helping people write their first vibe coded app. So the first one is like, "Hey, write a JavaScript application or do a bouncing ball with a trail of leaving a trail behind them, et cetera. Write a Tetrahedan cube shader like many of us wrote as an undergraduate student. And a lot of it relies on running in a VM, whether it's Google Cloud or Digital Ocean or whatever. And
(00:58:35):
We had a battle tested VM set up that spins it up kind of infrastructures code, et cetera. And I was just like, wow, we're in Salt Lake City. The last one I did was in Chicago. Shoot, I really should move this VM to the US West. And so I typed that into Cloudcode and it actually did that, but it deleted all the backups and the logical discs. I'm like, fuck. And it was one of those classic, what have you done? What did you do? Oh, sorry. I should have done that. You're absolutely right. And so it's funny because in my talks, I always described vibe coding is amazing, but it's also very dangerous because it's not very good at instruction following. It's not very good at not hacking the reward function. And so all right, it's T minus 90 minutes. I no longer have a working VM and the build scripts have disappeared.
(00:59:38):
The data sets have disappeared. And I was like, holy cow, I need some help. You know one of those feelings when you have a crisis and there's just not enough people thinking about the problem and there's not enough people typing. And so I tapped the person who's in charge of labs at Rize8. And this is where I met your colleague. And we did get something running, which was amazing. And it was just So I was able to focus on my lab and then the second lab we made sure that we were using the coder environments and so forth. And that was a huge relief because it have the problem size, but then I got to leverage all of the investment that they had made and standing up every environment that a student would have was already wired into their CI/CD pipelines. It was already wired into everything had the license key baked in.
(01:00:26):
And so that was just amazing. So he had a much less stressful experience than I did. And so it just really reiterates that for so many environments where you want dev environments ready to go, everyone productive out of the gate. Oh man, I was so impressed by the code experience. I actually asked your colleague to co-present with Max Real for Miz8 and present at the enterprise
Nicky Pike (01:00:49):
Age. Austin Brun. Yep.
(01:00:51):
Yep. Austin was part of that. And he was phenomenal. Yep. Good guy. He definitely knows his stuff and it is great. I mean, that's what takes you beyond just infrastructure as code into true AI ready environments as a service when you can spin these things up wherever you want to fairly quickly, have all that governance baked in. I'm glad that worked out for you. I though that was a great story. I do want to talk about what is on the minds of every enterprise leader out there right now, which is money. So now Gene, I'm not going to throw you under the bus on your token spin. If you want to tell the audience your number, that's on you. But needless to say, it's a lot. And I think that's the point. You're one person and here's the enterprise questions that we're seeing. When everybody in the building, when we start bringing in citizen developers and the domain experts, when they can spend like you do, who do you think's holding the bill on that?
(01:01:40):
I mean, how does any finance team really start forecasting that when we start talking about and how we're in this token economy?
Gene Kim (01:01:46):
Yeah. So I think a lot of us are looking at the fact that these subscription, whether it's the OpenAI Codex, 100, $200 a month, whether claudropic equivalents, I mean, they are the best deals ever because they're very permissive. You are easily getting at least 10X, 100X.You can consume more tokens than you're actually buying. I mean, it's sort of like an all you can eat buffet where you don't get kicked out. But it has been amazing to see for when you are on an enterprise plan, you are paying retail token costs for everything. And it really does... In fact, I just saw a tweet, many engineers spending some 500, 1000 or more per day. So you're spending $1,000 per day, that's like say $300,000 of just token costs per engineer. And I don't think many R&D organizations that budgeted basically for every FTE. They're spending an FTE equivalent on tokens.
(01:02:44):
So I think that's definitely true. And so I'm very excited about this talk that you're going to give at the Enterprise AI Summit about how can you use cheaper models where you're not paying an external frontier AI lab. You can actually just maybe spin one of these up in your data center. It really resonates with many things I've heard, which is that many smart firms are using these models. In fact, Nikki, I wanted to ask you this. I think I read something this morning that said Microsoft's new frontier models are actually based on DeepSeek. I just read that this morning, which
Nicky Pike (01:03:13):
They will be - I haven't seen that yet, but I'll be looking it up.
Gene Kim (01:03:16):
Yeah. Yeah. Which I think resonated with me because it says that these fears of non-US LLMs, you can still use them for important things even if it's not writing code. And it looks like Microsoft is actually going to be taking a bunch of their next generation frontier models are going to be based on non-US companies. I'm sure you're going to have some opinions on this, Nikki.
Nicky Pike (01:03:41):
I do. This is why I love you so much, Jean. This is supposed to be an interview about you and here you are hyping a talk I'm going to do at your enterprise AI conference here in October. This is about you, buddy, but I appreciate that. But yeah, I agree with you. I think we are going to be seeing a lot of the open source, open weight models coming in. There's a lot of fear around the foreign models coming out and yeah, there are ways that you can architect a lot of that fear out. But yeah, appreciate that.
Gene Kim (01:04:09):
Right. So you were right all along, Nikki. And by the way, some feedback on this potential talk is like, yeah, if you can actually get enterprises to say, absolutely. We also came to the same conclusions and we are actually using these open source models for these domains. Oh my gosh, that is a powerful contribution to the body knowledge.
Nicky Pike (01:04:32):
Yep. And we'll put this in the show notes as well. I am doing the talk October 7th or 8th, I don't remember which one, at the enterprise AI conference. It's going to be in Charlotte. We're going to put the link down there when you give us the okay that we can do that, Gene. So please check back in there, register everybody. Go take a look at this. I will be doing this talk there about how we can use open weight models.
Gene Kim (01:04:53):
Just to really land the point, the point is, oh my gosh, if you are spending $1,000 a day and someone can come up with an open source model alternative that costs $2 a day and you get like 90% of the benefits. Oh my gosh, that is an irresistible economic case.
Nicky Pike (01:05:17):
Yep. I agree. All right. I want to be cognizant of time. We're going to go through a couple more things here. This is the rapid fire section, Gene. I'm going to ask you a question. Don't think about it. First thing off the top of your head, just answer honestly. Let's see what you come up with these. Vibe coding for production code shipping to customers. Yes or no?
Gene Kim (01:05:35):
Yes.
Nicky Pike (01:05:37):
Okay. Vibe coding for internal tools and automations. Yes or no?
Gene Kim (01:05:40):
Yes.
Nicky Pike (01:05:42):
The bottleneck a year from now, the developer productivity, or is it going to be the outer loop?
Gene Kim (01:05:47):
It's going to be how many good ideas can you come up with?
Nicky Pike (01:05:50):
Oh, so it's neither one of those?
Gene Kim (01:05:52):
No.
Nicky Pike (01:05:53):
All right. The most overhide phrase in AI right now?
Gene Kim (01:05:59):
Slop. I love the word, but yeah.
Nicky Pike (01:06:03):
All right. Explain that. Why?
Gene Kim (01:06:06):
I think when people start posting these incredible numbers of like, we built a product in two months, it's arguably one of the most strategic capabilities for the company. We now have a term for apps that were written by amateurs and they have no idea what they're doing. We've all seen that, but not everything is slop.
Nicky Pike (01:06:27):
Yep. So I'm going to challenge you. We've all heard there's tech debt, which could be arguably slow. You could argue that that is slop that was created by humans, but we're seeing to put a different emphasis on slop that's created by AI because it's not done by typical developers. Do you agree with that? Do you think we're misjudging the words and we're just putting our own bias on it?
Gene Kim (01:06:48):
Oh, 100%. I mean, there are many dimensions to slop. There are many dimensions to like, oh, that's AI generated. I mean, I can look at some slides. It's clearly AI generated, but I don't always fault them for that. It's like, oh my gosh, there are many presentations. If you give once, you're an idiot if you are hand writing that PowerPoint slide. There are many AI apps where you look at it. It's like, oh wow, that was definitely written by an AI. The certain rounded corners with the left shading. But it doesn't make it bad. It just means that it was written by an AI. Does that make it useles? No. Technical debt. It's like, oh wow, you really cut a lot of corners and there's no automated unit testing and it's following bad patterns that make it very difficult to maintain. Yeah, that's bad. But humans do it too.
(01:07:34):
So yeah, I just think it is a wonderful word that defines a feel, that captures a feeling that we have, but it is multidimensional and maybe oversimplifies that many, many different things into one, which is often kind of a bad sloppy thing to do.
Nicky Pike (01:07:54):
Yep. Well, one of my favorite things on the same line is you've watched the movie The Hangover, right? You always go to Vegas. So I adopted. So along the same lines is somebody says, "Well, that's clearly AI." Okay, but did you die? That's my question. Who cares that you could tell it's AI? Does it work? If it does, then shut up. That's right. If it doesn't work, then you've got a reason to complain. But just because you can tell it's AI, what does that mean?
Gene Kim (01:08:21):
100%. I mean, you just nailed it exactly. And it could also be AI generated and generate awe and wonder.
Nicky Pike (01:08:30):
So here's a fact, just put this in the back of your head. They've got all the validate, look and see if this is AI detectors out there. I read a study. They put the Bible through an AI detector and the AI came back and said it was 98% AI written. Awesome. I don't think you can put a whole lot of validity in that anymore. All right. What was the biggest rewiring of how tech orgs worked in your time? Was it DevOps or is it going to be vibe coding?
Gene Kim (01:08:57):
Oh, it's going to be Vibe Coding by 100x. I'll re-see it. I mean, every shape of team, every process, just like in DevOps, is now widely sufficient and it's happening a hundred times faster and the impact is going to be 100 times faster. Oh, I have a friend who was recently quoted in the Business Incider for the way their organization is using AI created a little firestorm. And we're joking after the latest kind of PR crisis. He said, "Yeah, no one actually, you can never have imagined a PR firestorm about whether someone's using Kubernetes or not.
Nicky Pike (01:09:34):
" Right. True point. 100% true point. All right. You're sitting there with a developer that's a 20-year-old veteran developer or somebody that's never touched code in their life before. Who's going to surprise you more using AI?
Gene Kim (01:09:49):
I think both for different reasons. One is for many developers, every developer I've hung out with who's been experienced, I'm always learning from. Whether it's like someone said, "Are you using open code?" I was like, "Why? Why would I bother? I'm using Codex and CloudCode." He said, "Well, until you experience seeing two AIs argue in front of each other in the terminals, you really haven't experienced my coding." I'm like, "Okay." He was right. Junior coder. Oh my gosh. Met someone who's never coded before and he wrote an entire app to match patients and providers for people with handicap and disabilities. Wow. He understands the domain and he built it all by himself in clogged codes inside of a desktop app on a Windows machine. I mean, it was incredible. As you said, these citizen developers are building so many cool things that are going to make the world a better place.
(01:10:41):
I'm always full of awe and amazement.
Nicky Pike (01:10:43):
That was almost a trick question. Every time I do these rapid fires, I ask myself what I would answer the same question. And I actually came up with the same answer you did and it was kind of along the analogy. It teaches a person that's never picked up a paintbrush, they start making real art. If they already do art, now they're elevated to creating the Mona Lisa. So it's different for different reasons.
Gene Kim (01:11:03):
Yeah, 100%. All
Nicky Pike (01:11:05):
Right. Let's go ahead into the predictions. I kind of think I know what you're going to say on this, but I want you to put a number on it. Three years from now, what percentage of software inside of a typical big enterprise is going to be built by people who aren't developers? And does the Outer Loop catch up or does it stay behind the whole time?
Gene Kim (01:11:21):
I think there'll be 15 export software written by non-developers. And I think the
Nicky Pike (01:11:28):
Really - This is the internet, Jean. Once you say this, you can never get rid of it. So we'll be coming back.
Gene Kim (01:11:33):
I'm not really good at predicting the future, but I mean, I feel like that's possible. And the important stuff we'll be going through, developers will be involved. Things involving data, missions growing old stuff that involves testing. Yeah, you have to have professional developers still involved.
Nicky Pike (01:11:50):
Yep. Yeah. And I don't think we're going to see the catch up, what we're seeing with productivity and velocity right now. We're going to see, in my opinion, move into the Outer Loop. It has to if we're going to see the value out of it. Or we redefine what the Outer Loop looks like 100%.
Gene Kim (01:12:06):
I actually do believe in what the Outer Loop looks like has to change. All
Nicky Pike (01:12:11):
Right, buddy. You have been studying high performing orgs since before a lot of people that are watching this even started writing code. You've watched DevOps go from heresy to gospel. You are now watching the next thing rewrite the rules again, right? You're watching AI come in, we're rewriting the rules. You're in it, you're burning tokens, you're working with non-developers. After all of this, Gene Kim, what does it mean to you to be a coder?
Gene Kim (01:12:37):
I talk about this a lot to my kids and I still feel like they understand. They're 18, 16, and 16. It is literally conjuring up goals and aspirations I've had for 15 years that just materialized before my eyes. And that is a magical feeling where your goals, dreams, and aspirations are just coming true. I mean, part of in the last few weeks, what made it so magical is that I finally created these reliable ETL pipelines for all my screenshots, all my Zoom meetings, all my recordings, all my clean shot photos where I kind of capture moments of magic during coding. All my liked tweets, all the articles I wanted to read. I mean these numbers in the hundreds of thousands and all came to bear in two moments. I told you about one of them. I tried to find this article that I read, this tweak that I saw of how the invention of the spreadsheet upended the job market.
(01:13:31):
I looked for this for about five years and I actually looked for it while we were running the vibe coding book. And it didn't make it in because I couldn't find it. So after gluing all these ETL pipelines, I mean, as my test case, I'm like, "Can you find this tweet?" And they found it. I'm like, "Oh my gosh." And now I have the slides minus 400,000 plus 600,000 in terms of clerks versus accountants. The other thing that happened, I turned in my car lease back in the end of April and my wife was like, "How many miles were on it? " And I went into my open claw in a Slack channel and said, "There's a picture I took at the end of April of the odometer. Can you find it and tell me how many miles it was? " And it found us, it was 29,350 miles.
(01:14:17):
I have been dying for these moments and to do that for
Nicky Pike (01:14:20):
Every
Gene Kim (01:14:20):
YouTube I've watched, for every whatever. And I feel like it's now 90% done. And so I've dreamed about that for 10 plus years. And just like the screenshot podcast analyzer. It's now not something I do manually like I did back then, but it's something that the agent uses to recall anything that I've seen, heard, whatever, taking a picture of. And I cannot tell you how magical that is for me. And I built it and I'm so happy and it makes me so happy to be doing this every day. We live in an age of miracles and I've never had as much fun as I'm having now.Nothing's even close.
Nicky Pike (01:14:58):
Oh, I couldn't agree more. This is such an exciting time to be a part of tech, to watch not only what it's doing, but what the potential of it to do is. It's mind-boggling because I don't think we can see. And Jean, I wanted to thank you for coming on the podcast. As always, my friend, it is great to hear from you. It's great to talk to you. It's always a joy. Is there anything parting that you would like to say before we leave?
Gene Kim (01:15:21):
If you're interested in any of this stuff and you're facing these challenges in your organization, I promise you the best place to be at is going to be at the Enterprise AI Summit where we have people on the frontier who are working through these same problems. And one, there's no better place to learn about experience reports of how people are solving this problem. And as my friend Amy Willard from John Deere said, when you are in an era where nobody knows, the only way you can find out the answer is to do it yourself or know people who are doing it themselves. And so it's all about, I think one of the lessons in life I've had is it's all about who you know. And we're in one of those periods where more than ever, you have to be hanging out with the right people to, and ideally it's the best in the game.
(01:16:02):
So I look forward to hopefully seeing many of you there. And I'm so much looking forward to seeing your talk, Nikki, about -
Nicky Pike (01:16:07):
Thank you, sir.
Gene Kim (01:16:09):
I
Nicky Pike (01:16:09):
Appreciate that.
Gene Kim (01:16:10):
Breaking the bank on these token costs.
Nicky Pike (01:16:13):
Yep. I appreciate that. And I will reiterate, this is one of the things I love about the Enterprise IT Conference is that our Enterprise AI conferences, that this is people that are actually doing it. This isn't pontificating. This isn't, Hey, look at my demo of unicorn status that only happens in these conditions. This is talks between people that are actually getting in the weeds and making this stuff work and they give out the warts just like they give out the hugs. So make sure you come there. Last question, Gene, can we consider you a member of the [Dev]olution, my friend?
Gene Kim (01:16:44):
Oh my gosh. Yeah, cut me in. In fact, I'm already flashing the badge.
Nicky Pike (01:16:48):
That's right. I appreciate it. All right. Well, I appreciate you being on and thank you for taking the time and until next time,
Gene Kim (01:16:55):
Buddy. Right on, man. Thank you, Nikki.
Nicky Pike (01:16:57):
Thank you for listening to [Dev]olution. If you've got something for us to decode, let me know. You can message me, Nikki Pike on LinkedIn or join our Discord community and drop it there. And seriously, don't forget to subscribe. You do not want to miss what's next.