Subscribe
Share
Share
Embed
Candid conversations with the builders shaping the future of engineering.
Braintrust dives into the operational realities of running high-performing engineering organizations, from production readiness and migrations to AI adoption and operational excellence.
Hosted by Ganesh Datta, CTO & Co-founder of Cortex
Naveen Puvvula (00:00):
Architecture is one thing, which is what I want to touch. There are two key things, like architectural mindset and then the product mindset. I think I am trying to see how I can use the current agents and harnesses to bring both at the same time and lay out the detailed steps.
Ganesh Datta (00:26):
You're listening to Braintrust by Cortex, where we explore how engineering leaders blend AI, platforms, and culture to build high performing software teams. I'm your host, Ganesh Datta, CTO and co-founder of Cortex, an engineering operations platform designed to help organizations continuously improve their operational maturity and reduce developer friction. In each episode, we go deep with CTOs, VPs of engineering, and technical leaders who've been in the trenches navigating the tension between speed and quality, building reliability at scale, and figuring out how to lead through major platform shifts. Whether you're running a team of 10 or a thousand, this is your space to learn from people who've made the hard calls and live to talk about it. Hey, Naveen, thanks for joining the podcast. Very excited to have you on. Obviously, we've worked closely together in the past. I think your thoughts around AI specifically in the SDLC and how things are changing for agents, for humans, and for teams and organizations are really interesting and excited to talk about that today.
(01:33):
But before we dive in, I'd love to hand it over to you for a quick introduction and I'd love to hear a little bit about yourself.
Naveen Puvvula (01:39):
Yeah. Hi, Ganesh. Thanks for inviting me here. I've been closely following the success of Cortex over the years and congratulations for you and your team. My name is Naveen and I started as a embedded mobile engineer. I worked on Cortex pipelines, then built customer apps over a bunch of series of startups. And recently acted as a director with Life360 and headed infrastructure and platform tooling over the years and took care of a bunch of the ilities, I'll call reliability, scalability, availability, but with velocity and efficiency. And as of last week, I made a switch and I joined a startup called Planera as a IC leader, senior principal engineer, and super excited to be talking here. I
Ganesh Datta (02:37):
Would love to dive into that a bit more. Well, thanks for joining me on the podcast, first of all. I think your experience is quite fascinating and we're starting to see this in an industry a lot where senior engineer leaders are moving back into IC roles, even folks that have seen the gamut of things. Like you mentioned, you've seen everything from embedded mobile development all the way to, I never heard anyone call it the LTs, but I'm going to steal that now and use that more. But having been through that life cycle and managing large organizations with a variety of responsibilities, you made the decision to move back into granted a very senior IC role. Walk me through that. What prompted that decision? Obviously it's a very exciting time to be building, but why move back into an IC role today?What's your thought process there?
Naveen Puvvula (03:22):
A couple of things there. You mentioned this, it's an exciting time to be building. All my years I've been at startups and building a lot of exciting features and products that are useful to customers or people like us. And Life360 is where I got the opportunity to shift from zero to one and then from one to 10 scale. There are certain elements I thoroughly enjoyed as part of that comes with scaling, not just technology, but organizational dynamics, organizational growth. I did really enjoy those challenges, but when I took a break and thought for myself and see, hey, where do I enjoy most next? Take a look back. It's like, okay, I enjoy solving challenges and what's the best shot I can give it? That's the first aspect, get as much time to do that, A, and B, I also want to not just be a user of the AI tools and really dial down and understand and be part of the first phase is done, the models.
(04:35):
And the second phase to me is about the vertical AI use cases, how we make it useful. The developers have, it's pretty beaten down thing that okay, developers enjoy coding and using AI and all, but there's a big world out there. If you think of it, the construction space, I'm saying it's real scheduling, delays in scheduling, time and money, millions. So I wanted to actually jump into that and see, understand the AI, how it works and how it can solve. It's not just AI, it's how you use data, how you use harnesses and all that. So that's the exciting stuff. That
Ganesh Datta (05:19):
Makes sense. Before you made that decision, were you hands-on with AI coding agents yourself? Because to listeners, it sounds like a very drastic shift going from being a director to a senior principal engineer, very similar influence and scope in an organization, but you're much more accountable to the actual delivery of code itself. Did you get your hands dirty with coding agents leading up to this decision? Have you been more hands-on with the actual writing of software either at work or outside of work? What led to that decision?
Naveen Puvvula (05:50):
Not as much as I would've loved to, and that probably also prompted me as a factor, but I was also involved in AI adoption
(06:02):
In organization, not just engineering. So trying bunch of things in the leadership layer. As part of that, yes, definitely tried. I mean, I was doing AWS Q business prototype. It was one of the first takes on enterprise agents before back in 2024. So building that stuff with partnering and there's layers of my geeking out in terms of coding and PR, it's probably less than what I wish to, but more on the architecture side. There was an interesting earlier this year, we had some problem in our infrastructure, EKS infrastructure, and we've taken some time to solve it at scale challenge. And the team is working on that. And then I was obviously not at a coding implementation level due to my role, but I got a chance to dig in and add my own context to using the cloud codes and came up with a couple of architectural changes.
(07:20):
So it took me to the 80 to 90% of the thing, but I had to work with the tech lead there to iron out or filter out one of the solutions because there is additional context that's missing. So in that sense, it was fascinating. It's a huge complex architectural change. I stay at 90%. And then the last 10% you had to put the human in the loop. So that was one of the recent before I made the switch.
Ganesh Datta (07:54):
I'm guessing that'll be a big part of your new role in a senior principal capacity is shaping those architectural decisions. In my own experience, what I'm finding, especially with some of the new frontier models like Fable and whatnot, they're very good at following instructions once you get to a good point, but they're not great at architectural decisions yet. So if you can help shape the architecture in the early parts of the journey, it can go out and execute on that really well. It can review its own code. It can make sure that what it's implementing is up to par with the spec that you defined upfront, but the architectural component is still really important. Sometimes it'll make really silly decisions where if you can iterate on that very early on, you'll get to the right outcome. And over time, obviously the entire life cycle will get better, but I think we can have a huge impact.
(08:38):
The human in the loop I think can have a huge impact at the beginning part of the life cycle, which is, is this actually the right architecture before you let agents loose? Is that how you're thinking about the evolution of your own role, providing that kind of architectural guidance to the broader team and that then plays a multiplier effect on the team? How are you thinking about the evolution of your seniority level in the organization as an IC now with AI?
Naveen Puvvula (09:02):
100%. First of all, I kept my own self challenging that I should not code. I'll try to see how much, not personally coding, but let the agents do it. But as you mentioned, architecture is one thing, which is what I want to touch. There are two key things, like architectural mindset and then the product mindset. I think I am trying to see how I can use the current agents and harnesses to bring both at the same time and lay out the detailed steps. We all know, as you mentioned, the agents are very good at once you have clear instructions and all. I think the biggest challenges is about validation, judgment, and that comes from both product and engineering architecture. So as you can see, the organizations are kind of condensing SDLC probably pretty soon, old term PDLC is what we are hearing or already there.
(10:08):
And you see organizations evolving into EPD, engineering product and design. That's where I see. And in terms of my role, I think yes, bringing that perspective, judgment and other aspects that the lower level tasks or iterations for the agent and build a harness around that repeatable harness. I
Ganesh Datta (10:34):
Think what you're describing, you've written about this extensively as well, is this idea that it's not just about raw resources anymore. And to some degree it never was, but I think you could get away in the past with just throw more resources at the problem and you'll kind of get to a better state. But now, especially with AI, organizational bottlenecks become much more apparent than they were before.You could have a lot of resources and agents are giving us more resources, but bad organizational structure, bad context, bad design are barriers that you can't overcome even with throwing as much compute and tokens as you want is the problem. Is that how you're thinking about organizational structure, like bringing EPD more closer together? What does that mean when you say organization structure is more of a bottleneck than resources? What do you mean by that?
Naveen Puvvula (11:30):
Yeah, I can go multiple ways on this one. You probably heard recently read about the Uber's CTO coming up and laying out the experiment results of their AI investment. First, everybody heard about their AI build, but I think, I don't know how many heard about what the results were. It's interesting that they mentioned that it's the real results came when they paired AI engineer with actually the domain. It's not about just tokens or anything. Each team like sales or marketing or every, they have 30 or 50, I don't remember. So they have the 30, 50 parts and each part has an AI engineer as a domain expert. And the key thing was to unlocking that workflows by questioning the existing workflows and how AI can replace that. So I think that's one way to, which also brings to go to foundational principles, hey, what are we trying to do and go back and then throw technology at it versus trying to interject the technology AI into the existing practices.
(12:45):
So in terms of organizational challenges, I think we'll see a lot of that. The shift I can see a little bit.
Ganesh Datta (12:53):
Yeah. It's kind of this concept of, you mentioned this with the PDLC, even in the past, the reason we would front load product and design work was because if you do that work properly, then the actual development life cycle becomes a lot better because there's less thrash, less back and forth, you're building the right thing. And I think what you're describing with this kind of pod model with the domain expert and the AI engineer is basically going back to this idea of the problem or the job to be done. What are we actually trying to solve for? What is the end state that this team is trying to accomplish? And then applying AI to it versus like, oh, let's just throw agents at it and let agents figure it out. Even with humans, it's like, yes, humans have raw intelligence, but organizing people the right way, giving them the right context, making sure that they understand what the goals are, are the best way to get outcomes from humans.
(13:42):
It's the reality. And so why would we think that
(13:44):
It would be any different with agents? You want to make sure that they understand the domain, make sure they understand the goals, and then you can work backwards from, oh, actually here's a better way of implementing that. And like you're describing, it's almost like what's old is new. You describe this as going back to pure computer science concepts like retrieval and representation, organization. Have you seen that kind of apply in your own work? As you mentioned, you were kind of owning some of the AI rollout at Life360 and helping that implementation. What did you find worked well? Where did you apply some of these principles?
Naveen Puvvula (14:16):
I think we have to see different organizations based on their style or their level. I think what works for a startup doesn't work for maybe a middle organization like Life360 or it doesn't work like a bigger organization. I'm going to take a step back, give the context what I think, and then we can go into specifics. There are multiple factors here, org factors or org themes. One is you have a FOMO, and then other thing is as the companies grow big, you have an isolation pattern, or you have more multi-functionalities, you do take more things to do, and that breaks your organization into different things. And then you have all these some things like Amazon's two-pizza model, which is like, okay, you have this conflict or always a bit of a fight together. So all these things, they don't exist normally.
(15:24):
The outputs are different patterns and different approaches of the engineers. It's not that engineer's fault, it's just the organization is kind of sending those things. Some teams are fast, some teams are slow, and some teams are process oriented, some teams are more risk taking. So understanding all these things to me is a bit critical as part of the AI adoption or challenges. And then applying at different things. When I saw in my previous experience, understanding mobile versus cloud versus infrastructure, those three things are different and you can take different risks at different layers was one of the key things that helped. You have in a mobile iOS Android, so you have less repositories. The complexity is a little bit different. You have tooling, different tooling, which is you can use for testing ahead and all that stuff. And then when you come to infrastructure, okay, even the risk, when you ship the code, there is a delay also before it gets onto the app store or something like that.
(16:43):
And the mobile, that's there. And also the revert is also time lagging. You introduce a bug, it'll take a long time to fix the bug. The impact is bigger. On the cloud, on the infrastructure side, you take down one cluster during an upgrade, then yeah, you can fix it faster, but the impact is blast radius if many people feel. So in that sense, we took more hands on the approach, co-piloting in one area versus more autonomous in other area. And certain teams are more comfortable, so you could let them make them a part and let them do that. You need to be comfortable as leaders try all these things. Do
Ganesh Datta (17:29):
You think that's primarily for the adoption phase or do you think it's going to stay that way for the long term? And the reason I ask is, is it possible that coding agents can become an equalizer of sorts? For example, actually in both approaches, maybe things like feature flags or things that can make for iOS. I don't know if this is true for app development, but hey, if we could put things behind feature flags and maybe a rollback is feature flag and a rollback for cloud infrastructure as a rollback, but conceptually the systems or the modes of operation are the same. Can agents equalize that or do you think that the SDLC or the PDLC are always going to look different for those teams going forward?
Naveen Puvvula (18:13):
I think some companies are already doing it,
(18:16):
And I think I'm bullish on that happening pretty soon. I'll tie back to our old teams and how things. If you see the EPD, PDLC kind of thing, if you put the engineers, product and designers together, what's the obvious first thing that they will do? Engineers will build frameworks or AI tooling that let product people to test faster iterations, same thing for the designers. And by working with same thing, product people can iterate faster even in production, which already is happening in some companies. And then the same thing working with product people, the engineers have this, they are not blocked on product people to try different variations. I can make some changes and have all these experimentation frameworks that the product people use. So it's like you have a collective brain there intersecting circles and the tooling and repetitive work is all given to the agents and you can put the agent also as your partner, but I think the judgment and what needs to be tried, how fast to do and all that, I think that's where I feel it'll pretty soon happen.
Ganesh Datta (19:27):
Yeah. Do you think the distinction between these different orgs will state a lot of people, I think your story is unique in the sense that you went from mobile development to true cloud platform and infrastructure, but a lot of folks who are in mobile development specialize in that and they stay focused in that area or people who are in infra tend to specialize in that over time because there's such a large knowledge base you have to really build to be a very senior engineer in those areas. Do you think that's changing with this model now where you can kind of mix and match product and design and engineers of different varieties and throw them into a code base? Or because of things like judgment, do you think that experience in a specific domain area, even on the engineering side, is still equally important? Do you think that's changing or is that going to stay the same?
Naveen Puvvula (20:15):
I think every question we ask this year is all time based, time relevant. The answer changes next year or two years from now. So we are in this transition period. Let's just take a small example. Earlier we have a lot of white collar workers working on machines and tools and all that. Now if you go to factory, you don't see all that stuff. You have a special technician. Pros and cons, do we expect the technician to know everything about that machine and what happens when something goes wrong? So there are those people, but they may not be available right near the machine or. I'm bringing this because my wife is a mechanical engineer and works in a semiconductor tooling. So in that sense, I think eventually that's what happens.
(21:13):
If you talk about mobile cloud, it's just that you can get maximum mobile exposure or knowledge of a database creation, complicated locks and patterns with the agent. So that's a powerful combination. So I'm a mobile, but I can get the 80% of the knowledge of the. But your brain is not tuned to behave in the scalable systems and all, right? You can read, you can understand that. So there will be some percentage of specialist need knowledge. Is it like every team needs it or is it like a bunch of teams have one or is it like the whole engineering has one? We'll see. Another example is a DBA example. 10 years back or 15 years back, I think DBAs were very pretty important. Six months back, I was trying to hire a DBA and it was very hard. It's not many people are there with that skill at that seniority level.
(22:19):
That's without AI.
Ganesh Datta (22:23):
I love that analogy. I use the factory analogy a lot in the drive framework. I don't know if you had a chance to read it, but as we're moving towards software factories, this concept of. A lot of the things that I think manufacturing learned over the years are things that we can apply in software engineering. We went from the initial humans do everything by hand to humans are steering machines that then do things to humans step away from a lot of the machinery, but design the machinery to optimizing the floor. The abstraction layers keep changing. And so I think to that example, when you think about whether it's semiconductors or car manufacturing, you have mechanical engineers and then you have industrial engineers and two very distinct skill sets. You have somebody's designing the headlights. There are people who especially, they spend their whole careers designing headlights and how to really build great headlights for cars. But then they may not necessarily be experts in, okay, now that I've designed it, how do I go and tool an entire factory go to produce those things at scale?
(23:28):
That's a very different skill set. And so it may not be the person designing the headlights as the one on the floor building the headlights, but you have this abstraction layer of people specializing in, let me design the headlight, industrial engineers will figure out how to optimize and tool the factory floor to actually produce them. Then you have operational excellence experts who are like, okay, then how do we continually optimize the floor to produce more of those? So I do agree with you. I think that's where we're heading where it's not that everybody needs to be a specialist anymore. I think in software engineering, there's kind of this expectation that most engineers will have quite a bit of breath. Most engineers should understand database locking and transactions and all this kind of stuff along with testing and QA and the entire skillset is very broad.
(24:10):
And then you have a handful of very senior engineers who are very good at distributed systems or whatnot. And I think maybe that'll shift over time where a handful of developers will become specialists in certain areas and they're the ones designing the architecture. In the factory analogy, again, it's like somebody needs to design the Corolla. It's like the factory is not going to invent the Corolla and build it and ship it. Somebody's designing the Corolla, somebody's figuring out how to manufacture it, and then the factory manufactures it. In the same way, somebody's going to design the specs and the architecture for the systems, and they may use AI to help them with that, but at least for the foreseeable future, that's where human input will be very, very important. And then you have people who specialize in, okay, well, I know how to build a factory that takes those specifications and can validate it and can make sure that the factory is turning that architecture into something real and can unit test it and whatnot.
(25:02):
And then you have people who can optimize a factory for cost and quality over time. So I totally agree with you. I think specialization is not necessarily going to go away, but it's going to just stratify in different ways over time. It's going to be hard for us to reason about necessarily today. I
Naveen Puvvula (25:15):
Mean, there are engineers or product people who can do end-to-end. There are.
Ganesh Datta (25:19):
Yeah, absolutely.
Naveen Puvvula (25:21):
We have Elon Musk, and there are a lot of people like that. They will be there and they will be absolutely amplifying their productivity with AI. That's like a Swiss army knife for that. But it's not meaning that regular stuff is not needed. Yeah. Exactly.
Ganesh Datta (25:44):
On a similar note, you've talked about with agents, the way we think about our teams is changing, like you said now. And a lot of the industry, I think for worse, is talking about agents as replacing human headcount. And what we're just talking about, it's more like there's going to be specialization and the types of work we do I think might change, but it's not that those roles are going to go away. Especially software engineering, we are going to be building those factories and architect. Those things are not going away. But at the same time, we talk about agents as producing PRs and change and things like that, but you've talked about agents as empowering teams, not replacing teams entirely. What is the distinction there? Why do you say that it's focused on empowering teams versus replacing headcount as I think a lot of gloom and doom storytelling tends to focus on?
Naveen Puvvula (26:34):
I go back to one small thing. This is back from 2016, I think, Google IO, 2016, and one of the AI head came and presented UI leads AI. That still kind of, I think, makes sense now. It's basically customer value or user experience, user interface is what is powered by AI. So that's the core value. I mean, businesses, teams, organizations, everything is there to deliver a value. We can call any number of things, and I think that's a big factor I believe in. Once that is clear, once all the teams, everyone is aligned to that, now the headcount, team compositions, all those things doesn't matter when everything is aligned. I think over the last four or five years or historically, I think that's one of those problems. When you don't have visibility from top to bottom or across and align, that's when you create boundaries and different parameters because you can't see the initiative intent or outcome and then you can't measure always to course correct.
(27:51):
So you build other stuff. So that's where I said headcount doesn't matter in that sense. That's not important. So if we can use AI to solve these problems, the core one and the leadership invests in the alignment, and I think at that point, all your focus is kind of taking it to, okay, what am I trying to initiatives? And a couple of things you cover in the drive framework, that's a very key thing. A lot of organizations, I think, struggle in that. A particular leader or few particular leaders have that, does the same thing is aligned with bunch of other engineers or other people. If we can solve that, then I think just AI becomes a facilitator and that's kind of orchestration. You focus on the right problems, focus on the outcome, not activities. It's not about the PRs. It's about what user can consume the value.
(28:49):
Okay, I'm giving you 10 things. Have you thought about whether they're needed? Is he asking? Yes, you have newfound superpowers that you can produce more, but is the consumer ready to pay? Is the consumer has the complexity to consume that? So I think that's where the amplification should be. That makes
Ganesh Datta (29:09):
Sense. I mean, like you were describing earlier with the Uber example, initially it was like shock, like, oh my God, we blew away all of our tokens. And then the follow-up was, no, no, actually the value is there. We just had to reorient around it in a different way, and now we're seeing the value. So it was maybe a misalignment in what are the outcomes we care about. It was like, "Hey, everyone just go use AI." And yeah, people will use it and you blow through your budget, but it's when you focus on what does this particular domain need, what problems do they have, and how do we apply AI to that particular set of problems, that's alignment of organization and humans to the problems, and you're then accelerating the outcomes that those teams want to achieve. It's not, "Oh, we're replacing team X with Y." It's, no, we're taking domain expertise and we're amplifying certain parts of it with AI.
(29:56):
So like you said, it's the outcome oriented approach there.
Naveen Puvvula (30:00):
I'll give you another example for my new company startup, construction technology. So it is real industry. You're building a bridge, you're building a data center, huge hundreds of millions schedules, multiple subcontractors, materials miss in software, you miss a schedule. Okay, we release in two days later. But there's real impact here, liabilities and all that stuff. So how do you say, "Hey, I'm going to use AI here." There's a big trust factor there. I think that's the next challenge of accountability framework or not just convincing your own employees or teammates, it's about convincing the customers. Okay, I powered the software with AI. Can I use it? How can I feel confident? I built it so I'm confident, but how can you translate the trust to the consumer? And I think that a lot of vertical AI use cases is what we'll see. It's easy to convince developers because they're builders, right?
Ganesh Datta (31:08):
Yeah. And it's a very verifiable outcome.
(31:12):
Especially in an existing code base, it's like, okay, well I have tests, I wrote some code, it passes, very easy to verify. In a greenfield area, it's even easier. I run something, I see it in the browser, whatever, it's working very easy. But something like this where there's most of the process exists for some historical reason. It's like human safety, it's physical world things, like you said, there's liabilities, there's regulation. It's much harder to just replace certain things with AI. So how do you think about building that trust? I think there's parallels to internal work outside of engineering, getting teams to adopt AI, a lot of it is around trust. How are you thinking about that on either end, both within the organization, getting non-engineering teams to think about AI and trust it? And I think it's parallels to the outside world. How do you think about that?
Naveen Puvvula (32:02):
I think it starts with two things. One, as you covered, non-engineering teams to start trusting it first because they're the ones who's talking to the customers. And the other crucial thing I observed is engineering teams to understand the customers or the value they're generating. It is one of the big lessons for me being a director or engineering leader. A lot of engineers are focused on what they do.
(32:31):
There's different motivations. The technology is awesome or there's a problem I want to solve. It's a lot of time needs spending on alignment and whether it's the right alignment, bringing, hey, how do you think express, how do you think this is valuable to the customer? That's a big thing. So I think that's another thing. Use AI. Okay, why do you think it adds value to the customer? Engineers should be exposed to that. Those are the two things, very high level, but I think that's a model shift. That's what I was saying, accountability framework. I think maybe if I have to bet on something, there's some investment that will happen as part of the AI that we have not done with software, general SaaS and all that stuff. This is something which is impacting real lives and real things. So when you bring it there, then it's not hidden somewhere.
(33:29):
It's on your face. So you have to create education. You have to explain people why they should trust. I don't know exactly how, but I think that's starting to come up, right?
Ganesh Datta (33:44):
100%. Yeah. Especially as, like you said, it goes into the physical world. The bar is much, much higher.
Naveen Puvvula (33:51):
And that exposes some of the things that we do under the black box as software industry or any other industry. Now if you have to put an accountability framework, then you're exposing some of those things, which is not comfortable.
Ganesh Datta (34:03):
Exactly. Especially when it's non-deterministic and we've tried our hardest to make most software as deterministic as possible to whatever degree is possible. You talked about this for internal agents as well in the past, this idea that we're treating agents, we talk about it kind of like headcount, but
(34:21):
Can you evaluate it the same way? Accountability frameworks I think are in that direction when a lot of our token spend is going towards agents now and we're having those conversations of, do we hire a few more people? Do we give our existing developers and teams more tokens? There's this conversations that's happening. You've mentioned performance reviews. We know how to think about the performance of a contractor versus somebody on our team, but we're now giving tasks to these agents. How do you think about performance and accountability for our own internal agents? Do they get reviews the same way? How do you think about validating like, yes, we're getting value for the tokens we're spending or it is meeting our expectations for the things that it must be doing both internally or externally?
Naveen Puvvula (35:08):
Multiple approaches here. And I think again, time relevant when you ask the question. I think of it like, do we do performance reviews? I'll go back for an example of the manufacturing industry. We don't review machines or tools. We review humans like what you do, outcome, output. I think that's one way to look at it. Okay, if you want to go further, how are you able to articulate or orchestrate those agents? Do you have clarity of what you want? Are you able to. I'm not talking about autonomous agents here, human in the loops still. So that is the skill you can evaluate as you evaluate or review your employers. Have you provided the right guardrails? Did you give enough evaluation or did you let it figure out? Okay, give a question, go figure it. So there is that aspect, but if you can, again, the other factor here is the organization level.
(36:21):
I have a friend who started a startup and I was asking, are you worried about tokens and all that? No, I'm not worried. It's like if it means thousand, if it means 10,000, it's okay as long because I need to compete, I need to get value out.
(36:36):
But when you have 500 people or a thousand people at scale, obviously that becomes a number in the public results. Now you're talking. Even in that you have to segregate. I was talking with Braintrust meet and greet and one of the CTOs mentioning, okay, for a greenfield, I don't have that much restriction on the budget, but if it's regular engineering initiative evolution kind of thing, then I want to be looking at and managing the budget of the tokens. So there's multiple things here. I think historical industry, if you can communicate, I'm delivering this, I need this, then nobody's going to question. I think it's applied to the agent spending or a team with agents to me, as long as you can prove. You mentioned we trust contractors, how do you trust? You build trust and then you start trusting over time. And also you don't trust on everything, you trust on what we know already.
(37:48):
Okay, this project succeeded with five people. That's the factor that you take. Now we are comfortably saying, okay, I give this to Deloitte or PwC, they'll give this value. And then you have some trust because historical, we don't have that historical with agentic. So we have to build that.
(38:06):
And I think that might feed into the accountability framework that I was telling, right?
Ganesh Datta (38:11):
I think that's exactly right. Yeah. I do think there's this open question of right now, the music hasn't stopped yet with tokens where in the past, if you had a very large backlog, if you're a team and you have a whole bunch of medium and low severity things sitting in the backlog, you would have a conversation about the budget, like do we need another two developers? Is it okay that we're not doing these things? But right now, because everyone's being told to use tokens, you're going to build agents that go and ship all those things on the backlog and suddenly it's like, oh, we need these tokens, but did we actually need to spend those things? Are we actually getting the value from those medium and lows? Because when it was humans, we said, actually we don't need the additional headcount. We can't afford three more developers.
(38:51):
That's the trade off. But right now it's here's $100,000 in tokens, go spend it. Okay, let me clear my backlog.
Naveen Puvvula (38:56):
Maybe a question to you. Let's say we have P0, P1, P2 bugs, P3 bugs. So let's say we have 100 P3 bugs over the last two years and 50 P2 bugs. And so far we have been caring about P0 and P1 only, that's the time we have. Now because the tokens are there, are we saying that just go ahead and fix all the P2 and P3? Yeah.
Ganesh Datta (39:22):
This is what the drive framework is trying to answer, which is you cannot answer that question until you map out the customer outcome. Like were saying, you had to start with the outcomes. If your customers are able to see value and you're meeting all of your SLOs, then you have to have a real conversation. Are those P2s and P3s worth the token spend? The same conversation we were having with human capacity, the same thing is true with token spend. And if anything, over time, it's going to become more of a blend.
(39:50):
Just because it's tokens doesn't mean we can go fix all of our reliability problems. In the past, tech debt versus product functionality was a trade off conversation. It was like, hey, tech debt is getting to a point where we need to pay it back. It was never tech debt is just a thing, it sits around or we're not doing it because we don't have capacity. Engineers would say that obviously because we want to fix that tech debt, but the reality was it was debt. Eventually you'll pay it back once things started getting bad, but it was a trade off conversation. And now I think a lot of the industry's talking about it is like, oh, we're never going to have tech debt because we can just have agents fixing it. I don't think that's true. Tech debt is you have finite resources, whether it's humans or tokens.
(40:27):
Eventually you're going to have to have a conversation of this $500 of tokens, do I want it to ship a feature or do I want it to fix a P2 bug?
(40:35):
What is the thing that we want to do? And how do you answer that question? You answer that question by looking at the health of the organization, the degradation of your customer outcomes or the delivery of customer outcomes. And then you say, okay, well, based on where we are, we are going to spend 300 of those tokens on bugs and reliability issues and $200 of those tokens on customer features and that's the split and it might change over time. I think right now we're just saying, oh yeah, we'll fix all the P2s and P3s. Even if
Naveen Puvvula (41:02):
You're willing to put all the tokens, the question becomes, oh, should I fix that or should I do something else?
Ganesh Datta (41:08):
Exactly.
Naveen Puvvula (41:09):
Do I want to put in new features, send those tokens? Exactly. It's the same thing, right?
Ganesh Datta (41:13):
Exactly. And so I don't think that a lot of the industry has accepted the reality yet. I think tokens are free where a lot of organizations, we want to spend tokens. Everybody's please spend tokens. And so everyone is doing everything. They all want to ship features. Let's automate code reviews on everything. Let's ship P2s and P3s and tech debt and all this stuff. But eventually the music will stop. And so the question is, do you have the framework to pause and say,
(41:38):
What do we actually want to spend our tokens on? Do we have that accountability framework? Do we understand the outcomes we're trying to drive towards? I think the organizations that can spend time today thinking about what are the outcomes we care about? How do we measure those outcomes? Like you were saying, you have to start with the outcomes and then work backwards from there are going to find it a lot easier when the CFO comes down and says, sorry, we're going to have to put a pause on spend past a certain amount. The organizations that have done that work I think are going to get a lot more, it's going to be a lot easier for them to deal with that.
Naveen Puvvula (42:07):
Yeah. I think this phase also will go away. Just trying parallels back when I started on the mobile first video recorder, it was 12 FPS on a QSIF, which is called 176 by 144 pixels. And Nokia problem, I think Nokia wanted 15. Our company couldn't, 12 is the best at that time and that's okay. And there was a lot of spend on that for the first iteration, then you increase the resolution Codex and all. After a few years, everything became commoditized. Nobody works on Codex because now this pendulum switched, it's commoditized, it's not expensive, it's there. So that's a different focus area. So I think right now we are in the very first phases of it. And here all the tokens will be very important, right? How you spend, how do you manage, which ones you use and all that. But there will be a point where it'll be commoditized, where it'll be acceptable utility pricing kind of thing.
(43:17):
At that point, I think things will change.
Ganesh Datta (43:19):
Yeah.
Naveen Puvvula (43:19):
Yeah,
Ganesh Datta (43:20):
Exactly. I know we're almost at time, so maybe a last question for you, going back to the early part of the conversation, for anyone who was in a senior injury leadership position like yourself who's contemplating going back into an IC role and getting their hands dirty with Codegan, do you have any advice for them on how to think about it, how to make the leap, how to decide if that's the right move for them? Any advice for folks who might be in a similar boat?
Naveen Puvvula (43:46):
I don't think I'm an expert on that, but I can share some framework analogy that I used and if that's useful. I've worked in multiple startups. I started a few things, didn't work out, then started something and then the Big Apples and Googles came and kind of put your steel or whatever. So what I would say is that what do you want? If you want to really enjoy the technology or what's the thing that you value? And obviously there's no answers right now with how AI is shaping the industry. It's much easier than any time. At the same time, it's also very harder. Anybody can do that.
(44:31):
So all these things are there. Nobody knows answers. If you enjoy, if you're risk taking type, take the plunge. Nothing will happen. You learn some things. If that's not your cup of tea, obviously, I mean, there's other side where you are a lot of opportunities too. So I think just use that framework. It's about what you love, what you want to, and whether you want to take that jump and do it or not. One thing I'll tell is it's a switch you need to be prepared for. A quarterback, I was leading five initiatives, critical initiatives per day and kind of switching my brain at that level, sometimes going deep. And every day you have to talking to partners, talking to engineers, talking to the rest of the company, all that stuff. When you become an IC, then a lot of that context switching goes away that your brain muscles are not, you need to retrain.
(45:36):
So it's fun, but it's one small fun observation. So there's a lot of all these things. Again, I think everybody should do what they love.
Ganesh Datta (45:50):
I love what you said about what's the downside? The downside is that you learn something new, you build different muscles or train different muscles and you can always find a new way. And I do think that now more than ever before, it is important for managers to get their hands dirty and understand. The SDLC is changing a lot. And if the management role is about giving people leverage and helping your teams, individuals grow, helping your teams accomplish more than they would've without your guidance, how can you do that if you don't understand how much the SDLC has changed? And so whether you're shifting to IC role or whether you're staying as a manager, I do think getting your hands dirty and learning, really getting an intuition for how AI is changing the day-to-day work is really important because how else can you give your teams that much leverage?
Naveen Puvvula (46:39):
One thing I want to add, there is no not doing work, right? Whether you are still in the leadership or an IC, you will be because I think the middle management and all will be gone a lot, right? Or minimized, you can change. It's changing. So you still are supposed. So I think it's choosing between, hey, do you want to do both at the same time or do you want to fully do it?
Ganesh Datta (47:04):
And some people might say both. Some people are going to say one. It always depends. Naveen, thanks so much for joining me on the podcast. This was awesome to have you on after. It was a long time coming.
Naveen Puvvula (47:12):
Thanks, Ganesh.
Ganesh Datta (47:19):
Thanks so much for listening to this episode of Braintrust. If this resonated with you, do me a favor. Share it with another engineering leader who's wrestling with these same challenges. And if you want to continue the conversation or learn more about how we're thinking about engineering operations platforms at Cortex, reach out to us at cortex.io. Thanks for listening and we'll catch you on the next one.