Subscribe
Share
Share
Embed
Candid conversations with the builders shaping the future of engineering.
Braintrust dives into the operational realities of running high-performing engineering organizations, from production readiness and migrations to AI adoption and operational excellence.
Hosted by Ganesh Datta, CTO & Co-founder of Cortex
Tamar Bercovici (00:00):
I remember sitting down to try to write my self-evaluation the first time after becoming a manager and I was like, shoot, well, what did I do versus what did they do? And you almost feel like it's hard to put a finger on exactly what your role is. But then when you take a step back from that and you understand, okay, I'm responsible for this team, how do I set this team up for success? Part of that is individual coaching and management of the people and ensuring that I have the right people on the team. Part of it is also ensuring that we're focusing on the right things, that we have the right roadmap, that we're working well cross-functionally, that we understand the context that's around us, that we're using the processes and tools in the right way. And that helps you see, okay, my role versus the individuals.
Ganesh Datta (00:46):
You're listening to Braintrust by Cortex, where we explore how engineering leaders blend AI, platforms, and culture to build high-performing software teams. I'm your host, Ganesh Datta, CTO and co-founder of Cortex, an engineering operations platform designed to help organizations continuously improve their operational maturity and reduce developer friction. In each episode, we go deep with CTOs, VPs of engineering, and technical leaders who've been in the trenches, navigating the tension between speed and quality, building reliability at scale, and figuring out how to lead through major platform shifts. Whether you're running a team of 10 or a thousand, this is your space to learn from people who've made the hard calls and live to talk about it. Today with me, I have Tamar, who is an engineering leader at Box. Thanks for coming on the podcast.
Tamar Bercovici (01:42):
Thanks for having me.
Ganesh Datta (01:43):
I would love to hear a little bit about yourself, if you can introduce yourself to the listeners.
Tamar Bercovici (01:46):
Yeah, of course. So my name is Tamar Bercovici. I'm a vice president on the engineering team where I lead our core platform, which is I acknowledge a highly ambiguous name. Naming is hard, right?
Ganesh Datta (01:58):
Very
Tamar Bercovici (01:59):
Hard. But at Box, what we mean by that is it's almost like the foundation of the Box product. So it's a lot of our big data stores for customer data, like storage infrastructure, metadata, search index, et cetera. And it's also where we build the core behaviors of the platform that kind of govern how things work within Box. As of the past couple of years, a focus on building in AI capabilities woven throughout that. So yeah, it's a fun team to lead.
Ganesh Datta (02:30):
Sounds like a very important part of the platform. But yeah, it's interesting. Is it safe to say that the way you think about your organization is kind of exposing capabilities that the rest of the organization stitches together to build products? Or is your organization also responsible for specific product areas or is it more like the core, truly the core?
Tamar Bercovici (02:48):
Well, here and there we have product surface area as well, but we are mostly backend focused. Yes, we build capabilities that other teams leverage. I would say the difference is if you think of the box product, it has a lot of different capabilities and sort of parts of the product, but they all coexist in one application or in one platform. And so the way that we create that is by having all of those experiences be built on top of or as extensions of a common backend. So to a degree, it means that a lot of the teams don't have a choice as to whether they ... If they want their object to show up in the search bar as a result, they have to integrate into our platform. But that means that we have a huge responsibility to ensure that we're enabling the types of patterns and capabilities that they need, especially as we're evolving the product over time.
Ganesh Datta (03:44):
That makes a lot of sense. I would love to double click into that in just a second, but you've had a very unique career, especially when it comes to Silicon Valley tech companies. You've been here for a very, very long time in tech standards. So we'd love to hear about your time at Box. You started off as a senior IC and you've gone into engineering leadership and clearly own a very big part of the Box platform. What was that like? What feedback or advice do you have for Senior IC 15 years ago and how has that changed?
Tamar Bercovici (04:14):
Yeah, it's actually going to be 15 years in just a couple of days. So sort of a milestone in its own right. When I joined Box, I was looking to join a smaller company because I like that sort of dynamic environment where you figure out what role you're going to play. Actually, Box was on the bigger side of what I was looking for. The company was at around 130 people, but only 30 in engineering. So I figured..
Ganesh Datta (04:40):
Oh wow!
Tamar Bercovici (04:41):
..I'd still know everyone, but it was still a much earlier phase. And so I definitely did not join with the notion of myself sticking around for this long or frankly, what the path for the company would be. But I liked that we were tackling a real problem. The product was real. It wasn't fluff. I liked that Aaron, our CEO, had a vision for the company, which I think, especially when you're joining something earlier stage, you want to make sure that you have someone that you want to follow. And everyone I talked to felt like a good person. It was like a good group of people to be a part of. And it's interesting that even 15 years later and how much we've grown in many ways, we have an even more ambitious vision for the types of things that we want to enable our customers to do with our product and our platform.
(05:31):
Aaron continues to be the type of CEO that pushes us forward and is still a good group of people. So in that sense, it's just always been fun and interesting to stick around. Definitely the role has changed a lot, but both my own individual role, but also I think to a degree, the phase that we're in as a company. So it doesn't quite feel like I've been doing the same thing for 15 years. And I think that's actually important. I think if you want to continue to grow and develop in your career, you have to put yourself in an environment where you are interested in pushing yourself to learn and grow, and you have the opportunity and the support for learning and growth. And you can't, at least I can't plan out so many steps into the future. There's something, I think, naturally opportunistic about career paths as things change, as opportunities emerge, as challenges emerge, what are you going to be able to take on and grow into?
(06:35):
But a lot of that is just putting yourself in a position to continuously grow and learn new skills.
Ganesh Datta (06:41):
Yeah. Yeah, I love that. I think what's really unique about your tenure at Box is, you've obviously risen through the ranks, have been in very different scopes of management responsibility. What in your perspective has changed over the years now running the entire organization versus your first time being a manager? What are you thinking about differently? And obviously the company has grown and that itself brings a different set of challenges, but at its core, if you could distill the differences in those two roles.
Tamar Bercovici (07:09):
Maybe I'll start with what's the same. Yeah, that's great. I think every person in the organization at any level
(07:15):
Ultimately has been entrusted with a scope where if you're an individual contributor, that scope is yourself, your time, your energy, your capabilities. If you're leading a team, then it's also sort of that team and the services that the team owns, the resources. So you're entrusted with a scope and what you're held accountable for is delivering impact. And ultimately that's what matters. Now, when you start thinking about delivering impact and delivering impact over the long run over time, then that pulls in a lot of the other good things that we think about for good teams. You need to have a ambitious vision that you're excited about. You need to understand it so that you can make good choices. You need to have an environment that facilitates good work. There's a lot. If you're managing a team, it needs to be a healthy team where all the individuals have those same characteristics.
(08:09):
But ultimately, it's all in service of delivering impact. Now, I think when you shift from individual contributor to manager, it's actually a really, it's one of the more complicated changes. It's a very different role. It's a very different role. You're still delivering impact. You accountable for an even bigger scope and you're still expected to deliver impact, but you almost need to reframe your relationship to how you do that because it's no longer what you hands-on keyboard are doing, but what your team is doing. And I think I remember sitting down to try to write my self-evaluation the first time after becoming a manager and I was like, shoot, well, what did I do versus what did they do? And you almost feel like it's hard to put a finger on exactly what your role is. But then...
Ganesh Datta (08:56):
Yeah
Tamar Bercovici (08:56):
...When you take a step back from that and you understand, okay, I'm responsible for this team, how do I set this team up for success? Part of that is individual coaching and management of the people and ensuring that I have the right people on the team. Part of it is also ensuring that we're focusing on the right things, that we have the right roadmap, that we're working well cross-functionally, that we understand the context that's around us, that we're using the processes and tools in the right way. And that helps you see, okay, my role versus the individual. So I think that was a kind of shift number one. And then for me, I see senior manager was more pro mode of that. So it's not so much a different role. It's more, you now have more experience. You can manage a more complicated team. You can start maybe managing another manager because you know the role well enough so you can sort of coach and manage through them.
(09:47):
And then when you get to the director level, that's more about starting to take a strategic lens. So understanding where your organization fits into the broader org, what the broader goals are, and how you can maximize your team's contribution to those goals, which could require how you collaborate cross-functionally, how you set the broader longer term roadmap for the team, how you frame, how you structure your organization. There's a lot that goes into that, but it's a lot more about being in the driver's seat from a strategy perspective. And then again, senior director is pro mode of that. And I think VP was again a bit of a shift. It took me a ... You move from like, okay, I have this important scope and I'm driving and I'm figuring out this strategy and I'm seeing where we're going to. Okay, well now I have a slightly larger organization and I have actually really capable senior leaders driving those domains.
(10:49):
What's my role in all of this? And again, it's a little bit like, how does my organization fit into the broader context? How do we maximize our impact as an overall group? So how do I build a healthy collaborative management team? What processes do I set for our organization that sort of facilitate good execution? And yes, what are the sort of strategy opportunities that maybe span a broader scope or how do I need to think further out to set things up? But so in some sense, there's a through line of understanding, given where you are, what's the broader group trying to accomplish and how can you help accelerate that? But then the sort of scope at which you're looking at, the type of tools at your disposal, the type of challenges you need to solve, they obviously shift quite a bit as you go through those levels.
Ganesh Datta (11:47):
I have a hypothesis that maybe the jump from manager to director is one of the harder ones because even as a manager for the first time, while it's a very different role, you can still, not that you're writing code, but it's very much just a different lens of the same problem. It's like, okay, I have a team and we're working on tickets and how well can we do these and how do I improve that leverage? Versus as a director, you're suddenly not that involved in the day-to-day versus you're trying to get your managers to do that for their teams. And so is that a fair hypothesis? Did you see that yourself?
Tamar Bercovici (12:20):
Yes and no. I think, well, first off, yes, as you become more senior in an organization, you accumulate more distance from the code base and that can absolutely be a challenge. In my opinion though, successful engineering management at all levels is a highly technical role. And if you don't at the right level understand what your team is trying to accomplish and what challenges and opportunity, how can you build a strategic roadmap for an organization without deeply understanding the technology and deeply understanding the challenges and the opportunities, especially in a technology space like today that is so dynamic, you have to know that. You can't delegate that away. There is a challenge of knowing how to make decisions and how to guide the team with broader
(13:12):
Details and less depth. That's definitely a different mode of operating. I personally like that, but it is an adjustment. Even I had to find how to not get sucked into the weeds on everything, but also how to still let myself dive into the weeds when that was the right thing to do. If you can't do that and you can't dive in where that's required, then you're also doing a disservice to the team. I think one of the things that's easier though about managing other managers is that they have the same role that you do, but at a different level, as opposed to managing engineers where there's more of a distinction and actually they don't always understand what you're doing and why. So there is this sort of element where when you shift into managing, even though actually, and I like that we do this at Box, at all levels, we have both people managers and individual contributors at a more senior level reporting in.
(14:08):
So not all of my direct reports actually manage people. Some of them are architects, but just that sort of sense of managing people that are more senior, that have more of that themselves, that sort of broad organizational view made our roles closer to each other in some way. And I do find that to be a benefit. But to me, the biggest challenge with stepping into that director role is just understanding that you're no longer working around problems. It's on you to actually- But comes with you. Yeah. I think a lot of very successful managers is like, "Okay, here are the constraints of the environment that my team and I have been put in, and I'm just going to go optimize those. I'm going to make this the best experience that it can be. " But if you're at a more senior level in the organization and you're seeing problems and you're not addressing them, then well, who's going to address it for you?
(15:05):
And so I think that sort of understanding what you own and is within your control can be a little disconcerting initially because it was like, oh, whoops, wait, I have to actually do something about this. But it can also be very empowering when you realize that you can fix these problems for yourself and not just live with them.
(15:23):
So to me, when I see someone sort of make that leap of understanding that they can reshape reality around themselves more so than they could before, and hence create a more conducive environment for their team, that's a really exciting moment, I think, when someone unlocks that capability.
Ganesh Datta (15:42):
I like that framing a lot. And I think it's, like you said, as a manager, even as an IC, you're very much focused on operating within the constraints and making the most of them. But once you're at a level higher than that, you're thinking about why do these constraints exist? What are the set of constraints that I should be providing that the team should operate within? And it is a very difficult transition to think about that, but I personally find that more fun and interesting, and I'm sure you do as well. You made a comment. Actually, there's two things you said that I thought was interesting. One that very senior ICs report in at the same level. And I think I love that because one of the things I think you see a lot of organizations struggle with is showing with action that the parallel IC path is truly an equal path to the management track.
(16:29):
And I think a lot of organizations struggle with that. And reporting structure is maybe one of the cheapest ways of proving that it's like, "Hey, all these people report into me, look at them, they're directors also have senior IC one, two, three." You can imagine they are functionally at the same level. And I really love that. Is that the intention behind-
Tamar Bercovici (16:46):
Yes, I do absolutely think that there's a visibility aspect to it, but it's actually material to their role as well. I'm actually very proud of the individual contributor track that we've built over the years. By the way, when I joined Box, we had no titles in engineering whatsoever, so I didn't actually even join as a senior IC. I just joined as an engineer. And then one day we announced that, oh no, we have titles. And we only had I think three levels or something to start with. And I remember a very awkward subsequent one-on-one with my manager then where we just, we had a normal one-on-one. He didn't say anything about this. And at the end I was like, so where are you slotting me in? Yeah. So over the years, as we also had more of a need for it, you need to have enough scope and breadth of technical complexity and a size of a team to merit a more senior, anyone at any level, whether people management or individual contributors type architect.
(17:43):
But over the years, we've built up that track. And our principal architects, they are expected to not just deeply understand their domain, but actually work cross-functionally and help identify challenges and opportunities and really help us drive our cohesive technical roadmap. And that requires a partnership between them and their sort of management counterparts.
(18:08):
You need both sides and we want them to be working collaboratively together. And then they also need the context. A lot of what I'm sharing in my leadership team meetings is that sort of business context, organizational context, focus areas, the conversations we're having are understanding what's going on within our organization. And it's critical for them to have that insight and they have a critical perspective and insight to share as a part of that context. So that's not something unique just to me and my team. That's how we run our engineering organization. I think it's been very, very successful for us.
Ganesh Datta (18:43):
You made a point earlier as well that the most effective engineering leaders, whether it's managers or senior leaders are deeply in touch with the technical aspects of the job and they understand not necessarily the code base, but the architecture and how things fit together. I would love to hear a little bit more about how you have built that over the years, especially as Box has gotten more and more complex and your scope has grown complex. How do you do that today? And I guess maybe a corollary to that, the senior ICs are reporting to you, are they part of a strategy? Do you see them as a way for you to understand what's going on in the weeds, which then helps you better direct the rest of the organization in that direction? Or how do those things fit in or are they separate?
Tamar Bercovici (19:22):
It's a good question. And I mean, I think every leader needs to find their mix of how they stay in touch with the team. I think for me, even as an engineer, maybe I was never overly focused on what programming language or what technology stack. And in fact, before I joined Box, I was actually wrapping up a grad school. I have a PhD in theoretical computer science, which is not even at all. I remember when I was interviewing, I got this question of like, "You do realize this is not a research position." I was like, "Yeah, I like coding and I like building things." I'm an engineer, but I never thought the point was the specifics. To me, engineering is taking a complex, messy, ambiguous, real world problem, and how do you decompose that? How do you simplify it? How do you find the right abstraction to actually then build it with technology?
(20:30):
And so having that ability to understand the abstract set of challenges, constraints, problems, to ask questions from the team and to sort of learn more about what they're hitting, to me, that is a little less impacted by hands-on code. Having said that, there's a lot of insight that you get about the experience of being a developer on the team from actually developing on the team, which obviously I had way more of when I was closer to that, I have less of now. So yes, I absolutely, both for the more essence of the technological architectural decisions we need to make, as well as just the day-to-day, getting insights from everyone that I interact with in the organization is important. And I expect my people manager, directors that report into me to also have a strong sense of what's going on in their organization, what's working technologically, what isn't, what are the big challenges, what are the trade-offs?
(21:28):
They similarly need to have that perspective. The benefit that the principal architects have is that they get to focus way more of their time on that. So they can actually and do, dive way more into the details. Most of them do commit code in the code base. They get the benefit of also operating at that level, as well as spending a lot more time cross-functionally with other architects on understanding the broader picture. So to me, it's a focus, what you focus more on, but ultimately
(22:00):
It's a collaborative leadership model where you don't always have all the roles filled, but you want to have a manager, an architect, product manager, designer if appropriate, all those people working together to figure out the right set of things for the team to do. And each of them are coming from a different focus area, maybe looking at different data streams, but ultimately they're coming together to figure out what's the most impactful thing that we could do next. And I think that leads to good and strategic roadmaps.
Ganesh Datta (22:31):
What have you found has worked the best for you personally? Like you said, it could be different from leader to leader. Do you, I don't know, join architecture reviews? Are you using AI to understand, tinker with code bases? Are you in operational excellence reviews? What are the things that you do personally to try and maintain that mental map of the systems?
Tamar Bercovici (22:51):
Well, we do pretty consistent operational reviews as a part of our execution reviews. So looking at how our various systems are functioning, especially for an organization like mine where it's a lot of high scale distributed systems, there's a bunch of standard KPIs that you would look at to ensure that things are as they should be or to see trends as they emerge. And yes, being involved in some of the architectural discussions, absolutely. I personally get a lot of value from that because I think it helps me understand where the optionality is and what the team is considering. And then also to a degree, I think there are those architectural opportunities that align challenges across several domains that you can then help influence or
Ganesh Datta (23:45):
Pattern match.
Tamar Bercovici (23:46):
Kind of see, especially for a team like mine, because we're providing that product abstraction layer almost, understanding what are those capabilities that are missing, but also what is the right way to model them within the platform to provide the most leverage? There's a lot of overlap between the technical architectural considerations as well as an understanding of the product and the strategy and what we need to focus on. So I both enjoy that and I also want to spend my time on some of those types of decisions because I see them as highly strategic to what my organization is going to deliver in terms of the value.
Ganesh Datta (24:26):
Yeah. And it's interesting, it's actually a good segue to another topic, which is, like you said, your organization is very focused on those product abstraction layers and delivering those to the rest of the organization. And in a world with AI coding assistance, writing more and more of a code, especially with newer models, how have you thought about the guardrails for that, especially when it comes to reliability and quality for your organization? And maybe I'm making an assumption here, but reliability is probably a feature of your platform.That is one of the core things you deliver to your end users. And so if that's the case, then how do you think about AI and your developers adopting that? Is it different from the rest of the box organization? Have you taken different measures? How does it fit in?
Tamar Bercovici (25:10):
It is and it isn't. And yes, absolutely. For a team like mine, reliability, performance, efficiency, all incredibly critical. And so just in general, regardless of whether AI powered or not, any code change that we deploy needs to be done so carefully and we need to have the right validation and we need to have the right incremental deployments and have the right metrics. And I think what's interesting about AI coding is that it's really pressure testing what you already technically should have had in place. People like to say a lot about, oh, the AI can write bad code or make mistakes or whatever. Well, guess what? Humans also write bad code sometimes or make mistakes for sure. And in fact, the AI to a degree when configured correctly can actually take more, what would take a human a lot more time to go and peruse the rest of the code base or read the standards or all these things that as engineers, we start kind of shortcutting usually at some point.
(26:19):
So it's not necessarily a different class of challenge. It's more that imagine pre-AI, you start the 30-person organization that we had at Box when I joined, it can be very closely aligned on what's happening. You can actually know what everyone's working on. If something goes wrong in the production environment, it's like, "Oh, that sounds related to what Mary just deployed." And you lose the ability to do that as the number of engineers grows because generally the complexity of the systems is high or the scale that you're operating is larger and you no longer have that context, natural, ambient context of what's happening. And so you have to start putting all of those processes in place to ensure that you have the right observability, that you have the right gates on your CI/CD pipeline. All of a sudden all those things make sense to invest in. So in a way, AI when done right accelerates that team growth all of a sudden it's like something between your one developer is maybe committing more code or committing more quickly, or if you take it fully into that, I'm now orchestrating over a team of agents, then maybe that one developer is actually kind of like three or four or however many.
(27:33):
And so I do think it sort of puts a bit of pressure testing on those systems to ensure that you have the right observability, that you have the right gates. One of the things that we're looking at is also just looking at all of the phases of the software development lifecycle and even the product development life cycle, because it's not just about generating more code, but rather can we also generate more tests or better tests? Can we leverage AI to help us better monitor and observe what's happening in our production environment or to help us identify signals across incidents that maybe we didn't see as much before. So just like AI can put more pressure on certain parts, it can also help you relieve pressure in others. And then I think finally, it's definitely true that if I run a team that has sort of a user facing experience, the ability to use AI to just quickly prototype a bunch of different options that I can play around with and use that to iterate, there's something very compelling about that use case.
(28:41):
I think what gets missed sometimes is that in a sort of a platformy backendy world, when we're contemplating some of those like, "Oh, should we model things this way or that way?" Sometimes you only really get a full sense for what it's going to look like when you try to code it out. So actually even in that quick prototyping phase of like, well, let's try to throw something together to just inform, is this a good direction or not can still be very relevant. There's also a lot of cases where you need to migrate from one version to another. I feel like actually if I had to best capture what I've done as an engineer at Box, it's just been one after another migration from different data stores, different production environments, different everything. And guess what? AI is really helpful for just knocking out a bunch of those types of changes.
(29:29):
So I think it's causing all of us to need to reassess how we do our job now that we have this very, very different set of tools than we used to. But I think there's ways to get value across the stack of engineering work, and that's definitely what we're focused on iterating on and improving on internally on the team.
Ganesh Datta (29:53):
I had Nathan Harvey on, who runs DORA at Google a couple episodes ago, and we were talking about how AI is an amplifier, like It makes high functioning teams function better and low functioning teams start to crumble because the bottlenecks in their process become more and more bottleneck. And maybe it's safe to say that a team like yours that was in the critical path that already built the muscle of if shit goes wrong, it can be really wrong. And so we should probably have systems in place to help us touch it. To
Tamar Bercovici (30:18):
Identify that.
Ganesh Datta (30:19):
Exactly. And so maybe that the teams adopted and I have more confidence. We're still
Tamar Bercovici (30:24):
All collectively iterating on this too.
(30:28):
I think this is sort of fun, but also is adding a lot of tension where because the technology space is so dynamic, it's not even that you can really optimize like, oh, here's the best way to do AI development. Because whatever that was six months ago is no longer true. Six weeks ago. And probably six weeks ago as well. And so it's almost our ability to live on the edge of consuming the new technology and ensuring that we're getting proficient in it so that we can figure out how to get more value from it, I think is really key. But yeah, it's exciting times for sure.
Ganesh Datta (31:09):
You mentioned that you've been using, or the teams have been experimenting with all parts of the SDLC. Are there certain things that you found particularly interesting or maybe things you wouldn't have expected to have been accelerated with AI? One of the things that we found internally is internal tooling, like being able to debug certain customer accounts. And we had all these queries that we can run against databases and try to piece together a picture of the world. And that's a good thing. We'd always talked about, "Hey, it would be really cool to have an admin tool where we could piece together more context and make it easier for us to debug things." That's a great example. So we can use vibe code. And actually we don't even care what the code looks like. It's an internal tool. And so those kinds of things have been accelerated.
(31:47):
Have you found things like that where AI has been impacting certain parts of the SDLC?
Tamar Bercovici (31:52):
I mean, I think there's just a lot happening in a lot of different phases. I wouldn't say that I've at least seen something that was highly surprising. We're also probably a large ... We're not very large, but we're large enough that even internal tools need to have a certain amount of rigor behind. Although absolutely, I see a lot of cases of just folks building something for themselves that they couldn't do before or using AI to analyze a set of data or knock out a few options. So I think a lot of what we're seeing though is just like you said, strong teams versus not, but also engineers that are naturally somehow better able to jump into this, whether the way they think or their interest, there's definitely a bit of a skew there. Actually,
(32:41):
One of my peers in another part of the organization started running a program that I thought was very cool where they actually identified engineers on the team that were using AI a lot and successfully, and then ones that maybe not as much, and they paired them up with a mentorship program, which sounds straightforward when you say it. But I think first off, the fact that they were all from the same team made it a lot more concrete. I think sometimes if you're not quite figuring out how should you be leveraging this to do your job, having someone that's doing a very similar job help you out is a lot more effective. I also kind of like the fact that it was sort of the definition of not level based or tenure based. So you have this, in some cases, junior engineer mentoring a staff engineer.
(33:29):
But I think that's sort of this moment that we're all in, finding ways to share best practices to surface things as they're happening across the organization has been really, really interesting. But yeah, I'm excited about a lot of the opportunities to quickly iterate as something that's historically been a little more difficult to do.
Ganesh Datta (33:54):
We hear a lot of engineering leaders talk about this low grade anxiety they have about what the adoption and coding assistance means to the reliability of their platform. Do you feel like the operational rigor you already had in place in your organization has made that better for you? Whether it is better CI practices and deployment practices and the ability to monitor that stuff for your operational reviews or not?
Tamar Bercovici (34:19):
Well, gosh, I don't want to jinx anything either. I do think that in particular for the teams that run highly sensitive, like if you're down boxes down type services, yes, they've had to build that muscle over the years. And so we have not seen AI topple that,
(34:43):
But I think it's something that we need to keep tabs on. I think it's also just, again, sort of assessing, are we using AI successfully enough? What could we be doing more? What metrics should we even be looking at? I mean, even taking a step back from AI, the perennial problem of engineering leadership is like, how do you measure the productivity of ... It's just like, I've been in so many engineering leader networking something or another where people are trying to either propose something that's never a full capture or kind of commiserate on how difficult this problem already was. Now you throw in all this AI agents into the mix and it just- It's impossible. It just becomes like even assessing the impact of what you're seeing on anything beyond qualitative is difficult. So we're also spending time looking at metrics and trying to get a better sense of where is this working?
(35:37):
What should we learn from the places where it's working well to bring to others? So I would say that this is all very much still dynamic and it's exciting. It's exciting to be in a state where we don't know exactly how it's all going to pan out because that's an opportunity for learning and growth,
Ganesh Datta (35:53):
Right? Absolutely. Yeah. You said something that just prompted an idea in organizations like that where maybe there is a little bit more of a risk aversion given the sensitive nature of what you deliver, there may always be room for more adoption of AI and maybe it's not the case, but maybe there is more room. And I was reading Alex Hidalgo's book about SLOs recently and it was talking about how the point of an SLO is to define acceptable levels of good and acceptable level of good or bad. And how you look at it, it's probably very, very high for teams like that, but there is still some error budget. And if your error budget is still fully available, then probably that means technically you have room to try something new. And is that maybe evidence that like, hey, we should maybe push the boundaries a bit more and take advantage of that.
Tamar Bercovici (36:38):
At the end of the day, every team has the onus of pushing code and innovating. And unless you're in an organization that's stagnant and nothing is changing, then yeah, keep the lights on, don't do anything, but probably you don't want to necessarily stick around there. But assuming the landscape is constantly changing, right? What we want to do with the product is changing, the scale at which we're operating is changing, the set of technologies that we can leverage is changing. So as soon as you push code to production, it's starting to age.
(37:12):
And so platform infrastructure teams are constantly shipping code. It just means that they have to ... And in fact, some of the code that they're shipping is exactly the code that helps them put more guardrails, helps make the systems more self-healing, helps make the pipelines to modify them more incremental. So in some ways, AI can accelerate our path to more resilient, more robust, more performant, more efficient systems, but we still have to use it the right way. If we go back to what is engineering, you got to take this ambiguous, complicated problem and you need to break it down into its sort of sub-components. If you can't do that well, no matter what AI technology you're using, if you give it a bad prompt, you're going to get a bad outcome. Just like if a product manager gives you a bad set of requirements or your architect doesn't explain what they want you to do, the engineer's not probably going to do a good job.
(38:12):
And so that problem doesn't go away. But if you understand what you're trying to solve and you can sort of say, okay, what are the components of this and where can I have AI help accelerate my path to delivering or my path to validating, then that's the type of question we need to ask. And of course that's going to look different. If I am a user facing kind of front end team, a lot of my risk is around shipping the wrong interface for my product, like shipping the wrong product experience. And so being able to spit out a bunch of options that are sort of semi-working and try them with users is a great way to de- risk that product effort. If I'm shipping a backend implementation, my risk is different, but that doesn't mean that I can't still find ways to use AI to accelerate mitigating that risk.
(39:05):
Execution is burning down risk.That's a one-to-one. And so it's just going to manifest a little differently based on where in the stack
Ganesh Datta (39:12):
We're- Yeah. I'm going to attack that risk with more rigor now. The last topic that maybe I wanted to chat about is a lot of platform teams, and when I say platform, I mean everything from product platform to platform engineering teams. I think they all struggle with a very similar thing, which is kind of showcasing to the business what success of their organization looks like. How have you thought about that? Especially because like you said earlier, there's not a ton of truly user facing capabilities that your team builds, it's stuff that is consumed within Box to build those capabilities. So how do you think about the success of your organization? Yeah. How do you tell your leadership that your team has been successful? What do you hold yourself to?
Tamar Bercovici (39:53):
This has been a journey for me to better understand my own team because if you don't know how to answer that question for yourself, you also can't position it or explain it to anyone else. I think the first thing you need to figure out is who your customer is. And it's not always the team that's building on your platform. So if you're an internal tools team or even providing sort of internal platforms, but where engineers can choose to use what you're providing or choose to use something else, then you need to measure yourself on their satisfaction as your customer. Are they opting to use your platform? What are your adoption metrics? What features do they require that might be stopping them from adopting you? And also, if you zoom out from a business perspective, what's the value of investing in internal engineers is you're making them more productive.
(40:55):
So you need to be able to say, when you invest in me, I make your entire engineering team more productive in this way, here's my step. So you're kind of like saving engineering time. That is not what my team does though. So what do you do if you're more of this sort of backend infrastructure type team? So now you got to look at it through two lenses. First off, first and foremost, your customer is actually the external end user customer. We are responsible for uptime, for performance, for maintaining various guarantees that we provide around data access, data compliance, security. And then to the business, we of course also make efficiency guarantees. And so that is a true, that's business value. If I can invest in a project that makes our infrastructure more efficient, that's dollars saved. If I invest in a project that makes our systems more reliable, that is less outages, less grumpy customers, less churn, that is dollars saved.
(42:00):
So if I can operate at a higher scale to help address a new use case, that's dollars earned and you have to think about it through that lens. Now, the complexity comes in where you're also working with these other teams to enable them to use your products. And so that's where you don't get the benefit of an easy KPI that you can just look at. And you need to understand, as you're building these new capabilities, first off, how much are other teams blocked on you? Are they coming and wanting to build something on your platform that is a valid and reasonable thing to build, but you can't support them?That's a negative signal. And when you build something, how much leverage are you getting out of it? Can you find a pattern that actually unlocks multiple use cases that are emerging so that you effectively increase the product velocity for the entire organization?
(42:52):
If someone figures out how to put a metric on that one, I'd be happy to learn. But I still think that you can easily describe the challenge and the value by saying, "Hey, these four teams are trying to build something and they each need to recreate this notion of collaboration for their particular flow, but if we build this one time, then they will all be able to adopt it easily. We can have a more consistency and guarantees around access control, and we now have this as a pattern for the next four things that are coming next year." So it does become kind of an engineering efficiency and a product velocity type conversation, but it's a little more nuanced because they don't have as much choice on not building on your platform. So it ends up being a slightly different than a developer tooling.
Ganesh Datta (43:44):
Do you still maybe think of their end user, the actual customers of Box as the proxy to your ability to support internal customers?
Tamar Bercovici (43:53):
Yeah. Because at the end of the day, if anything that we are trying to standardize or consolidate is not so much about, yes, it saves them work and that's important, but it also effectively means what can we guarantee for the customer? And again, this is a bit specific to Box and how we run our product, but the fact that we started the files and folders, but now we have so many more things within our platform, whether it's an eSignature document or a form or a workflow or an AI agent, the fact that all of those things from a user perspective can have a common pattern for how you share them, how you lifecycle manage them, the fact that they show up in your search bar when you're looking for something, there's an end user value to that, but you cannot manifest that value with every individual team needing to one-off code how their product works with every other possible iteration of every other team.
(44:58):
It's just like the matrix of all of the combinations becomes impossible. And so you have to have that standardized layer to be able to provide that cohesive product experience, which is good for the end users that are trying to use it. And it also, for us, is very key to that sort of admin narrative around data security, access control, policy management, because everything's running through the same system. And so now you can provide those guarantees. That's a customer
(45:27):
Facing value, even if it's not my team and my product manager that are out there talking to the customer saying it. And then when AI comes along, it was very clear to us, and to me, this is sort of the signal that we had done a good job understanding internally what each layer of our stack meant because by building our AI capabilities into that platform, it again let us not just provide leverage, but basically provide standardization and ensure that AI access to content was adhering to those same access controls and data policies and compliance policies. And so it's just another example of how having that integration layer, that sort of layer that manages the invariance of the system is valuable to the customer. And if you assume that the product teams must meet those customer requirements of having a cohesive and secure experience, this is the only way that we have a chance as an organization to deliver that.
(46:23):
And so that helps explain the value proposition. At the same time, I would say it's challenging. I remember I met someone who was a pretty senior in the internal platform backend team for search at Google or something, and he was also saying how hard it was to make the case for headcount. So I'm like, I guess some things are just randomly challenging. But I think if you have a clear understanding of what the value proposition of your organization is meant to be, that yes, that helps you make the case for investment. It also helps you explain to the people on the team why what they do matter and matters and how they should be thinking about the value that they're delivering. And it even helps you explain to the customers what benefit do they get from working with Box because we have this high leverage platform that underpins everything that we do.
(47:12):
And so they're benefiting from that scale and from that robust set of capabilities.
Ganesh Datta (47:16):
I love it. I mean, like you said at the very beginning, engineering is about taking these problems and building the right abstraction so they solve real things at the end of the day. And if you zoom that out even further, the point of a software engineering organization at a company like Box or Cortex or whatever is turning customer needs into code that solves the customer's problems. And so even if we don't know necessarily how to measure the success of a platform team, I think even internally, if we can think about the end user, the customer and our ability to serve them as a North Star, things will kind of work themselves to the right thing.
Tamar Bercovici (47:48):
Exactly.
Ganesh Datta (47:48):
Tamar, thank you so much for joining me on the podcast. It was an absolute blast.
Tamar Bercovici (47:51):
Thank you. It was such a fun conversation.
Ganesh Datta (48:00):
Thanks so much for listening to this episode of Braintrust. If this resonated with you, do me a favor, share it with another engineering leader who's wrestling with these same challenges. And if you want to continue the conversation or learn more about how we're thinking about engineering operations platforms at Cortex, reach out to us at cortex.io. Thanks for listening, and we'll catch you on the next one.