Subscribe
Share
Share
Embed
The Hangar DX podcast focuses on developer experience and learning how different companies solve developer productivity challenges at scale.
www.aviator.co/podcast
Ankit Jain (00:00)
Hello folks, and welcome to Hangar DX. This is a podcast about developer experience and how engineering teams solve productivity challenges at scale. I'm your host, Anke Jan, co-founder of Aviator. At Aviator, we are building a verification platform for AI generated code that creates guardrails to avoid AI slop. Today, my guest is David, all the way from Down Under. I'm very excited to have this conversation. David works at Canva
In the platform team. So we will be talking about what does it mean for platform teams and today with AI and how do they evolve? I also recently wrote an article about how every company will look like DevTool companies. So I think it'll be a good intersection of that. before that, David, thank you so much for having being here. I'm excited for this conversation and I understand this is a weekend for you. So really appreciate you making time.
Why don't you start with a brief introduction for our audience?
David Curlewis (00:52)
thanks very much for having me. it's exciting to be asked to speak. I'm Dave. from the New Zealand side of Down Under, just to clarify, not Australia. originally South African, moved here many years ago now. I've always worked in the platform side of technology since the sort of late nineties. and moved through various fields, seen a lot of change.
as you mentioned, now at Canva, I've worked through government, large org, lots of startups. yeah, it's an exciting time. it kind of feels like AI's been around for decades already. I think it's equivalent to sort of dog ears, right? So it's
Ankit Jain (01:21)
Ha ha ha.
David Curlewis (01:22)
hard to believe to to imagine what it was like before. yeah, we're working with AI platforms. I lead four or five teams in that space.
And we essentially build out the platforms that enable our product developers inside Canva to build our AI powered features and and and things that we provide to hundreds of millions of people.
Ankit Jain (01:40)
Awesome. So part of this conversation started with what you wrote, which was what happens to platform teams. I have this article in front of me as well. maybe you can talk a little bit about, you know, what compelled you to write that article, maybe give a little bit of, you know, some high level points of what you tried to convey in that article.
David Curlewis (01:59)
Yeah, I think I was reading something. I think I I I linked to it in the in the article or in in my post.
Ankit Jain (02:03)
The Martin Fowler, yeah.
David Curlewis (02:05)
there was something that Martin linked to that I I found interesting and it was something that I'd been mulling over and I you know we talk a lot within the teams that I lead in terms of analogies. I'm a very visual person and we'd been talking about how we
We're seeing people beat their own paths through the bush and we provide this golden path of this paved road, this highway. And now with AI, we were seeing that behavior change. We were seeing people, you know, the barrier to developing your own platforms and your own frameworks, your own tools, has dropped to, you know, near enough zero to make no difference. And yeah, we were just starting to struggle with how that influenced how we work, how we engage with our customers.
we were seeing some interesting behavior in terms of folks vibe coding their own solutions to things that we weren't able to move fast enough on. And yeah, I think it came out of that. I I kind of read a few more things on that in in that space and yeah, I I don't write particularly often, but I when I do write, it's something that's kind of been sitting in the back of my mind for a while. So felt I needed to get it out.
Ankit Jain (03:06)
Nice. Do you wanna share a bit about your thoughts on the article? Like what you covered in that article?
David Curlewis (03:10)
I mean the core of it is is this idea that, you know, years ago, as I say, I've been in platform engineering pretty much the whole time. And the way I used to think about it is that we would build the the golden path for the majority of users. So the the old 80-20 rule. And so we build for the most users, if you want to get from point A to point B, whether that's, you know, CI C D systems or, you know, mailing systems, whatever it might be, we'll build the platform that allows you to kind of
decouple
your thinking. You can just think about the product, the customer. you don't have that cognitive overhead of maintaining everything else down to the nuts and bolts.
yeah, so the eighty twenty rule, building for the majority, and that's held true for for decades. and the idea being that or the analogy, again, going back to this this visualization and and I like pictures, we're there handing out machetes to people so they can kind of beat their own way through the bush if they want to, but the majority of people are just gonna jump in their car, jump on the highway and
You know, it's a nice signposted highway. We maintain it very well. If you want to go and do your own thing, we'll give you a machete, but you're on your own. We're we're not going to support you. If you get lost, it's you know, you kind of have to find your way back. Now with AI, it's not really a machete that you're giving people now. You're kind of giving them like bulldozers so they can kind of
Ankit Jain (04:20)
Yeah.
David Curlewis (04:21)
go off with these highly powered bulldozers and really beat a a a a pretty reasonable representation of a road themselves.
if we have a rainstorm, it's it's might might wash away a bit. It's not going to be maintained to the same standards. It's not going to have lighting on it like a highway would have. So, you know, you can take this analogy as far as you want. It it's it's it's pretty close to the behavior we're seeing. But the problem is, is that now there's just all of these kind of pretty good dirt roads going all over the place. The position that I've taken is I think instead of us fighting that or trying to pull everyone back onto our highway.
the better position is maybe to go a layer down in the stack and think of what we're offering as more APIs. allow users to have a set of blessed tools that they can then go off and build pretty good roads, you know, do some paving, add some lights, some reflectors on the road, whatever. And so instead of basically fighting it, we're giving them a better class of bulldozer. We're giving them some like road laying machines that they can go and ro you know pave their own roads.
And yeah, we'll still in in a bunch of places we'll still offer a highway. but it's an interesting mix now because we're kind of offering platform products, but more and more we're offering much lower level interfaces into the the building blocks, the Lego pieces that allow you to go and build your own road through the bush, yeah.
Ankit Jain (05:34)
Very interesting. Can you maybe give an example like what does a Lego piece look like?
David Curlewis (05:38)
Yeah, I mean, I guess the primitives around things like standardizing on observability patterns, right? I mean that's a whole rabbit hole we could go down. I think there's there's a
Ankit Jain (05:46)
Yeah.
David Curlewis (05:47)
bunch of interesting loops and feedback there that is hugely important. I mean, observability's always been important.
Ankit Jain (05:52)
Mm-hmm.
David Curlewis (05:53)
but now with being able to feed that back and close the loop and have these flywheels where
You observe patterns of behavior, you can feed that data back in, you can make your your model or your harness or your whatever, your system better overall. makes it even more important to have that built in and kind of baked in from the from the get go. I mean, there's a whole bunch of examples, right? If you have standardized gateways or routing layers, your security and authentication, you know, you might offer a standardized set of MCPs and tools that you want people to use rather than going and again doing their own thing.
So offering those primitives and then but a a bit more flexibility in terms of how people actually get from A to B.
Ankit Jain (06:31)
Got it. So I know kind of like in your article you talked about like sort of like two core responsibilities of platform teams. One was around like helping with centralization, generalization. I think the other one was helping with cognitive load, reducing cognitive load for product engineers. So do you think that with AI it makes it easier or like the load is reduced for platform teams or has it increased?
David Curlewis (06:55)
I think it's changed. It's it's weird. I find myself giving this answer to a lot of different questions these days where everything it's nice and easy to have a binary answer, right? It's it's either black or white,
Ankit Jain (07:04)
Yeah.
David Curlewis (07:05)
it's true or false. And there's so many it depends answers in in this space.
as I said, I think the balance is changing regarding what a platform team does. I know in your article you were talking about, harness engineering being another form of platform engineering, essentially, right? It's platform engineering for the AI age.
Ankit Jain (07:18)
Right. I mean we talked about like yeah. Right.
David Curlewis (07:22)
yeah, so the where we focus and where our efforts need to go, I think is shifting.
As I said before, we're we're potentially in some places talking about going down to a bit of a lower level. You know, I always think of platform teams as levels. There's there's a lot of teams calling themselves platform teams these days, but you have the actual infrastructure teams that are like the real base level platform teams working with the the well, what used to be in the day, the tin, the cloud, provisioning things. and then you've got layers built on top. And so I'm just seeing things maybe move down a few levels.
Ankit Jain (07:53)
Mm.
David Curlewis (07:53)
As I said,
offering more primitives. I don't know that it's reducing the load. I think in some ways it is increasing what would the right word be? Not maybe cognitive load, maybe almost stakeholder management load, is is interesting because we're having to do, you know, platform teams stereotypically have been quite inward facing. We haven't been known as the most sort of
extroverted teams in terms of having lots of stakeholder relationships and product managers and all of that sort of stuff. whereas I think that's really important now because you are having to try and understand what the customer's going to do because they're not going to do it over six months anymore. They're going to do it over maybe six days. if if you can't help them straight away, they're just like, well we can just do it ourselves. So yeah, I think the I think the load has changed. It's interesting.
Ankit Jain (08:37)
Yeah. I mean, I think it's also just kind of reiterating also something which I wrote in my blog, which was about how every company is becoming a dev tool company in some ways. the reality is like just that platform engineering itself is become such an important vertical for the organizations. even five years ago, pre LLM, we used to talk about how
Platform engineering is sort of like this engineering enablement which happens towards a when a company becomes somewhat more mature. And now it's like, you know, even for like a 10% engineering team, now you you are starting thinking about the platform strategy. Right. You kind of like need people who are actually like building these things. So it's sort of like becomes something which used to be a you know something that organizations would adapt when they need it, or like sort of like when when sort of like, you know, there is more
David Curlewis (09:24)
Yeah, more of
a retroactive Yeah.
Ankit Jain (09:24)
Yeah. And versus now
it's like sort of like become a very critical part of pillar in the organizations, right? I even speak with even our hanger DS community. We talk about how like this has sort of become now a vertical that they talk about in their you know, QBRs and your sort of like organization organization goals, which is like, you know, you need to be able to activate and enable
Everyone in the organization.
David Curlewis (09:48)
Well, it's I that's great to hear as a platform engineer, because I'm probably biased in that I feel like it's always been important and it's it's always been a critical piece. I I agree though that I think it never made sense, you know, and as I said earlier, I've worked in startups from very small scale through growth phases and and been lucky enough to do that a few times. And yeah, in the early days everyone's a generalist and there's no dedicated platform team, front end team, you know.
API team. And yeah, I think platform engineering is is the same as any of those where at the beginning you maybe have someone who spends half their time building the frameworks and the things that you need for everyone else to be a bit more productive. And then as you mature, you you realize the the advantage of that centralization, right? The removing the cognitive load, but also centralizing that cost center. And so having that multiplicative effect across all of your developers as that count goes up.
I think the difference now is that there's so many more people that we consider builders, right? You don't have to be a developer to be a builder. We like internally at Canva, we've referred to folks within our AI platform group. we we've always had this concept of personas that we serve, right? So we have researchers, we have MLEs, and more recently we've been referring to them generally as AI builders.
Because
it is such a broader like it's a broader customer base. So I think that tipping point where it makes sense for a company to say, Okay, hang on, we might actually need a platform engineering shaped team or role, is is moving earlier,
Ankit Jain (11:11)
Earlier.
David Curlewis (11:12)
left, whatever you want to say.
Ankit Jain (11:14)
Right.
David Curlewis (11:14)
yeah.
Ankit Jain (11:15)
I think one of the interesting things which I found in your article, and I feel I may have slightly different opinions on this, is this whole aspect of like centralization and ownership in some ways. You know, there is an argument to be made that with AI, more people can be enabled and then they will end up owning things themselves versus being able to centralize and own certain things.
I'm curious, like kind of like how do you think about that balance, right? Like in some ways you're also talking about like, you know, giving them APIs, giving them building blocks. so you are still owning certain things. Like where do you define the line where okay, we will build a dashboard for you versus sort of like we'll give you certain things to build the dashboard yourself?
David Curlewis (11:54)
Maybe it comes down to adoption or maturity. we talk about maturity quite a lot and in combination with adoption. So we're absolutely not against people doing their own thing. So if you need to go and build that dashboard, that you know, interface, that framework, unblock yourself. we'll support you where we can, but you should actually, you know, absolutely move fast, do what you need to do. When that
maybe becomes a bit more generalized is that we start seeing three or four teams doing the same thing. Maybe we see your your one thing that you've built that you own is now actually being used by four or five other teams. That's the point where we at least ask the question of, you know, should we start discussing some sort of graduation process? So we we have this idea of of graduation and kind of you have things within
like a proof of concept or early stage environment, not a physical environment, but you know, that that that concept, that sort of bracket. And then we can graduate out of that and we can start providing the additional lighting on the highway. We can provide the the the the checks and balances that make it a bit more of a productionized hardened service. So I don't think it's a case of again a a very binary thing where we're dead set against
everyone owning their own thing and we're pro centralization. I think it it's the question of when does that product team that is focused on building the thing, right? They're building the the widget that the customers are interacting with.
Ankit Jain (13:16)
Mm-hmm.
David Curlewis (13:17)
When a significant portion of their time and effort and cognitive load is spent managing the things that they use to build that thing, then maybe it's a time to for them to stop and think, well hang on, this isn't the best way for us to spend our time. We should actually get someone else to do this for us.
Ankit Jain (13:31)
Interesting.
David Curlewis (13:31)
who's
who's looking after something that actually looks very similar, but maybe it doesn't quite meet our needs. And then that's where we need to be adaptable and flexible. Yeah.
Ankit Jain (13:39)
Interesting. I love that analogy of like putting lights on the highway.
David Curlewis (13:42)
Yeah.
Ankit Jain (13:42)
I think I think that's also interesting, right? In some ways, if you think about it, the more you're able to enable others to own things, the more things you can do, right? Then kind of like in some ways it frees up your time. One of the things which I
David Curlewis (13:54)
Absolutely.
Ankit Jain (13:55)
which one of the things which I find which is challenging many times, especially in enabling others and kind of like passing over the ownership, is the ownership can get modeled.
Like for instance, you know, this is an example which we saw during token maxing era, which was, okay, if we enable everyone to build their dashboards, like you'll probably have like 20 different versions of the dashboard. Like, how do you think about that type of a problem? Where it's like, okay, who owns it? Like who should build these dashboards? And like how do we kind of like even make them more consistent? Because if you have 20 different dashboards, very likely you have 20 different different data points and different data representations, and they may not even be consistent.
Right. So which is like a benefit you get when you centralize it. So how do you think about like that kind of a balance?
David Curlewis (14:39)
Yeah.
Again, I mean it's funny that you're talking about dashboards because I mean that's not a a type of thing that we support. But you know, going back to
Ankit Jain (14:45)
Yeah.
David Curlewis (14:46)
back to my data days, my my nickname is database Dave for a reason. And so I've come from
Ankit Jain (14:50)
Ha ha ha.
David Curlewis (14:51)
a I've come from a very data heavy background. And in the pre AI sort of the the the the olden days,
Ankit Jain (14:57)
Mm-hmm.
David Curlewis (14:58)
it was pretty common when when when folks would go and create their own dashboards in, you know, Excel or create an access database, create a
Power BI interface. And yeah, everyone would have their own. And the the biggest struggle there was like what is the central, you know, the the source of truth? What is the number that everyone can rely on? And so again, it's as I keep saying, it's it's not a new problem, but I think the scale and the pace that it's occurring at now is the the thing that's changing. So I definitely don't have I don't think anyone has the answer because we're still figuring this out, but
One of the things, one of the levers that we're playing with is how early can we start that engagement with the stakeholders? hence to my earlier point around having more stakeholder engagement, more product input, more, you know, roles like TPMs and things involved as well. how early can we have that engagement and try and preempt what they're gonna go and do their o on their own? even if they do end up doing that, at least if we know about it, we can start.
maybe slowly steering the ship in that direction. and it might take us a few months to catch up. But then there's a a clear point where we can say, okay, we we do what you need it to do. And so we can now we'll fold your thing into our thing, or we'll take ownership of your code base. something that I don't see talked about probably enough is is the idea of within bigger organizations, this sort of inner sourcing your code. Because
Ownership
is always a problem, right? ownership has forever been an issue. You know, who owns, who signs off on PRs, people stepping on toes and making architectural decisions that other devs don't want, you know, don't like. that's still a problem today and it's only gonna get worse. And so what the the the tack we're taking is trying to encourage this inner source model. So inner source being sort of your open source contribution model, where yes, technically perhaps we own the code base and we're on like the owner's file and we
sign off on PRs and but it's not a blocking action. We don't gate things. It's more a collaborative environment where you're almost contributing to like a inner source open source project. And again to your earlier earlier point, this actually helps us in terms of velocity and and getting more done with the limited headcount that we have because we can rely on other teams to to come to us and say like we need your
this dashboard to have this new metric on it. And
Ankit Jain (17:11)
Mm-hmm.
David Curlewis (17:12)
we can say, well, we don't have time to do that. It's not on our roadmap. And you're the you're the only team that wants it. If you want to make the change, you can go and add it as a new drop down or whatever. And if you commit the change, that's cool. We'll we'll approve it. and so you're kind of getting some headcount for free to do some stuff that may benefit you down the line.
Ankit Jain (17:26)
Very interesting. So this is almost like an open source model where you kind of like build
David Curlewis (17:29)
Yeah, yeah, yeah.
Ankit Jain (17:30)
build up kind of like scaffolding, build sort of like the infrastructure and enable others to contribute sort of like, you know.
David Curlewis (17:36)
Yeah, I'm not
I'm not sure whether it's an actual thing or or who came up with it if it is. But I know I worked at GitLab for a while and I know there it was it and and they've written about it and I mean they they're famously sort of transparent and everything's documented, but there is this concept of inner sourcing, which is that and and they were obviously an open source company, so the a lot of the the muscle was already built to to do this, but yeah, this idea that you you treat other teams just like the community.
And you own this code base, which is open source inside your company. And if they want to make a contribution, it's a similar sort of thing. They might might talk to you beforehand if it's a bigger change, you know, hey, we're thinking about this quite big shift. Or if it's a smaller, you know, minimal thing, feel free to just go ahead. this is another whole rabbit hole that I've been thinking about and and even writing about is is also this idea of it's often not
people that are doing that now, right? It's it's their agents that are wanting to contribute to your code base. And this is maybe a bit more applicable to kind of your space, which is the how do we put guide rails around that? So how do we codify what a good PR is? What are our standards? You know, the like in the old days you'd have a contribute a contributors file on an open source project.
You you still need that contributors file for the agents, but I'm wondering whether there's a point that makes sense to that it makes sense to sort of
address the agent as your primary user rather than just a by the way, like, our agents can read the contributors file and they'll get the gist of it. But actually having like an agent's MD file that explains this is how you contribute. This is the type of PR that's gonna go with go through this is we we we're not gonna go in this direction. So this will just get blocked. yeah. Codifying a bunch
Ankit Jain (19:10)
Right.
David Curlewis (19:11)
of of stuff, maybe even having examples like implementation examples, execution examples, like
All the stuff that helps that we know helps LLMs do better in those scenarios, it's kind of you wanna automate that and have that as part of as like a first class artifact within your code base.
Ankit Jain (19:27)
Right. I think part of it is also kind of just defining what does responsibility mean, right? Like if you get a contribution, like who owns if things break because of that? And that also becomes an interesting problem.
David Curlewis (19:39)
Yeah. Yeah. I think that's a bit clearer in my mind. I think and and that is really the the kind of the meat and potatoes of the whole thing is that when you talk about responsibility, it's not necessarily, you know, who decides if we structure the code in this way or that way. that's a a kind of a lesser part of it. But the bucks it it's like who who does the buck stop with? if it goes down at two AM and it's an important system, who's gonna get paged?
And that's where I think the platform team still does have a play to a role to play.
Ankit Jain (20:07)
Hmm.
David Curlewis (20:08)
if something's vibe coded and you're just running it from your laptop and it's you know, there's obviously no expectation that it's going to become mission critical. But again, going back to my earlier example, I myself have personally created some dashboards just, you know, thrown together in a couple of days for an exec who wanted to see some specific view on some data.
And the next thing you know, that becomes mission critical. So at the end of the day, it was it was before
Ankit Jain (20:30)
Ha ha ha.
David Curlewis (20:31)
even laptops were a common thing. But at the end of the day, I I shut my my machine down and go home. And the next thing you get in the next morning and the person, the the exec's hitting you up about like I couldn't access the thing from home and you know, this is unacceptable. And it's all of a sudden become a 24 seven expectation. So I've seen how quickly that can tip over. And I think that's part of that stakeholder engagement. We need to front foot that, we need to get ahead of that and understand that.
this framework, this thing that they've vibe coded. it's now used by three or four teams. It's maybe even like underpinning some development that's on critical path for some really important feature that we want to release. Shit, we can't, we can't let this go down, right? So we kind of need to forcefully kind of go in and ha have conversations about look, if this goes down, these are the implications, let us at least take on some part of it so that we can, we, we get to know the code base, we understand it and we we can support it for you.
or w work with them to at least understand what's n what's needed if they if they're gonna support it themselves. Yeah.
Ankit Jain (21:27)
Right. I remember in the article you also talk si about signal problem, which is sort of like, you know, design documentation, specifications. You're already touching up on a little bit on this, right? I'm curious like how how you think about like some of this, in some ways, you know, we were talking to Margaret and she mentions us cognitive debt, which is like, okay, even if agents understand some of the things, they may not be able to understand the complete business context. And and even the examples where
your there are like design talks and everything being written. how are those things changing with AI? Like have you seen any kind of like patterns which are successful or some things where knowledge is degrading? and kind of like are there any things we can do about it?
David Curlewis (22:06)
Not
sure I'd be brave enough to say anything successful yet because as I say, it's it's so early that we're still seeing things play out. yes, I've definitely seen this occurring, right? I've as as I'm sure most people have in production settings.
Ankit Jain (22:19)
Mm-hmm.
David Curlewis (22:19)
it's this I think them I'm not sure who coined it, but there's I've also heard intent debt. It's like the was it? Yeah, okay. So
Ankit Jain (22:24)
Mm-hmm. I think that was also Margaret and maybe. Right. Yeah.
David Curlewis (22:28)
so yeah, I mean.
How do you how do we document the intent and then you know that how that flows through to what you're actually producing at the other end, which is kind of where that, you know, the the the next stage of debt kicks in with it's which is that understanding of how the actual code works and and how do we close that gap? yeah, I mean I've seen just more broadly in the industry and and within Canberra itself.
Various people have tried things like discussing those early stage artifacts, whether that's you know, PRDs or specs, discussing having those in the code base somewhere, having those as the artifacts that are reviewed. and I mean I know optimizing PRs and making that process more efficient is something that you definitely focus on with your your products. kind of
I don't know, I don't want to take the ease way out and turn it around to a question to you. But like I am interested in kind of what you're seeing customers doing in that space because I I see too many issues with that. So having just purely going the other way and saying, we don't read code, we're just gonna read the spec, and that's gonna be the PR process as a human reading the spec. And I'm like, well, that's you're just shifting the problem somewhere else, but it's still a problem. yeah, I don't feel like that's gonna work for a few reasons. Some I can't even put my finger on, but yeah, I'm interested to hear kind of what
you're seeing actually work.
Ankit Jain (23:38)
I mean honestly, like, you know, we are also figuring out some of these answers ourselves. We have some mental models around creating, you know, intent-based development. We call it kind of like intent-driven verification, where we try to extract more information from the agent sessions and and see kind of like can we actually use that to create those intents and specifications, which are not necessarily always, you know.
Leading in the sense they're not always created before the code, but they're sort of like created hand in hand. Right. In some ways, the challenge we find with specifications is it's hard to maintain them. Because at the moment you write specification and you start implementing some capabilities, there is always a gap between them. And that becomes hard to document. So that's why in in some ways.
David Curlewis (24:24)
Yeah. They diverge straight
away, right?
Ankit Jain (24:26)
Right. And then yeah, exactly. And that's kind of like what we're trying to also figure out and answer is like how do you actually bridge that gap? How do you bridge the gap between the specification and the implementation? Because without that, like you can't really like go one level above and read the spec and not the code. and that's definitely like, you know, in some ways we talk a lot about in our Hanger DX community sessions as well, which is the tooling today is so much lacking compared to what we want and need for
AI software development to succeed. Right. It's just that nobody really has the answers. Yeah. Sorry.
David Curlewis (24:53)
Yeah. And I think we're seeing yeah, and
we're seeing that just by the fact that there's there's a lot of stuff in in, you know, provided from vendors in terms of tooling that
Ankit Jain (25:03)
Mm-hmm.
David Curlewis (25:04)
various vendors will claim solve various problems. And even given that sort of rich environment, a lot of companies, you know, friends and colleagues that I've ex colleagues that I've spoken to at other companies as well, everyone seems to be building their own solution.
It's kind of like a meta version of this whole platform engineering problem we've just been talking about. Is like at a at a intercompany level, there's like no platform team helping solve the problem at a centralized location. It's it's everyone's doing their own thing. some are successful. we've done some really cool stuff internally at Canva, but I've I've I've even heard of other examples from friends who have gone through that pain of
developing a a a a fully featured thing, you know, whether it was like you know, before Codex and and and Claude Code were kind of what they were, but building their own versions of that or even more sophisticated tooling to do the full end to end, you know, PR review, spinning up multi agents and and all of that. And then realizing that they're just now a product company maintaining a product instead of their actual product. So yeah, it's funny to kind of well
I don't know if it's funny, funny, but it's it's it's interesting to watch.
Ankit Jain (26:05)
Ha ha ha.
David Curlewis (26:06)
It's probably not funny if you're dealing with it yourself, but it's interesting to see that evolution. I think you you you mentioned the word validation, like you you you you how you validate. And I think that's a key thing as well. So PRs are not I think you have to boil it down to like what is a PR for? And is it for you to go and read the code and make sure that the indentations and the linting is the way you like it and the
you know, the function names match your it's like that can all be automated. again, you go back decades, we didn't there was no linting, there was no automatic code verification that th the way there is now. And I think this is the next iteration or the next evolution of that is we need to rely on solid validation rather than solving how people just review more code. I I feel like that's the wrong angle to attack it from. So
Yeah, it's interesting kind of having from what I've seen of of like your offering is focusing more on the how do we get the the thing that's causing this pain for us, this AI stuff, how do we get it to do some of the work itself in a deterministic way that we can validate? And I know there's like there's been some conversation recently about more sort of like formal methods of validation.
Ankit Jain (27:11)
T L A is yeah.
David Curlewis (27:12)
yeah, so I I I know you've had Annie Vella on
recent or a while ago now, but on this podcast. I've worked with her before and I I just saw she wrote something recently about starting to think in this direction as well, which which put me in touch with a bunch of other articles and things which I found interesting. So yeah, it's interesting. Everything's a cycle, right? We're kind of going back to things that were were cool back in the nineties or even earlier. And it's like things that were maybe too difficult to do are maybe now more possible to do. Yeah.
Ankit Jain (27:39)
Right.
David Curlewis (27:40)
Interesting.
Ankit Jain (27:40)
Interesting. Okay. I did want to cover other topic. maybe if you have any broader points we can try to cover around measurement. Like you kind of like also wrote that other article about like, you know, code you know, how we are like measuring lines of code and stuff like that, like how we are sort of like also regressing on that.
David Curlewis (27:59)
Mm.
Ankit Jain (28:00)
do you have any thoughts on sort of, you know.
Are there any things that you have found which are useful or less useful today when it comes to measuring productivity or measuring sort of like the impact of AI?
David Curlewis (28:12)
Yeah, I think I I wrote that post again because it was something that I cared about and it actually got that that was my my top performing post of all time. It got kind of on hacker
Ankit Jain (28:21)
Ha ha.
David Curlewis (28:22)
news and that was a scary experience in itself. and it was kind of funny, it's a bit of a tangent, but you know, I was writing about how sort of output focused metrics are not the right metrics and
this article got so much traffic that all I was doing was refreshing my dashboards, looking at output metrics of like page views and d that wasn't lost on me. yeah, I think I there there's so many people that have spoken about it, I feel like, more recently that I'm just reiterating things at this point. I suppose we all are, but lines of code as a measurement was kind of laughable, right? We we we threw that away ages ago, decades ago.
And it feels like we've gone full circle again, but in a bad way, in that that's or or equivalent metrics have become or were becoming more prevalent. I feel like that's maybe tailed off. You know, there was there was the whole token maxing thing, which hopefully has gone away and died somewhere. and that focus on output of just pure whether it's PRs, you there were there's folks in the industry trying to come up with
Clever ways of measuring the complexity of a PR, how much engagement your PR has had with agents and humans and even weighing those differently. And there's all sorts of interesting stuff that I've heard and and and seen. whereas I think and I and look, I think some of that plays a part. I'm not saying it's completely irrelevant. I think even just the old school, like how many PRs has a developer pushed out this year. If one developer's pushed out 10 PRs in a year and another one's done five hundred.
There's something going on there. There's something interesting there, right? But if someone's done 500 and someone's done five hundred and twenty, that's irrelevant. I I don't care about that. So it's it's a very indicative at a high level metric, but I see too many people focusing on that as a as a core metric. I prefer to focus on the the harder to measure things. I mean, that's why I liked your interview with sorry, what was it, Margaret Ann
Ankit Jain (30:06)
Moderna.
David Curlewis (30:06)
with the space, one of the co authors here.
So I am a fan, you I'm a fan of Dora metrics and and I love those back from the sort of DevX and and and DevOps days. it feels funny saying that it's like that's a past thing, but yeah, back in those old days. space I I I do quite like. I know it's a not a lot newer, but I I quite like the mix of it's got a bunch of stuff in there which is very measurable.
There there's there's a lot of overlap with parts of the Dora metrics. But it adds on the the pieces that we're missing, I feel. So that less tangible stuff, the the developer experience piece. but I'm seeing a bunch of interesting stuff and I mean we're doing some of it internally as well, which is having these regular developer surveys trying to measure at least at a high level, the developer experience overall, you know. and that's
kind of why I even originally started talking again after a few years to to Annie about her master's work in that space, you know, measuring and and finding a an actual accurate way to measure that feeling in everyone's gut that people have been having of like, I feel much more productive. I feel like I'm, you know, I'm kicking ass today. I did a whole bunch of PRs and I wrote a bunch of c or my agents wrote a bunch of code. But
Ankit Jain (31:14)
Ha ha ha.
David Curlewis (31:15)
at the end of the day I kind of don't feel great about it. Like I feel
like something's missing or yeah. So it's it's I'm not in that sort of league of the folks trying to actually come up with definitive measures on that. But it is an interesting space that I'm I'm yeah definitely watching. But I guess I I I have a bad habit of rabbit holing, but going back to your question in a nutshell I think it is just focusing on the outcomes rather
Ankit Jain (31:38)
Hm.
David Curlewis (31:38)
than the volume based measures. Because the volume based measures I think are just
gameable, they're less relevant. yeah, they they're just a a a a race to the bottom basically. Whereas your outcome measures are much harder to quantify, much harder to measure, but they're the ones that actually provide value. Yeah.
Ankit Jain (31:54)
So in some ways, like, you know, things that we talked about even ten years ago for DORA metrics meant like most of them are still true, right? Which is an interesting aspect, which is, you know,
David Curlewis (32:03)
Yeah, totally.
Ankit Jain (32:05)
like even in the era AI era, most of the software development principles more or less remain the same. Right.
David Curlewis (32:11)
Yeah, well I mean if I if I have one gripe and I don't know how much of this is me sort of the you know the the angry man shakes fist a cloud meme. I have enough grey hair, so I think I'm allowed to, but I the thing that makes me grumpy these days is just the renaming of everything. like having to come up with new names for everything when I don't know, I heard I heard someone just yesterday talking about I know
like forward deployed engineering is a big thing at the moment. FDEs. And there's job listings all over the place for FDEs. I still haven't been able to get anyone to describe how that's different to a software engineer. You know, just a good old generalist software engineer. You know, we talked earlier about startups in the early days where you have you val you valued generalists over specialists, right? And I think this is another area of interesting discussion is this specialization trap. I think that was coined by again Annie
But it's it it matches what I've seen in real life is this we went down this path of everyone specializing, maybe to our detriment of, you know, now that we need to be generalists again and I can be a product manager all the way through to a front end you know designer. you have to kind of brushle for those skills and be be happy working across lots of spaces and as you said before, you know, context switching between lots of different domains.
Yeah. so I'm not sure I'm not sure what the next step is there, but just
Ankit Jain (33:27)
Ha ha ha.
David Curlewis (33:28)
r renaming everything for the sake of it, gets old. maybe it's just 'cause I can't keep up and that's another whole thing. It's like how do you keep up with all this stuff? But yeah, I I think
Ankit Jain (33:36)
Yeah. I mean
David Curlewis (33:37)
we we there is a danger of throwing throwing away things that are still relevant. Dora being one of them, right? Like it's absolutely still relevant. nothing changes.
Even with everything else changing, they're still important for the same reasons they always were. yeah.
Ankit Jain (33:49)
Right. I mean it's kind of like just also a common human tendency to always look towards new things. Like even when you read blogs, books and blogs, you always want to read like the new books and the new blogs, even though most of the things in old books and old blogs may still be very accurate and very relevant. Right.
David Curlewis (34:05)
Yeah. Yeah. No, I understand
it. I mean, even in terms of engagement, it's you're gonna get a lot more engagement on a tweet or a LinkedIn post or a blog that talks about some new thing. Is this the new wave of software engineering, rather than just hey, here's why Dor is still important or here's here's how software
Ankit Jain (34:23)
Ha ha.
David Curlewis (34:24)
engineering is changing. it's not gonna get as much engagement, right? Yeah.
Ankit Jain (34:27)
Right, right. Okay,
all right. I think David we have already spent more time than was allowed for us. So thank you so much. This was such a fun conversation. And thank you everyone who was listening. If you are interested in developer experience or work in DevOps or platform engineering, join us at hangar dx community. It's dx dot community. With that, David, it was a pleasure. Thank you so much for your time.
David Curlewis (34:48)
Thank you. It's been it's been great fun.