Subscribe
Share
Share
Embed
In this episode, Emanuel Mueller Ramos, Head of Developer Experience at Skyscanner, discusses the evolution of his team as they transitioned from focusing on frameworks and middleware to becoming a customer-centric, impact-driven organization. Emanuel details the strategies he used to gain stakeholder buy-in, why it's crucial to rethink traditional productivity metrics, and how they made a cultural shift to prioritize developer happiness and effectiveness. This conversation highlights the steps necessary to build a developer experience function that delivers meaningful impact.
Mentions and links:
Discussion points:
- (1:14) The beginning of Skyscanner's developer productivity division
- (3:53) Gaining stakeholder buy-in and refocusing the teams
- (5:57) Redefining success metrics for developer productivity
- (8:57) Pitching the developer experience focus to leadership
- (17:26) Moving from frameworks to feedback loops
- (20:45) Fostering a customer-centric culture
- (23:20) Defining the collaboration between platform and developer experience teams
- (26:41) Choosing the right metrics for developer experience success
- (31:31) Risks and challenges ahead
Creators & Guests
What is Engineering Enablement by Abi Noda?
This is a weekly podcast focused on developer productivity and the teams and leaders dedicated to improving it. Topics include in-depth interviews with Platform and DevEx teams, as well as the latest research and approaches on measuring developer productivity. The EE podcast is hosted by Abi Noda, the founder and CEO of DX (getdx.com) and published researcher focused on developing measurement methods to help organizations improve developer experience and productivity.
Abi Noda: Thanks so much for your time and for being willing to come on the show. I'm really excited to speak with you today.
Emanuel Mueller Ramos: Thanks for having me. I'm very glad to be able to be part of this conversation in particular considering that so many times I've been listening to your podcast and through it learning new things that I could apply in the transition I've been doing in my group and now being able to share those experiences is also a great opportunity. So thanks for having me.
Abi Noda: Well, I'm really excited to have you share what you've been working on. We met a few weeks ago and you told me about how at Skyscanner for several years you've had a platform organization that's been focused on things such as CICD, observability, internal frameworks, and more recently you've started a new division focused specifically on developer productivity and happiness, consisting of four teams. So today I want to unpack this story and journey of how this decision came to be, the challenges you're navigating as you make this transition and insights and stories along the way. So let's start with, tell me how this came about. What is the thing that sparked the idea or the decision to go in this new direction?
Emanuel Mueller Ramos: Yes, of course. So I guess my journey started earlier this year. I took the leadership of actually an existing group. One of the interesting things is that when I hear about going towards developer productivity or experience journey, normally this starts with a small group, maybe like a couple of people that are interested to doing and that naturally grows through time and gets funding. In my case, the journey was a bit different. I took the leadership of a group of four teams, which include some of the most senior members of the company, which was very capable and doing very well. But at the same time as I was a new leader and I started to diagnose the organization, I could observe that there were some challenges that we needed to navigate. That challenge, mainly being how diversified were the areas that you are operating.
To give a little bit of illustration, if you think about a technology stack going all the way down from the platform all the way up to the end-of-user application, you are operating a little bit everywhere. You are doing a little bit of the platform with Docker images then with foundation libraries and frameworks, which was a good part of the work you are doing and then moving all the way to the application level, sometimes doing capabilities that are used by other teams.
And the diversity was one of the main challenges because it created a little bit of lack of focus. It was very hard for us to work with our clients, in this case, our product engineering customers and set them for success and be able to provide them what they needed. So feedbacks that I would get from the team members themselves would be, am I doing the right thing? Am I focusing on the right thing? And the customers also to ask like, is these projects the ones that are going to bring the best impact? So this is where I started to try to understand where you should focus and where to bring the best impact.
Abi Noda: So it's really interesting you described how a lot of the journeys you've seen with other organizations that this developer productivity, developer experience begins as, like you said, just a couple of individuals who get started with an initial project and it organically grows from there. In your case, like you mentioned, you immediately kicked off with four teams made up of really senior folks across the organization, how were you able to do that? Maybe take us through the decision-making process, the buy-in process, the stakeholders involved. How did you make the case that this is something that needed to happen?
Emanuel Mueller Ramos: Yes, so I always start when I, as a new leader, the first thing I have tried to do is to make sure that that organization was set for success. And the way I did to do this is first doing a review of our strategy, our metrics and the needles that you are moving in terms of is this doing the best for the organization? How I did this? Once I started to realize that there was an opportunity or an area to develop was to number one doing what I call a domain mapping exercise. If you think about like front-end topologies, the domains, the areas of focus and how I started to do this is looking to the different areas of what a group could bring impact and map then around.
Actually if you think about if you were to put a pyramid of different quadrants where you could think about being more of a platform tribe or a developer experience tribe or being more like a group that's owning central capabilities, I could map that we are doing a little bit of activities across the board. So when I started to realize that, that was my call to say we needed to bring focus, we needed to find a direction and that's where I started to engage stakeholders and leaders to start to bring that discussion. The way I operated was in the two phases, first speak with the stakeholders and understanding the need to change and after actually executing the change.
Abi Noda: So there was almost a topological case around the makeup of the organization. You referred to team topologies. What other evidence or business justification did you bring to the table to convince stakeholders about this proposal?
Emanuel Mueller Ramos: I think one of the main areas was in that you find what are the main metrics to define success of the organization. Until then, the main metric that you consider to be the area of success was the adoption. If you think about in a developer productivity having several frameworks and middleware, you'd say the more the tools are used, the better value they are bringing to the business. But this is a challenging metric because how can you prove that having the usage of this is actually going to bring the best value for everybody?
So we start to think about we need to see differently, maybe there are other ways to measure our success, to measure our impact. And we start to ask ourselves, is the focus purely middlewares and frameworks and capabilities, the best way to accelerate the organization? That's where we start to see maybe we need to diversify where you are focusing on. If you think about different pillars of where maybe a developer productivity or experience group can bring value, and referring to the developer experience framework for example, there are cognitive load, flow state and feedback loops.
At that point in time, the group was focusing mainly on cognitive load with that idea that if I build a framework, a middleware or a capability, we are removing some of that cognitive load from teams. But depending on what areas you are focusing, that brings a multiplicative effect of one in the sense that if I'm not doing this, maybe someone else is not doing it, but how can you bring a multiplicative effect of? How can you make multiple teams to move faster and move better? And that this is where you start to say maybe there are other areas to explore. Maybe just focusing on reducing cognitive load by building the frameworks is not where you're going to get the best impact. So this is where some of the things you start to use is looking to metrics and thinking about other areas to explore.
Abi Noda: Got it. So there was a shift from thinking about the problem in terms of adoption of the tools and frameworks you were developing to a broader perspective around how is the work we're doing impacting overall developer productivity and then where are the most impactful opportunities for you to invest? Is that right?
Emanuel Mueller Ramos: Yes, exactly. Because as I did the diagnosis of the group, one of the main feedbacks of our customers were we are using these tools, we are using these libraries, but is this like what's going to help us the most? What about other problems that you might have? What about other things that you could focus? And this was also like an instigator for us to look about other opportunities to explore.
Abi Noda: Very interesting. I want to ask you, do you remember a particular conversation or meeting, a specific moment, where you felt like the ideas you're sharing with me really clicked with a particular stakeholder or group of leaders? I mean, can you take us in the room and share how one of those conversations went?
Emanuel Mueller Ramos: Yes, of course. So when I started to realize that there was a need of change that we needed to go there, I started that phase of influencing my stakeholders and started to get buy-in about what you are going to do. I'm a very methodological person, so the way I went for is two phases. First instigating the needs of change and then actually starting to suggest concrete changes on it.
So let's go into the first part, the part about instigating the change. I started by bringing together my different stakeholders. Of course, our key decision maker ultimately was like our CTO, but that included as well different members of our organization. And I tried to have a representation of members of all the product engineering groups, senior leaders that had needs or that have gained feedback in the past that they needed different things. And as I start to communicate with them getting their insights, I start to build a case explaining, look, these are the areas that you're focusing and these are the value that you're bringing. I start to map that to the different challenges, the different areas that were being valued and the areas that you're not exploring. As I prepared for that as well, before getting to the room, I did a lot of pitching and sharing from what other companies were doing. A lot of what I did in the background was doing research and thanks to your podcasts and several other areas that you can cover further, I was able to gather that data and start to say, you know what Uber is doing, you know what Netflix is doing and start to show them. So finally when you got into the room, they were already primed to some extent about all the opportunities that exist. So the conversation to some extent went smooth.
The first thing that I showed to them was that triangle that I mentioned showing the different types of works, the value they bring, the metrics that they cover and where you could get value. So I started to show to them, look, we could go here in this direction and this would be the value you are going to get. Here you could go in that other direction and the value would be. Just to particular concrete terms.
The directions that I was considering was either being a developer experience group or actually go to completely the other flip side, like saying we are central capability group that we're taking parts of the common capabilities that the product engineering tribes use and doing it ourselves. And I start to contrast them and showing this is what you're going to go if you go to the left and the right, and we need to decide because if you're doing just both, we are going to dilute our impact and you are not going to get neither a good developer experience, neither a good set of capabilities. So that's the first part. I got a very warm, I guess welcome about the need of discussing it.
The Skyscanner leaders are very open in terms of learning and growing the organization, bringing knowledge from outside. They started to, for example, suggest people that they could reach out connecting their own network so that they could also discuss with other companies about what they are doing. So that opened the doors to increase the network and learn more, and this allows us to then move to the next stage that would be actually pitching for the DevEx.
So moving to the second phase, so how we went about pitching the DevEx. Once I got the thumbs up that there was a need of change, I started to create an internal team to do studies about how we can decide if you really should go for developer experience or the other around, a central capability group. The way we did it was to investigate different pros and cons, what benefits each other would to bring, what would be the needs in terms of headcount, what would be probably key metrics or key problems you would be solving of course at a higher level because we would need later to dig further if you wanted to have a concrete impact.
But all those started to help us to have an idea and decide which area we would go. And finally, after all these studies, which took more or less a month from the moment we instigated the need to change, I came back to my leadership group, that same group that I reached before, and at this time we had that recommendation to some extent of going for developer experience.
How that go? The reason that you are explaining the need of that is as I mentioned earlier, the multiplicative effect. Building a group of capabilities there is that much that you can do in particular if you're speaking about four teams, which was the size of the group. You can take maybe one, two, three or four capabilities, but the company is going to have way more need of other capabilities. So you do not be able just with that group to bring that big multiplicative impact. While in the DevEx, it's that idea about teaching how to fish instead of giving the fish. We can help the engineers in the organization by working different initiatives and making them do better to build their capabilities better so they can accelerate.
Another thing that I tried to leverage a lot was the industry trend in terms of most of the companies these days that I learned in my research were more selecting the DevEx path than other paths. So I could pitch to them and show, look what the other companies did, look what the metrics that they moved and this is what you could do. I have also in my portfolio some examples of quick wins that we did during that one month about areas that you could accelerate, which I could showcase concretely. Look, we just during one month scratched the surface and here are already some wins you could get. Imagine that if you were to fully embrace this direction, how much more you could bring. And that's how finally they understood and gave the thumbs up for us like, okay, let's move to that and so on. So that's how we finally got their thumbs up.
Abi Noda: Well, I appreciate you sharing the in-depth process and I'm grateful that we're having this conversation shortly after you've gone through all this so it's all fresh on your mind. It's such an important, such a difficult challenge for a lot of leaders to navigate through this change management and getting buy-in from stakeholders. So hearing how you approached it I think is really valuable.
Just earlier this week I was on a call with a large organization getting their developer productivity initiatives up and running and we had a 30-minute discussion about just what to call it. Should we call it developer productivity, developer experience, engineering enablement, foundational frameworks, engineering effectiveness. I'm curious, you're obviously using the term developer experience here today. What was the process? Was there any deliberation around what to call this thing or was it pretty clear to you from the onset that it should be called developer experience?
Emanuel Mueller Ramos: We had a bit of a debate if you should call ourselves developer productivity or developer experience. And we were really on the fence in that. Even these days, to be honest, sometimes some stakeholders ask me is what you are doing developer productivity or experience? Because there is a little bit of a nuance between them.
Ultimately when you discuss about it, I think it comes also to Skyscanner values. We have one of the values in our company that you call, we care always and I worry that if you were to call productivity, that's a very hard matrix. It's very like how you say, it's more a brains versus a heart thing, and why you experience passes that value of the happiness of the engineers, their experience and their productivity indirectly. So by embracing developer experience, we know that, and you believe that, you're going to make it organization more productive but if you use just the term productivity that brings more that pure mind focus versus the heart focus that you want to bring to our engineer organization.
Abi Noda: If it's any validation, I recall a recent conversation with Willie Yao who's currently the head of infrastructure at Notion, but before that he was the first developer, productivity leader at Airbnb, and I remember him saying the exact same thing that they called what they did developer experience, although those terms are interchangeable and there was a lot of debate. Because developer experience just has a better connotation to it, especially for developers, whereas productivity feels a little bit more managerial, maybe a little bit more Big Brother, corporate, wants to extract more productivity out of you.
Emanuel Mueller Ramos: Exactly.
Abi Noda: So yeah, your rationale resonates, certainly. I want to ask you, I know that apart from getting buy-in and making this organizational structural shift that shifting the culture within your organization, there's been a lot of challenges and interesting problems to navigate there as well. One thing you've mentioned to me specifically, and I wrote this down from our previous conversation, I wanted to double click into it, but you told me that you're trying to drop historical baggage from being a frameworks, middleware organization. So what do you mean by this? What do you mean by baggage?
Emanuel Mueller Ramos: Yes, so because you are an existing group, we had years and years of different tools and frameworks that you've been building, and to some extent most of them are important. We are never doing things that do not bring value to the organization. That being said, we need to start to shift both how we tackle them, how you are moving them ahead, but also invest in other areas.
Let me explain my mental model. We've been, by the way, adopting the DevEx framework as our main way of defining our strategy and our metrics. And in the DevEx framework, as you have the three pillars, the cognitive load, the feedback loops and the flow state, I believe that the frameworks and the libraries were very, very focused on the cognitive load and there was a need to shift to the other areas. We are not exploring enough how, for example, to improve the feedback loops or the flow state.
Let me give you an example, that was like the early wind that I mentioned to you when we were trying to get the buy-in stakeholders. When we started just to look a little bit further in our quarter review process, the process of merging pull requests and so on, we saw a lot of opportunities about how the process of automated bot-created requests could be merged.
We looked to the data, for example, we start to look to the data instrument, the applications, and we saw that it took 150 hours in average for, sorry, not average, to get those bot-initiated pull requests to be merged. And one of the reasons we realized is that we are basically creating so many pull requests in an automated way. These were like minor patches, library updates and upgrades. So what was our early wind, we started to work in improvements in our tools that would bundle the pull requests that work for the same component or for the same service.
So instead of saying you have 10 minor versions and let's make 10 pull requests, we create one bundled pull request and now the engineers, instead of reviewing the 10, they would review one. And we thought even going further, for example, can if the change is mine or actually automatically merge leveraging our CI-CD and building confidence that those screen safely being merged without interaction of engineer if it's a minor change.
This is an example of feedback loops. Why? Because if you automate that process, the whole process of getting ideation to production development and so on, that machine starts to turn faster, something that you were not exploring before. If you were to focus on the libraries and frameworks, you never thought about going to this area. So this is a complete example of how investing in new areas and maybe investing a little bit less in our historical baggage brought value.
Abi Noda: Another part of this shift, the cultural shift, which you've described to me is bringing more of a customer focus. How are you leading that cultural shift within your organization? How do you become more customer focused? How do you instill that in your teams?
Emanuel Mueller Ramos: Yes. By the way, sharing a bit of background, before joining this group, in my whole career I've been working with the product engineering tribe, serving travelers, serving airlines and our final customers. So when I got to this group, that was one of the first things that I tried to instigate. I would use again and again the word customer with the teams and start to explain to them we needed to bring a customer mindset. And one of the reasons that this shifting culture was needed is that the engineers that were part of this group, they were the same engineers that were part of the product engineering tribes, meaning that their ways of working what was their main interface?
It was let's say the backlog, the set of tickets. It was not necessarily working with the engineers and that they didn't have that background or that habit necessarily of constantly looking, embedding with the teams, discussing with them and seeing their problems firsthand. Of course there would be surveys, there would be some techniques to get that feedback, but dialing up, that was a necessity. And this is some of the things I started to invest.
I started also to identify volunteers, engineers that actually really like to do this kind of job. In my opinion there are some engineers that naturally are going to have a preference to focus more in helping other engineers, going to them, speaking with them. And there are other engineers that prefer to be more secluded and work with things like their backlog and so on. And both are okay, both make sense. And it is actually one of the reasons that the way I organize my group is that I create one pillar that I call more the golden pass, which is about adoption of standards and so on where the work is a little bit less customer facing and another part of the group that I call more the software delivery efficiency, which is about the feedback loops and flow state where we look for engineers that are more interested on that.
And the way I started, because you are new, is this new area that we're investing let's look for volunteers, the engineers that are really interested on that and here's an opportunity, how can you help us? And I'm starting to navigate and getting engineers on this area that are interested and they're bringing others. And I hope that it's going to create a positive flywheel as these engineers start to show the wins they bring, they are going to create inspiration for others and then more people will be willing to join the group and naturally we are going to have that second part of the group growing through time.
Abi Noda: I know that as you've spearheaded this developer experience organization, you're still part of this broader platform organization. Today in terms of how you're thinking about your work, what is the overlap between what the platform teams are doing and what your teams are doing?
Emanuel Mueller Ramos: Yes, historically the two groups were very close together, so there is a lot of intersection with them. For example, areas such as internet developer portals were initiated by the platform team or for example, the areas such as the CI-CD were initiated by them but we see a need of developer experience to affect some of them. For example, like internal developer portals is one of the main examples of DevEx, or the CI-CD, how that improve the direct groups, again it's a clear example of that. We had a lot of conversation between the two groups and the way we are thinking about is what are the metrics that we are tackling?
Let me illustrate what I mean. Taking the CI-CD, in the platform world, the metrics or the responsibilities that they are going to be owning is how stable is the CI, how available is the CI, how fast is the CI? But they are not necessarily going to be looking the actual usage of the CI in the different teams. This is where the DevEx comes. We look to the angle of how productive the things are with the CI, what are the bottlenecks, how flaky they are. And there is the thing about going to the next level of detail.
So for example, you may have a team that the CI is very flaky because their tests are not operating well. There is some unpredictability in their tests or could it be that they have long build times because they didn't optimize the application? The platform thing is not going to go to that level of detail while developer experience will. So both can coexist because they are attacking the same problem from different angles with the different objectives and metrics. So that's how you are finding our way to balance that things out.
Abi Noda: And what does that collaboration look like? If your teams and the platform teams are almost two sides of the same coin, how are you interfacing, collaborating, assigning work or taking on projects? How's the division and the collaboration there?
Emanuel Mueller Ramos: This is a little bit early days, but the way we are going towards is to outline the metrics. We are really like metrics craving in the sense that what are the metrics that the platform team wants to own on the CI and what's the metrics that the DevEx team wants on in the CI? And if your project drives the metric let's say of how fast is the feedback loop, that goes to the DevEx. If the metric is, let's say the availability of the CI that goes to the platform. So this is how you are operating.
This is actually part of some of the research we did. As I started to work and look towards the industry, we worked with several leaders as well in other areas, one of the recommendations was the idea of value stream mapping, understanding how the different interactions in a system between the actors, the nodes and the transitions take place. And when you look to doing value stream mapping, we are going to very strongly identify bottlenecks and the metrics that you can affect. And the feedback that you got from this leader was the metric drives the responsibility and that's the way you are thinking about to do that separation.
Abi Noda: That makes sense. In terms of metrics, I know it's probably still ongoing, but there's a lot of research you've done into this space and is a frequent area of discussion on this podcast. I know you looked into DORA space, DevEX, where have you currently landed in terms of how you are defining the North Stars and what you'll be tracking?
Emanuel Mueller Ramos: We did a very thorough assessment if you wanted to use DORA `space or DevEx and actually your podcast was a good source of knowledge and inspiration, so you use that a lot in the end. In the end, long story short, we decided to go for the X. and what was the reasoning behind?DORA on one hand is a very no framework, it's actually very stable and there are several tools in the industry and a lot of people that speak about it. But a part that I worried about DORA is that it's a little bit more downstream oriented. It's about when you are already at release the CI-CD or the operating a system, which it has that roots from DevOps.
While from us in particular considering that you had data heritage about the libraries and frameworks, we needed to think about the development stage. And DORA can to some extent cover that indirectly by proxy because if you're developing where we are going to have a better release frequency, you have a better lead time, but you want something a bit more explicit. And that's where I found that the DevEx framework would cover it better. Because then you can cover really the cognitive load, which in my opinion the libraries and frameworks are a key player for it, a clear way to drive it. And together with the other areas.
And I think one interesting part is that it just doesn't commit to exclusion of DORA. For example, you can use DORA and lead time or release frequency as a proxy for feedback loops. So it's not like a choice that you should take the DevEx framework, you cannot take DORA.
The only parts that we decided to explicitly like let's not touch it now was space because we found that for our organization that's as new as ours, that would be a little bit overwhelming. Let's take something more concrete as a starting. Maybe in the future when you are rolling very well, let's expand and take a bit more ambitious, but we decided to start a little bit smaller right now.
Abi Noda: Yeah, well as someone who's worked with a lot of organizations on these journeys, I think it is natural to start with developer experience and then later broaden to more of the P word, productivity, if you will. And so I think what you're describing and how you're thinking about it are really insightful and also I think very similar to the patterns I've seen with other organizations.
Now as a leader and as an organization that thinks a lot about the users, your customers, the developers, how are you thinking about that in terms of metrics? Of course you've been focused on your charter and your success metrics, the cultural change within your organization, but when you think about these DevEx metrics and other metrics, how are you thinking about exposing or reporting or sharing or making accessible this type of data with the product engineering teams in addition to your platform organization?
Emanuel Mueller Ramos: We are still in early days in terms of concrete tools and things like this, but let's say once we have the metrics identified, this is going to be the key level to prioritize our initiatives. So different from today that we are going to be going towards the direction of saying adoption is our key metric, in the future, you are going to say, let's start this initiative and trying to have an answer of why. What is the metric between the three pillars that is going to move forward? For example, we have a project that is related to the foundation web frameworks that you are using in the company. In the past, that initiative already existed, but the way it was framed, it was more towards the lines of the framework that you are using previously is a little bit outdated. It means that you may be exposed to security risks if you don't upgrade it. Or soon if you don't do anything about maybe we're going to lose open source community support. So there is a need or a urgency to move there.
That argument and that metric, it's not like the strongest speech to get the organization to move ahead. So some of the things that you are doing now is to reframe it in the sense of the DevEx metrics. For example, if you adopt a new tool, you know you are going to have less cognitive load because now you are going to need to write less boilerplate code for example, or it's going to improve the build time, the new tool, so that it's going to improve your feedback loops. This kind of argument makes it way stronger and our hope is that when you deliver those projects, we are going to expose the metrics and show how they are moving those needles instead of just the adoption of it.
Abi Noda: Yeah. Well you are on such an interesting journey and navigating such large shifts within the organization. I want to conclude by asking you with where you sit right now, what's the biggest risk that you see and what's the thing that could potentially cause you to fail or fall short of your mission and goals? What are the biggest looming challenges or risks that you see?
Emanuel Mueller Ramos: I think one of the challenges that I'm trying to make sure is having the organization understanding very well the value that the DevEx brings. I still like working with my customers because of the change in mindset. For example, with one of my customers, when we mentioned about it, they said, "So does it mean that you're going to just be writing documentation and doing presentations?" And I had to explain, "No, no, no, that's a way more deeper. There is a lot of effort in coding and helping the engineers." So making the organization understand because you are going to need their support. We cannot do the DevEx without knocking the door of the teams in the other organizations and help them. So they need to be open and accept us as you are going to be supporting them. So getting the organization, the customers, to understand that change and helping us is going to be important.
The second one is finding a good balance in terms of the pillars. As I mentioned, we are shifting to do less about pure cognitive load and expanding the other areas. That means that sometimes we need to say no to some projects that would've purely impacted just the cognitive load. Why? Because that doesn't maybe not have the multiplicative effect. But setting those expectations in the organization to understand, look, I'm not doing this but in trade-off, you are getting this way larger thing, is going to be important. And I think the risk or the thing to navigate is starting to get wins in these new areas so that we can prove with speed that really that new organization is bringing value so that we can more and more expand on those areas and do a bit less of the other. So that's what I'm trying to navigate.
Abi Noda: Awesome. Well Emmanuel, I loved hearing about the early stages of an exciting journey and wish you luck. Thanks so much for coming on the show today, sharing your detailed experiences and insights on your journey. I know this will be really valuable to listeners, so I really appreciate your time today.
Emanuel Mueller Ramos: Thank you so much.