Adam Hawkins presents the theory and practices behind software delivery excellence. Topics include DevOps, lean, software architecture, continuous delivery, and interviews with industry leaders.
Hello and welcome to Small Batches with me Adam Hawkins. In each episode, I share a small batch of the theory and practices behind software delivery excellency
Topics include DevOps, lean, continuous delivery, and conversations with industry leaders. Now, letâs begin todayâs episode.
Thereâs a phrase thatâs coming up more and more in my daily work. It relates to one of my favorite questions these days: How do we know what we know?
This is a powerful question because it summons facts and knowledge about the current condition. We call that a mental model. We can test the mental model immediately in the real world.
The challenge is getting to the quantifiable and empirical facts. Thatâs where this phrase comes in.
That phrase is: as measured by. Hereâs how to use it.
Letâs begin with an example from my day job as engineering manager for an SRE team.
We run a golden signals workshop with ourselves and other teams. The aim is answering this question: how do we know the system is working? Answering the question creates a mental model for operating and troubleshooting the system.
The workshop consists of creating tier one and tier two signals for each of the golden signals. Quick refresher: the golden signals are latency, errors, traffic, and saturation. Also, link in the show notes for a previous podcast episode on the golden signals.
The tier one signals are candidates for monitoring, use in SLIs, and the stuff to chart on the teamâs primary dashboards. The tier two signals are for supplementary troubleshooting and diagnostic dashboards.
We encourage participants to throw stickie notes onto a Miro board, then work to refine them all into a coherent mental model. Hereâs how the phrase âas measured byâ drives that exercise.
Letâs say someone adds a sticky note for âHTTP Requestsâ under âTrafficâ. Great start, but no where near precise enough. This phrase is wonderful because it helps newcomers navigate operations and systems architecture.
This leads to questions like: âCan you show me where HTTP requests are measured in this system diagram?â If they point to the application and not the load balancer, then thatâs an opportunity to explain the gap in understanding. Itâs a chance to probe the path requests actually take to become âtrafficâ for that service.
Vague stickie like âHTTP requestsâ becomes one or more stickies like âTotal HTTP requests as measured by the load balancerâ and âTotal HTTP requests as measured by each application instanceâ. Now the mental model is more clear.
We continue asking these questions to refine the stickies. We know weâre done when they include âas measured byâ. That typically points to how and where to collect the telemetry in the system.
Thatâs my personal example. Iâll close out the episode with where I learned this phrase.
It comes from John Doerrâs book âMeasure what Mattersâ on using OKRs. The use of âas measured byâ notches in wonderfully there too.
OKRs stands for Objective and Key Results. The objective is a target-condition. The Key Results are how you know youâve met the target-condition. The idea is describe the what or where without the how.
OKRS typically read like this: I will (Objective) as measured by (Key Results).
Hereâs an example written by someone who wants to run a 10K.
Objective: Run a 10K in under 50 minutes by June; as measured by these key results:
1. Go for a run 3x/week for at least 30 minutes.
2. Increase distance of run by 1 mile every week.
3. Increase mile speed by 5 seconds every week.
Using this phrase guides you towards measurable and objective outcomes, instead of open-end and subjective assessments.
Here is a rephrased example from the book for this podcast episode.
Objective: Introduce the language of empirical thinking as measured by three key results.
1. Finish this episode in under five minutes.
2. Show two examples of the phrase âas measured byâ
3. Provide three links on empirical thinking in the show notes
All right thatâs all for this batch. Head over to https://SmallBatches.fm/90 for links to recommended self-study on empirical thinking and ways to support the show.
I hope to have you back again for next episode. So until then, happy shipping!