Guest Goerge Stocker cuts through the often polarizing debate about Test-Driven Development (TDD) and offers his view on when the practice does and DOES NOT make sense, based on technology as well as human factors which are often overlooked.
Solving big problems with small teams
Voiceover: Ladies and gentlemen, the Tiny DevOps guy.
[music]
Jonathan Hall: Hello, and welcome to another episode of Tiny DevOps. I'm your host Jonathan Hall. Today, I have with me a guest, George Stocker who is-- well, I'll let him introduce himself, but I know, among other things, he teaches TDD and so I'm here to have him today talk about TDD and some of his nuanced takes on the practice. Hi, George. Welcome. What can you tell me about yourself?
George Stocker: Thank you for having me, Jonathan. George Stocker, I've been in technology for around 15 years or so. I've been in all sorts of industries, large and small and all sorts of teams, large and small and that's from where my experience with trying to help a team scale come in. One of those ways is test-driven development. It is a tactic but it's a rather nice tactic if you buy into it.
Jonathan Hall: Before we dive in too far, maybe you can define for us and for any listeners who might be confused, what is TDD because I know there's a lot of opinions about it, what it is and what it isn't. How do you define it?
George Stocker: TDD was pretty poorly named, it says, the TDD is test-driven development, and the idea is that your view of the system or what the system should be should drive how the system actually exists. In this case, you write a test that defines a piece of behavior in your system, and you make sure that test will fail. Then you write the amount of code that will take that test to pass, and then you iterate on that.
There's a lot of nuance in those three little rules, and that's where a lot of the misconception and problems that people have had with TDD come forth is because they either liken it to unit testing, or they liken it to implementation testing instead of behavioral testing. The differences, of course, every morning I drop my daughter off at daycare. The behavior is she gets dropped off at daycare.
How I get there is the implementation. That can change and that should change as my circumstances change, but the behavior that she gets to daycare is the thing that does not change. The test should never break as long as that invariant behavior holds true. From there, that's where we get into the issues that people have with TDD is sometimes we get focused on the implementations and not the behavior we actually want to define in the system.
Jonathan Hall: How did you learn about TDD? You said you've been doing it for many years but what was your experience? How did you go from say completely ignorant of the topic to an advocate?
George Stocker: This was the late 2009, early 2010. It was the rise of software craftsmanship. You may remember Software Craftsmanship North America being a conference, and I had a mentor who had been there and drunken the TDD Kool-Aid and was starting to-- It was introduced into the team that I was on, I had the mentor, we started going through it and I said, "Okay, this is neat, this allows me to work in much faster feedback loops because you're using the test interfaces, actually now your UI."
It's the thing that gives you feedback, not running the entire system end to end, which is nice, much faster but also I was running into the same problems that everyone else has run into, which is as soon as the tests get to any appreciable size or any parts of the system start communicating with each other, these tests are now really hard to maintain.
That's where I hit the plateau with TDD. I said, "This is nice but this can't work for large systems. It just can't." The technical reason is that, when you get to larger components that communicate to each other, they're communicating and you need to mock them out if you don't want to deal with their long-running processes and once you start getting into mocking and stubbing, you're actually mocking and stubbing implementation behaviors.
You're mocking things the system does to accomplish that overall goal instead of accomplishing that overall goal. That's what ends up making TD tests unmaintainable at a certain size. For a long time, I didn't think there was a way past that. I just thought that this is it. This is where we're stuck. If you sign up for TDD, you are signing up for the pain of mocks and stubs and nobody wants that.
Jonathan Hall: No, no. Do you always use TDD?
George Stocker: No.
Jonathan Hall: In your own coding, and how do you decide when to and when not to?
George Stocker: I don't even use it most of the time. I use it where it makes sense, and so one of the precepts of TDD is that you are testing the behavior of the system. Well, that means you have to know the behavior of the system. It means you have to understand what you're trying to do. A lot of times if you're working unknown, for instance, when I started with firmware, no idea, there were multiple components and multiple systems that I didn't understand.
You had the BLE radio, you had the OS that we were running on top of, you had all the lights and the haptic motor that had its own events. I didn't understand the behavior of the system at all. If I had tried to do TDD from there, I would have probably spent months hitting my head against a wall as I learn new behaviors that I had to account for.
What I'll end up doing, is I will end up throwing the fishing line out there and seeing what I catch. I'll try something in a new system, see how the system reacts and then iterate from there, and only when I have an understanding of the problem I'm trying to solve and the behaviors that I want to see in the system, do I actually go with TDD? That comes down to if you don't know what you're trying to do then the test don't matter. It's, you've got to figure that out first.
Jonathan Hall: I hear some hardcore TDD advocates, some very famous ones saying things like it is irresponsible to write production code without using TDD. Do you agree with that or can you debunk that for me?
George Stocker: Anytime anyone tells you anything as you're responsible without knowing your exact context, you can safely ignore them. That's one of the issues that I'm trying to help resolve with TDD is that idea that TDD is a craftsman tool or that to be a professional, you must use TDD. No, absolutely not. It is a tool. It is a tactic. It has a set of times and a set of circumstances where it is appropriate. It has a set of times and a set of circumstances where it isn't appropriate, and production code-wise, I can think of lots of production code that shouldn't be TDD.
Most notably if you were writing, let's say, a service that interacts with a third party and you don't know what that third party does or even if you do know, does it make a whole lot of sense to put more effort into that system than you're getting the value out of? In our case, for instance, if we're talking with Twitter's OAuth API.
No part of me is going to be writing TDD-based tests that ever touch what it's doing. If I have to do something really complicated on my own on the inside, perhaps, but if I'm really just saying, "Hey, I'm connecting the Twitter API, and I'm turning this data into that data," I'm just going to write an integrated test for it, or I'm going to write a contract test.
Maybe not integrated if I have to run it through the CICD a lot and I have to actually depend on Twitter. I will write it an integrated or, excuse me, a contract test that will say, "Hey, I am pretty confident that Twitter will always return these values, and I'm going to make those invariants in my system." That's the kind of test I would write. I wouldn't necessarily if my interaction with something is purely at the grab this data and turn it into that, I don't see the reason for the ceremony around that, around using TDD.
Jonathan Hall: Makes good sense. Just briefly, I know you could talk for hours about this, but briefly, when should you use TDD especially if you've never done it before, when should you consider, "Maybe I should try to learn TTD for this?"
George Stocker: TDD is a team sport. Software is a team sport. If you're in a professional situation, you're like, "I would love to subject the rest of my team to this new thing that I barely understand," I commend you. However, being that there are other humans that you're going to have to interact with, maybe make sure the context of the team is right for it. Maybe make sure that you've got buy-in and real buy-in, not just the, "I'll be quiet and just nod my head yes, buy-in."
From a team perspective, introducing TDD means getting people on your team to buy into it. That's not going to happen generally unless you're very charismatic if you just say, "Hey, there's this new thing we're using it." TDD-driven tests do need to be maintained. Everybody's buy-in has to not just be, "Yes, we'll do this but also we understand what we're getting into." We are now not unit testing implementations, which is what you do typically with unit tests. We are now test-driving the behaviors we expect to see in our system. Everybody has got to be mentally bought into that new concept, and that change of how they operate. If you're talking on a team-wise, then you've got a lot of people work to do. If you're talking just you as an individual, there are several good, they teach you the tactics behind TDD and they call them, in this world, we call them katas which, unfortunately, there are a lot of analogies that we use that deal with martial arts, and that's one of them.
If you look up a TDD kata like the bowling game or Conway's game of life, those will happily teach you the mechanics of TDD, but there's going to be a giant gap between that and using in a production codebase. That's part of why I'm putting together a course on it. Part of how when I am teaching companies how to scale their technology and they ask about TDD, that's where we talk about, "Okay, let's actually teach you TDD according to your codebase because your context is a lot different than an internet kata."
Jonathan Hall: Definitely. I recall when I learned TDD the first time I was introduced to the conference, I ran home thinking, "Oh, I'm so excited to try this thing." I tried it. It worked in the technical sense, but then I found it to be so worthless because of one thing, the rest of my team wasn't doing it. For the other thing, I knew how to go through the steps, but I didn't understand the context of where do I use this practice in my daily work.
I became a TDD hater for several years before. I again later came around and it took me months of-- and this was working by myself. This wasn't even trying to convince a team. It took me months to get to the point where I felt confident and comfortable and that it was a benefit rather than something getting in my way. I can imagine trying to accomplish the same thing if I have five other people I have to convince at the same time.
George Stocker: Yes. That goes back to when we last spoke about the plateau, I'd hit a plateau with it. I was just like, "This can't be used." I encountered, I actually went to, it was PyCon 2012 I believe in Santa Clara, California. It was a Python conference of which I was learning Python at the time. I was like, "Hey, I'm going to go to this Python conference." By the way, wonderful community, love the Python community.
Gary Bernhardt, his site is destroyallsoftware.com, but he gave a talk on boundaries, and basically, his talk revolved around the fact that TDD either, inside out, which is go from the most interior component of your system and work outward or outside in, which is work from the user and come inward. They both have issues when you get to try to scale using TDD over anything that's basically trivial. Both of those methods either lead to over architecting in the case of inside out, or in the case of outside in, you've got too many couple of dependencies to how things actually work, so you over-rely on mocks and stubs.
His method, which he deemed a functional core and imperative shell basically says, when you have to deal with the outside world, you wrap that nice, little imperative shell around your actual system. That reduces the number of inputs into your system that you were running that is created entirely through TDD, the functional core, and that you use certain functional principles. You stay away from object mutation, being one.
You make sure that the types you're using are your types and then that way you've insulated yourself from the outside world where it needs to be and where there is an outside world that you've actually got to test, and you've got to deal with, you don't do those through TDD. You might use integrated tests. You might use contract tests. You might use end-to-end tests, but you don't do that part of the system through TDD. You're giving yourself a compromise option, which can work for larger systems where traditional and classic TDD will fail.
Jonathan Hall: For somebody who's completely new to TDD, or maybe they've tried and failed like I had a few years ago and like maybe they hit that plateau you've talked about. Obviously, we could subscribe to your course, and we'll have details for that later on. What are the steps that somebody can take to overcome that plateau? You just talked about a new paradigm of sorts for thinking about your code, but what are some concrete steps that somebody can take to overcome that plateau or to dip their toe in the water for the first time?
George Stocker: It comes down to asking the question of the thing that you're trying to accomplish in your system. What is the behavior I want to see? Not how am I going to get there, but what is the behavior I want to see in my system? That's the first step is decoupling the imperative steps we like to write down as programmers, do A, then B, then do C, then do D, to, "No, I actually want to send an email." That's the process, send an email. We will have known that this thing works if an email gets sent, and we will know it hasn't worked if an email does not get sent.
Steps A, B, C, D, and E might be implementation details in that system. If they're behaviors, like we want to make sure that we were sending it to a valid address, well, that's a behavior, validate that the address exists. The first step is to switch our mind, thinking from what are the steps I need to take to what are the behaviors I need to see in my system?
Once you get to that and you're going to hit your head against that one a lot, at least I did, is that it's easy to specify the implementation. It's hard to say what if the implementation didn't matter? What is the magic one thing that I could do to get to the end result? That's the behavior, and then how to test that.
You're going to spend dozens, dozens, dozens if not hundreds, and hundreds, and hundreds of hours just dealing with that. Once you get past that, and once you get a good eye for that, then it's, "How do we interact with the outside world?" You have a decision to make. One of the pitfalls is trying to use TDD for everything, it is not meant for everything.
If you find yourself saying that, "Hey, I hate using mocks and stubs," that's a personal preference, then don't write your system in such a way that functional core and imperative shell that I talked about earlier, where you don't have to mock out parts of the system you don't deal with. You don't have to mock out the database.
When the world enters your system, you have a user. You don't care how you got the user. Stuart could have dropped it on the keyboard for all we know, you just have a user. When you are sending a message, you send the message. You know that you sent the message, whether it gets to the other end and what they do with it is their a business, but you've sent the message.
When you start to, in your mind, mentally shut down the parts of the system that don't actually matter to you at that moment and stop worrying about them, then you're able to focus on the particular pieces you need. Too many times as programmers, we try to see the whole world, and try to fix the whole world instead of fixing that little part that's right in front of us.
We start to worry about, "Oh, how am I going to abstract this away?" Or, "What if the email gets sent, but gets bounced back?" Those are other people's problems at other points in time. Your problem at this point in time is sending an email. That way, once you've got your thinking to where you are and what you're doing, to borrow a Yodaism, then you're able to focus, stay in the moment, and write just enough code to do the thing that you're trying to do.
From a most of writing TDD successfully in a team, part one is that mindset shift, is that personal mindset shift. Part two is making sure the system models that mindset shift. If you're going into a system with no tests, you may end up the first thing you're doing, of course, is putting the system under some sort of test, even if it's just an integrated test. It's also starting to see, "Okay, what are the behaviors in the system that I can break out? What can I throw a behavior into and I might throw that into a class or a method that has several other methods that are doing things, but where can I throw a discreet start and end to the behavior in my system, and where can I now focus my test on that discreet start and end behavior at the state at the start, the state at the end and the behavior itself?"
That gets you a lot of the way there, but as I said, most of it is a mindset shift and a change to how people actually work shift. That's one of the reasons that I think that TDD has not been adopted writ large is because we have not been able to successfully market that mindset shift as actually being valuable, more valuable than taking steps A through E.
Jonathan Hall: I agree. Like you said at the beginning, TDD isn't really about testing. It's named poorly and that's put a bad brand, if you will, on the practice, I think because people think, "Oh, I need to compare a TDD to my other variety of testing practices I might choose from and making a false comparison."
George Stocker: No. If you say TDD and unit testing in the same breath, take a step back and realize they are different, they exist for different reasons. If you start to say, well, unit testing can replace QA testing or end-user testing or integrated testing or end to end testing or contract testing, take a step back. All those different parts of testing of which TDD is not a testing strategy. It looks like one, if you squint enough, you see one, but it's not a testing strategy.
All those other testing strategies have their times and their places and they can't replace each other. A 100% unit tested system that's under a hundred percent code coverage can never replace one QA person. It won't. We see those memes on the internet unit tests to integrate a test zero when you put two parts of the system together and something unexpected happens, but that's where the humans are.
They're the ones that try that unexpected thing that we as programmers when we're creating the system can't even conceive of. No testing strategy can subsume all the other testing strategies. They all have their time. They all have their place. They all have their reasons. They're all complementary. TDD is more of a-- it's architectural. It is an architectural strategy for your system. "How do I get my system to work together without being coupled to different behaviors and coupled to implementations?" Or, "How can I build on and allow this part of the system to be easily changeable in case we learn something new?" Which we do every single day.
TDD, poorly named, there are tests, not a testing strategy, it's a development strategy, it's an architectural strategy, but it can't subsume and it can't replace all those testing strategies that we have.
Jonathan Hall: Suppose that you're on a team, maybe you've learned TDD and you've done it well on a previous team, and you've changed companies, or you've changed teams, and you've joined a team that doesn't do TDD. What should you do? Should you give up? Should you try to convince them? Should you do it by yourself and not commit your tests? What are the options? What should somebody do in that situation?
George Stocker: It all depends on whether-- It actually depends on the topology of your current system. If your current system is a monolith, you're going to have a different strategy than if your current system is based with service-oriented architecture or as the kids like to say, microservices. It all depends because, in a microservice, the really nice thing about that is you're isolating your code deployability from the rest of the code so that any changes you make, if you add tests, and no other service has tests, it doesn't matter to anybody. No one cares. As long as when they're writing code in your service they don't break the tests and no other part of the system actually cares.
For there, if the rest of the team doesn't use TDD, I'm new on the team, and I think, "Hey, this would really help us." For one thing, I probably wouldn't bring it up right away. I would just quietly start to write my service with tests, test-driven. Then when I'm working with someone else on the service that I'm responsible for, showing it to them.
It becomes down to, if you go to a team that doesn't use TDD and you do, and you want to get the rest of the team too, you're going to work on your charisma. You've got to work on not selling it, but having people understand how it solves problems that they themselves have. It becomes learning how people who are on the rest of your team, how they think and what problems they're actually having, but will absolutely not work, and hasn't worked yet that I've seen as a top-down where we're going to use TDD without understanding, and without those mindset shifts that everyone needs to make.
Jonathan Hall: Absolutely.
George Stocker: For the last 20 years, we've been very focused on software delivery as a set of processes and techniques. What we've not been as focused on is that there are humans involved, and it's not about the software delivery, it's about the team making the software. When we try to sell these processes and these techniques, we're trying to sell the actual techniques themselves, we're not taking into account rather the team and the people and where they're at in their software journey, where their business is at, the context around the business, the context around the team, the architectural context, the financial context of what they're dealing with, what deadlines do they have? Why do they have those deadlines? Their customer context, all those little contexts that we ignore when we espouse our favorite technique or our favorite paradigm. All those contexts are absolutely crucial to understanding what set of tactics and what strategy would actually work for a team.
Jonathan Hall: Let's say that the team you've joined has-- they're all on board. They've all watched this or listened to this podcast. They're all onboard with learning TDD. How long can we expect before they start seeing a benefit from this?
George Stocker: That's tough and that's team dependent and that's topology dependent. That's one of the reasons why I've shied away from teaching TDD. I teach at the mechanics and I teach at the mechanics of functional core and imperative shell through application that I use, but if you're actually going to bring it to your application, to your context, then however you learn it has to be in that context. It has to be, it has to work in that context.
For instance, if you're a heavy ETL-driven system with data warehousing, there's going to be a separate set of strategies than if you're just a simple crud app. From how fast will a team be productive with TDD in the local sense, pretty quickly, but is this maintainable for the system I'm in since? That will change based on the team and based on whether or not you were successfully able to-- everybody on the team has the mindset shift and already has the answers or at least understands what direction to go to when it comes to communicating among other systems.
That's the part that varies from team to team, is once you get-- everybody gets the basics down pretty quickly, but then it's the, "Oh, wait a minute. This system was developed in this way, and now I need to mold it this way." Or, "Do I have the time to do that?" Will the manager guy or the manager person, the PHB, the point-headed business person say, yes, "Yes, you can do that." Or, we'll not listen to him and just do it anyway. All of those things are team-dependent.
Jonathan Hall: Great. This has been an informative conversation. Is there anything else you'd like to add before we bring this to a close?
George Stocker: Yes. TDD is one tactic. There are lots of other tactics that you can use. In the video that Jonathan-- and I can link to in the show notes, I gave a video on is TBD right for your team? It lists a lot of other tactics and strategies you and your team can use of which TDD can be one or of which it doesn't have to be. The most important part for your team and the people listening to this is that you choose the tactics and strategy that fit your contexts. That won't be the same as everyone else everywhere.
Jonathan Hall: I love that. There's so much dogma out there about you always have to use TDD or TDD is always evil. I like talking to somebody who takes a nuanced approach. It's a breath of fresh air. Thanks for that. Do you want to share any resources? We'll definitely link to that video. Are there any other resources you recommend for people to follow up on?
George Stocker: Yes. You can reach-- I spend a lot of time talking about this on my site at georgeStocker.com. There's a mailing list that you can subscribe to. I talk about this and other sorts of tactics and strategies you can use to scale your tech daily-ish, daily as I'm able to.
Jonathan Hall: How else can we get ahold of you then, George? Are you on social media?
George Stocker: I am. I'm at Gortok, G-O-R-T-O-K. I would like to point out that World of Warcraft stole that name from me. Mine was there first. There's a story behind that, but on Twitter-- I'm on LinkedIn, of course, at GeorgeStocker. My site is georgestocker.com. Cleverly named.
Jonathan Hall: Very clever. Great. Well, thank you so much, George. It has been a pleasure. I hope that the listeners have found a new perspective on TDD, and we will follow you on Twitter and watch that video.
George Stocker: All right. Thank you very much for having me, Jonathan.
Jonathan Hall: Did you know you can watch this episode and other Tiny DevOps content? Search YouTube for Tiny DevOps to see all of my guests' beautiful faces. My thanks to Riley Day for the Tiny DevOps theme music.
[00:29:32] [END OF AUDIO]