Subscribe
Share
Share
Embed
In this episode we talk with Joshua Greene and Michael Katz, authors of iOS Test Driven Development by Tutorials. In this episode, we talk about:
- Why is test-driven development is important for any dev team?
- How is TDD best employed with iOS development
- What are some exceptions to test-driven development?
Show Notes
Guests
- Joshua Greene - @jrg_developer - Senior Author for raywenderlich.com
- Michael Katz - @themikekatz - Senior Author for raywenderlich.com
Book - iOS Test Driven Development
Links
Previous Episodes
Show Notes
The Components of Test-Driven Development
- Keep Your Iterations Small
- Test First
- Circular Progress of Testing and Developing Functionality
- Avoid Testing other APIs (i.e. Integration Testing)
- Use Mock Data for Outside Dependencies
- Behavioral Driven Development vs XCTest
How to Encourage TDD in Your Team and Company
- Make sure Maintainability and Specs are Met
- Prevent Regressions with New Features
- Regular Code Reviews
- Tests Should Be Required Before Deployment
- Encourage a Culture of Testing
- Slow Add Tests to Projects Missing Them
Common Mistakes When Doing Test Driven Development
- Following the Golden Pathand not testing edge cases and errors
- Following Test Coverage too strictly or not enough
- Use Multiple Test Targets
- Write Tests First
For Experimenting with New APIs
- Use Spike Solutions to Experiment
- Write Temporary Tests
- Use Playground to Test Code
Continuous Integration Tools
- Jenkins
- Travis-CI
- CircleCI
- Xcode Server
- Fastlane
WWDC and TDD
- New Performance Tests
- Test Plan for Xcode
- SwiftUI, Live Previews, and Modeling
Social Media
Brand New Instagram! - @brightdigit
Full Transcription
Leo Dion (Host): So hey guys, how's it going?
[00:00:02] Leo Dion (Host): Good good, just been trying to catch up with WWDC. What's your general thoughts so far.
[00:00:07] Michael Katz (Guest): It seems like this is about the biggest one we've had since they introduced Swift. There's just so much stuff.
[00:00:12] Leo Dion (Host): Yeah, that's exactly what I was thinking especially with the UI changes. So have you started investing money to buy a $1000 stand?
[00:00:20] Michael Katz (Guest): Yeah, I wish. Fortunately at my company, we actually do video editing. We have editing bays already have racks of Mac Pros. So I assume that those will get upgraded with the new machines at the fancy monitor so I can go downstairs and drool over them when I'm not actually. It's working on my MacBook.
[00:00:36] Leo Dion (Host): Yeah, I mean that's the thing about these devices. They're not for developers like there for video editing like massive 3d rendering that kind of stuff and big production companies
[00:00:45] Joshua Greene (Guest): Maybe if you're doing like 3D games or something as a developer. I could see it being really worthwhile but for business or Enterprise type apps it may be a little bit of an overkill which is definitely pretty awesome. But day to day maybe not so much.
[00:00:59] Leo Dion (Host): Yeah, exactly. So if people want to tweet at us or we're also at brightdigit on Twitter at brightdigit on Instagram and Facebook. Let us know your thoughts on WWDC and any thoughts you might have about all the new stuff that's coming out.
[00:01:15] Guys, so apparently you have a book coming out pretty soon. Is that correct? So go ahead and introduce yourselves to let me know a little bit about this book.
[00:01:24] Joshua Greene (Guest): So my name is Joshua Greene. I am a longtime author for RayWenderlich.com
[00:01:30] I've done everything from creating tutorials to books to videos. This is a new project that we're putting together to teach test-driven development it's called iOS Test-Driven Development by Tutorials
[00:01:45] Michael Katz (Guest): And I'm Michael Katz and similar story. I've been with Ray Wenderlich for a number of years and also done books tutorials.
[00:01:53] Spoke at the RW Devcon. Haven't done any screen cast yet, but maybe someday and they are passionate about test-driven development and we both came I think independently to the idea of writing a testing book and editor-in-chief put us together. And here we are most of the way through it.
[00:02:13] Leo Dion (Host): So test-driven development, I remember once it was like almost more than half decade ago, probably almost 10 years ago, I went to a conference in Chicago and I think it was Bob Martin who gave a talk on test-driven development the idea being like first you write your test and then you write your code.
[00:02:33] Is that kind of the gist of it or what are the components of test-driven development? What exactly does that mean?
[00:02:39] Joshua Greene (Guest): Yeah, definitely writing tests first is definitely a big part of it. Keeping iterations small. I think test-driven development is all about, you know, writing one small thing you write a small test to implement something that you need to get implemented.
[00:02:54] You show the test actually doesn't pass you implement. Whatever code is required to get it the pass and then you verify it passes. And you just repeat this process over and over so that when you finally, you know, got your app out not only have you got all the features written you've got all of the tests that you're going to need alongside it but it's not like you just write all the tests in advance. It's literally a small step by a small step is kind of what makes it different and special.
[00:03:20] Michael Katz (Guest): It's a circle you just go over and over again at a little test and yet add some code to fix that test and you test the next piece and you add the code for that and so on so you're always in lockstep.
[00:03:31] Leo Dion (Host): So the problem I have had with test-driven development and iOS is some of the stuff that is pertaining to like the UI or perhaps like test-driven development, sometimes you have different screen sizes, obviously, you might have to deal with something like core data or networking. How do you overcome those challenges when you're trying to do test-driven development?
[00:03:57] Michael Katz (Guest): I think we think about using TDD. Is that it really forces you to think about what your code is doing? And so if you want to make sure that your code is testable and you're starting out with no code. You're going to write your code in a way that is so if you have a dependency on something external to the system like a database or an API or screen, you'll want to write your code, you know, where you have a model. For instance any test the models all the business logic and the data logic and so on and then you can add the UI on top of that and then use a different type of testing like UI tests to handle making sure that the UI is correct, but that way you're certain that your business logic and state is consistent and complete before you even get there.
[00:04:36] Joshua Greene (Guest): With TDD, even if you're following TDD from the get-go there. It Is not to say that you have to test everything per say. For example, if you're going to be caught by another their way like at the compiler is going to throw an error if you actually hit an error while you're doing TDD that counts as a failed test
[00:04:54] with the UI portion, I typically look at it as if it's something I can configure in interface Builder, I don't, you know, not necessarily going to test that too heavy in terms of TDD because it's something I can see on screen and might mention there's other tests that I can use if I really want to put test in place for UI testing and so.
[00:05:14] TDD is really about you know, structuring your code in such a way that it is testable putting in protocols instead of talking directly to a database for like Mike says, you know talking to models that are structured in such a way that you can actually put test where you get the biggest bang for your buck as far as testing goes.
[00:05:30] Leo Dion (Host): So it sounds to me like we're talking here about kind of this is similar exactly to what we talked about in our last episode. When we talked about architecture and the fact that you need to architect your app around models, and it sounds like you're basically talking about what specially when it comes to Swift, you know, doing protocol oriented programming and using mocks in your testing as opposed to directly connecting with things like user interface or your database or your network. Is that what I'm hearing?
[00:06:00] Michael Katz (Guest): Exactly? In the book. I have a chapter where we're building at a fitness app and instead of connecting to Core Motion we basically build a protocol around it so we can Mock and stub data and act as if these things were happening live, but they're actually just being faked by the tests.
[00:06:17] And that's the type of thing you would use for a pretty much any kind of Hardware or network or UI that you can do in your code. So we separate the logic and the pieces that are about the app that are unique from the things that are handled by the operating system.
[00:06:29] Leo Dion (Host): Yeah, that's awesome. And I think that's one of the things I really like about TDD is that not only is it a great way to make sure your code actually works in all cases but it also encourages healthy architecture in the end because you're allowing for different things to be plugged in like I need a fake core motion or health kit or core data or whatever. I can fake it in my test so that way when I test the actual application like you said those are different tests, they're not tdd necessarily.
[00:07:01] Michael Katz (Guest): It's particularly true when you're testing error condition. So if you have a logic in your code that handles detecting when a call is dropped and then when you reconnect to restart something in this kind of hard to test especially in a CI situation where you're building and testing your app on a server that's plugged into gigabit Ethernet.
[00:07:20] So by using TDD enrollment you've already separated out that networking layer. So you can just Supply errors and different combinations and ordering of conditions to test out that your app is going to handle those appropriately even if they're very difficult to test it manually.
[00:07:34] Joshua Greene (Guest): Even if you're starting from a point where it so you don't have a completely new application.
[00:07:39] It's not to say that you can you know, not introduced EDD later on, you know, a lot of these same principles of you want to talk through something else. Even if you directly coupled yourself to talking to a networking calls or to core data or whatever, you know external API is. You're talking to that directly.
[00:07:57] You can break those dependencies and you know move your architecture in the right way, even after the app is already, you know launched or had a few versions out. We actually have some chapters in the book that go into details about taking a legacy based app and moving it towards TDD. So. I'd say that tdd is a great start if you're just kicking off an app, but even if you're not you can still use and consider TDD just you know in sure you're writing those tests first and again, you know, just making sure that you've got the correct coverage and bang for the buck as far as testing goes.
[00:08:28] Michael Katz (Guest): Yeah, you can start writing test at any time in this way. So even if you have an app and even if you have tests that you've written sort of after the fact to validate your behavior when you write new code you can start doing TDD at any point.
[00:08:40] Leo Dion (Host): So what are some challenges that people typically face when it comes to developing like in Xcode for iOS that's different from say like your typical vue.js developer or PHP or python?
[00:08:57] Do you guys typically use the standard packages like the XC test stuff that comes with xcode and iOS, or do you use your own custom packages for mocking or what are some challenges that you've seen people face specifically in iOS.
[00:09:12] Michael Katz (Guest): I think Xcode default is he test case is a little bit cumbersome is very much based on the old Xunit or Junit type testing which is gosh has to be 20 years old at this point. Yeah. I think it's out of the 90s where it was a long time ago.
[00:09:29] Michael Katz (Guest): So my company we actually use a form of what we call behavioral driven testing. So it's more descriptive. We use of Frameworks called Quick and Nimble. Which is a layer on top of XC test, so you still run the test and xcode and you get the same reports out that you do but it's a different API and XCtest so it makes it you know easier to read the test understand but it also because it's a layer on top is slower than using actually test, directly so it's the trade-off there.
[00:09:59] Joshua Greene (Guest): I'm gonna have to disagree with you a little bit my in that I would say - yes Quick and Nimble is fantastic as far as making things more descriptive and so forth. But to me like it's more syntactical flavoring if you will then it is a strict requirement
[00:10:14] XCTest very much so used to be you had to have something in addition to it. You had to have some way to do like asynchronous testing the very first version of XCTest didn't really even provide a way to set up any sort of expectations. I'd say today though, if you really want to and a lot of my day-to-day stuff, I typically unless there's a strong reason why I like you're saying unless I need to do it and some sort of behavior descriptive way that I want to actually set upfront i don't necessarily opt for going to something else always just from the get-go. You can very much so use XCTest right out of the box and it is testing completes - well as complete as you can get with Swift and really having true mock support, but you can basically do everything with XCTest that you can do with Quick and Nimble.
[00:10:59] It's just like you said, maybe it's a little bit more verbose or maybe there's a little bit that you might do differently versus some of these BDD type Frameworks that give you some nice syntax.
[00:11:10] Leo Dion (Host): I've worked with like mocha for like nodejs and some JavaScript stuff. I think that's behavioral driven, but it sounds to me like Quick and Nimble gives you that like syntactical sugar in a good way to like explain what each test actually does.
[00:11:25] Where as XCTest doesn't give you that but it does give you the basic logic and API for doing just logical tests. Which is more or less what TDD is really driving at. Is that correct?
[00:11:37] Joshua Greene (Guest): Yeah, I'd say the XCTest doesn't give you the naming syntaxes or so forth or you know, like the nice thing about BDD and really the main difference, you know, but on the other end of the things I'm close to the plain vanilla here than Mike it sounds like.
[00:11:49] The nice thing about the BDD Frameworks like Quick and Nimble is it will provide you the ability to say okay here's a descriptive string describing what I'm actually trying to test here. You can do the same thing using method signatures and that's actually what we show in the book there.
[00:12:03] [That] is if you named your test correctly, you know describing what you're doing describing what is actually being tested in the expected outcome. You can get a lot of the same benefits from the just a vanilla XCTest. But like Mike said it's you know more so by naming conventions that you set up less so directly supported by XCTest itself.
[00:12:23] Leo Dion (Host): So I want to jump a little bit more into TDD and talk about some of the terminology and some of the stuff that's involved. So, we talked a little bit about like mocking what exactly is mocking and what's it used for when it comes to test driven development. If there's any other terms, I'm forgetting that are components of test driven development let me know.
[00:12:45] Michael Katz (Guest): Yeah, so. When you're running tests, there's a whole Suite of objects that are commonly called test doubles where basically you have an object. That's just for testing that parallels production code [or] main app code.
[00:13:03] So a mock is an object that behaves like the code. So for instance you have like a mock network connection or like an nsurl session. And if you supply a command to that like low data, your mock is responsible for handing back some set of fake data that you supplied to the test knows ahead of time what's going to happen and it's going to succeed or not with mocks in particular it is to verify that certain methods or behaviors happen along with that. So if you call we go back to the mock data URL idea that you would verify that the data was returned correctly or if an error was made if you set it to make an error that the error callback gets called and so on. You can also verify that certain methods are only called once or twice.
[00:13:49] So if you have a tertiary object further down so that when you save an object to a database you want to make sure that you're not saving that same thing twice so there's no duplication in the code. You can verify that using a mock by adding methods to count the number of times a method is called and so on. I 'm not sure that was clear but maybe Josh can help here.
[00:14:06] Joshua Greene (Guest): I agree with what you're saying there Mike the only sort of difference in my mind as far as Mocks vs. a test double there. A test double may not provide like verification that behavior like you verify that a sort of method was called.
[00:14:22] It may just accept those methods and just fill in as a dummy, you know, the simplest way to make just like a test double in my mind would be you know subclass something out or conform to a protocol and then implement nothing. And those are basically the two options that you have in Swift is you either implement a protocol to create a mock and then you can you know, just pass that to either be an initializer or you know setting it as a property.
[00:14:46] Or you can subclass something else and again, you know pass it in but one way or another you have to be able to insert yourself in this hierarchy of getting these calls. When creating a mock there we're actually validating whatever happens happens in you know, maybe the correct order or maybe it's that it happens the correct count whatever you're trying to actually accomplish at the time there we're not just filling, you know, just a dummy requirement.
[00:15:10] But with that said, sometimes we could create a mock it may be that the initializer requires you to have some sort of other object, but you don't really care you're going to intercept calls and inbetween you might just create something. That's just an empty object. And that's what I call like a test double dummy to me.
[00:15:25] So I guess in a way my point is the mocks are a special type of test double that provides verification.
[00:15:31] Leo Dion (Host): So basically like for instance, I have a protocol or a mock of what a network called would be guy just like create a mock that pretends to do the call and then have like a Boolean property says like is called and just set it to true and then that way I can verify that that network call has been made.
[00:15:50] Joshua Greene (Guest): Yeah, then the simplest thing would be holding onto a boolean and just saying that it was actually called. But you know, especially networking. You're likely going to need to return something. Right?
[00:16:01] Leo Dion (Host): Right exactly. So then you return some sort of like mock data based on that network call of some sort.
[00:16:07] Joshua Greene (Guest): Exactly and with the mocks or that test double is you could actually have like with a real networking API you got to go hit a server. It's going to take however long to connect to that get data parse it return it that you've got the full stack there right. Versus with the mock you can immediately like asynchronously pass something back or if you want to turn it in for you know, take it from an asynchronous to synchronous call.
[00:16:29] You can do that with the mark. So you eliminate having to talk to anything real then actually using a mock. It makes your test much much more faster and much more consistent as far as what you expect to happen. Given certain criteria, you can set up the criteria.
[00:16:45] Leo Dion (Host): Are there any libraries or Frameworks that you would recommend to help create those mocks or do you just pretty much implement whatever protocol is going to pretend to do like networking or database or whatever it is?
[00:16:58] Joshua Greene (Guest): For me, I guess the sort of nicety that Objective-C had but Swift is actively moving away from this is the ability to actually have mocked that are true mocks and that you say all right i want to mock this object and it magically happens for you using things like OC mock was a very popular framework that allows you to do this in Objective C.
[00:17:20] This isn't possible in Swift, because the Swift team actively wants to move things from the runtime to compile time and for production code that makes a lot of sense because anything that you can move to compile time, you can catch an error at compile time you throw an error it doesn't build, you know, the developer has to fix that.
[00:17:38] If something gets all the way through runtime, you likely have a runtime crash. So they're really trying to get rid of those runtime crashes. Unfortunately, that makes it basically impossible to do what OC mock was doing with just pure Swift because there's nothing to you know, put yourself in between as far as a runtime that you can actually intercept messages there.
[00:17:57] Instead, you basically only have two choices you either conform to a protocol and implement all the methods there or you take whatever real object is and subclass that and just override whatever behavior such as you know, what you're talking about maybe you capture a boolean instead of whatever the superclass - the real object would have been doing.
[00:18:17] Now with that said you could definitely do those by hand. You have no choice as far as doing this at compile time. You have to compile something so you can write it by hand or there are a few niceties out there like Sourcery for example is a nice tool that allows you to write code. Do it if I hear you say here's a template that I want to use anytime that I see this protocol, you know, that may be conformed some auto-generating or something Sourcery can write that for you.
[00:18:42] So one way or another it has to be compiled time, but there are tools out there to help with some of you know, the harder bits of that or the annoying bits or boiler plate bits.
[00:18:50] Leo Dion (Host): I've just started playing around with Sourcery. Sourcery is amazing what you could do with it. Yeah. Usually what I end up doing is just implementing that protocol in creating my own mock in the test Library.
[00:19:00] That's essentially what I do because like you said you Don't have that ability that Dynamic runtime ability to just like create something on the fly like you can with Objective C.
[00:19:10] So what are some other tips or tricks you have when it comes to like TDD besides mocking trying to think if there's anything else that I think is a major component when it comes to TDD but mocking is a big one and if you have a healthy architecture then I think you're good with mocking.
[00:19:29] I guess we'll jump into like what are some ways you can encourage TDD in your team like what are some ways that like somebody who was higher up like a CTO or a manager can make sure that we're getting the tests that we need and that code is actually being tested and written in a way that it is testable.
[00:19:50] Michael Katz (Guest): There are two things especially if you're talking to a CTO that they like to hear is that maintainability and scalability and making sure that you have a complete working set of tests allows the team to be more efficient and add features faster in the future.
[00:20:05] So. You know that you can add something without worrying that you're going to break existing behavior, especially as the app gets more complicated over time we want to make sure that as people come and go in the organization or just as you forget because you're doing a million different things that there's some requirement that doesn't get lost because it was never really written down in a spec somewhere but there's a test for it
[00:20:26] Joshua Greene (Guest): Yep, definitely maintainability is a win factor if you will as far as not having test versus having test. Also, prevention of regressions, you know Mike is kind of get up there to if you don't have any tests not only is it something that it may be that nobody on the team understands how it is because tests form documentation and maybe that later on you introduce bugs that you fixed in V1 yeah, that's a sad thing to actually see.
[00:20:53] The wins though are definitely those lines. As far as how to get your team to do it though. I'd say first and foremost, especially if a team of any sort of no more than just a couple developers you should be promoting code reviews and in the code reviews if I you know, see somebody on my team and they put out a code review that doesn't have any unit tests I'm immediately rejecting it and it has to have tests in order to even be considered to get into production.
[00:21:21] How do you know if it works without tests? So there is sort of a you know, a philosophy of you must do testing, but you can obviously write unit tests without following TDD. I'd say as far as if you want to get your team to do tdd and you know bring up this test first mentality and get the benefits that TDD. Provides with, you know you write tests for as you're going to get better coverage and typically things will be designed any more testable way versus trying to do that after the fact I'd say the first and foremost your team needs to know how to do it.
[00:21:52] If your team doesn't know test-driven development or know and just have any experience in that area. It's hard to say go and do test-driven development with that said, you know Mike and I you know part of the reason for writing this book was we looked at the community and said, you know iOS community knows about unit testing knows, you know about these things to some extent but we don't think there is a strong grasp for how TDD works. So that was a strong part of why we're actually putting this book together.
[00:22:20] So I'd say check out, you know things like art book check out the raywenderlich.com for you know, where you can go to get these materials to help get your team up to speed but you know, once they're up to speed keep the bar high make sure that there's something in place like code reviews to make sure that it's actually happening
[00:22:35] Michael Katz (Guest): When I've rolled it out on teams, usually what I do is I write a feature in with the team there so that we can serve like a pair programming or four-person five-person programming to see the process and go through it because it can be a little strange if you're not used to doing it a little slow in the first few days until you get the hang but once you do it, it's just like, you know the second nature because it's just this iterative step.
[00:22:59] So having someone that can can walk through and hold hands a little bit also makes it easy to make that transition. I also have a culture of testing that really wants this and is willing to allow developers the ability to take it on.
[00:23:13] Joshua Greene (Guest): I'd say as far as things that prevented him from going to this especially if the apps been around forever. If you've got an app and it's got you know several hundred or thousands or more classes in it just saying, all right, you've got to do this cycle where you build and compile every 30 seconds or are you know faster than that. It may take several minutes to compile the full app let alone run all the tests, right?
[00:23:36] You do have to do this in a way where one thing we are actually going to release a chapter on this in the book even is that you need design not only the core app in a way that it can be testable but there's nothing wrong with actually saying you can pull out a module and so you've got a feature that's implemented just in a module and all of the code for what it's doing is within that module any sort of externalities to its using this third party API or maybe even a different piece of your core application you can put a protocol in between.
[00:24:05] So breaking things up in a way that okay. Here's this small bit here. That's a modulo a different piece that you accomplished has a different goal or something else. That's a different module. So you go from this huge monolithic app to alright a set of libraries or frameworks, whatever you're often used based on your use cases that can be compiled to really quickly - you can run the test against them really quickly and you know can be changed pretty much independently without affecting the rest of the code base. That's fantastic. That's the you know, code that isn't tightly coupled anymore. It's Loosely coupled to the rest of the system. It's testable. It's something that can scale, you know as much as you need it to.
[00:24:42] Michael Katz (Guest): One of the things that was a game-changer for me and doing this with a existing large app was just accepting that it was okay to do TDD for just a new code and any file that was their existing did not go through that level of rigor and acceptance as it is and then in the book also we described how to like slowly add tests for the things that were there before so you can mix and match TDD even though what works best in doing it, you know straight up fresh you can add it into your process slowly and not have to to write, you know, three thousands tests before you can start writing that code because usually you're under some deadline to get something out there.
[00:25:22] Leo Dion (Host): Yeah. I think that's a big part about introducing anything new to a project. I'm thinking about it this week with SwiftUI like when they introduce Swift at you shouldn't go in and rewrite your whole project to be in Swift. If it's an Objective-C. You shouldn't, you know rip out every storyboard and convert everything to SwiftUI for a thousand reasons and I think just with tdd like you shouldn't just go in and start writing tests for all of your existing code.
[00:25:46] Like I think it's just easier to slowly introduce it as you write new features and naturally over time. It will develop that like, most of your code will end up being test driven in the end.
[00:25:56] Joshua Greene (Guest): Definitely, you don't need to just jump into here and go right to the deep end. You can start by just adding it like Mike said for new code only or identify hey, here's this core part of the app. We really need to make sure that this works or we need to make some changes around this. How can we make sure we're not going to break in the process? There's nothing that says you can't put in test place there. As you need them and identify them you don't have to go back and just add I actually be against this at my teens come and said we want to introduce TDD.
[00:26:26] I'm going to add tests throughout the app for I can chip it in new features. No, that's not a good thing to do. It's probably going to be something that you're going to waste time with that. So adding them slowly over time, especially for the very big apps may actually be not only the best choice, but maybe the only choice if you actually want to, you know, continue moving forward with delivering and, you know creating new features at the same time.
[00:26:48] Leo Dion (Host): So what are some other mistakes you think either with teams that are getting started with test-driven development or just teams that are doing thinking that they're doing test-driven development but actually, they're not. What are some common mistakes that teams and developers make when they're doing TDD.
[00:27:04] Michael Katz (Guest): So I think you know one of the mistakes is only covering the happy path or golden path through the code. So you want to make sure that you're testing the edge cases and different combinations of parameters. And if you find that your function has, you know, 4200, you know different combinations of input and that's kind of a code smell there. It may be an indication to break things up into smaller pieces.
[00:27:30] Leo Dion (Host): How do you create enough test data so that you know, you're testing as many cases as possible. Like let's say you have a function that takes in addition like does addition does that mean that I need to test every addition problem possible? What would be a great way to like break that down?
[00:27:45] Michael Katz (Guest): With math, you've got your basic, you know, making sure that you have you cover zeros in a negative values values at the max, you know max float, max int those type of things when you're doing it. Those are probably more academic. I think most apps are in the putting JSON into a table sort.
[00:28:03] So for their you know, your bounds or more, you know, did you get back an empty array to have text you know for a field that's 6 megabytes when you're expecting just a sentence, you know those type of things. It's probably a good thing. I get help somewhere. That's like common things to look out for for that.
[00:28:21] If you have code like an if statement or branch or you know, I do catch those are pretty good indicators that you should have a test at least covers that so you can use the test coverage as a way of least making sure that you've executing the code.
[00:28:34] Leo Dion (Host): Yeah. That's what it sounds like is like you're basically talking about test coverage which if I understand correctly is like basically making sure that every line of code. Xcode does it actually I don't know about you guys but it seems to do a decent job as far as the test coverage instrumentation. But like it'll actually tell you if you've covered every if branch and or switch case in every line of code and it gives you a percentage that's seems pretty reasonable.
[00:28:59] Michael Katz (Guest): Yeah, and that's a good start. It won't cover the out-of-bounds issues or overflows underflows the errors with formatting and things like that because you're still running that code so you can. End of what the function is trying to do as well, but it just covers a good way to start.
[00:29:14] The tricky thing we've run into with test coverage those we have multiple test targets and [with] Xcode I haven't found a good way to sort of sum up the coverage across multiple test targets and maybe the new test plans handles that.
[00:29:30] Michael Katz (Guest): So just like you can have multiple targets you can build Frameworks within a project, multiple apps. You can build multiple test targets so you can group your tests based upon you can have test that just cover networking or databases or test the cover each framework and a test suite and in your scheme, you can specify that when you run the test action which test targets get run and so we've broken up our app into many different libraries and each library has its own test target so we can either run all the suites or if we're just modifying code and one of the Frameworks we just run the tests when you're doing a local development.
[00:30:07] Joshua Greene (Guest): That's a good reason to you know, as far as why coverage isn't the end-all be-all story. So it may not be you know, especially in Mike's case there because he has the multiple targets. It isn't actually a true story in that they could be that you've got two targets and they've each got 50%/50% do you have a hundred percent coverage?
[00:30:25] Is there any overlap between the two of them? It's difficult to say right? So I'd say another sort of common errors, especially as far as management goes is you know giving too much emphasis to test coverage. Test coverage should be something you look at, you know, if you have zero percent versus 80% Obviously I'm going to say oh 80% is probably covering more than you know, the closer you get to the zero, but it's not the most important metric if you said as a manager I'm going to say a hundred percent of you know, the code must be covered by tests. I don't think that's a good goal.
[00:30:58] You know, it doesn't tell you actually, you know very much for what that is. All it indicates is a particular line of code was executed. You know, was it executed with the right inputs?
[00:31:07] Was it executed with all the edge cases? Like Mike is talking about. I don't know. So my point is use it as a starting point. But that's not the only thing you should consider you got to look at, you know are the cases that are the edge cases are the normal flows are all you know, all of the entire picture is what you should actually be considering
[00:31:26] The nice thing about doing tdd, by the way instead of instead of writing these after the fact and adding the unit test on after you've got all the if else has or switches or whatever your code is. Tdd says in order to add new functionality after I did the test first. So, TDD for the most part is going to be more likely to get you in a case where you've covered all of these edge cases. Now in certain times, especially with things like the out of bounds, you might miss something right and you might miss you know, here's this one weird edge case we didn't think about and you know, we accidentally wrote code that either car forms correctly that's a win or you know misses this thing without actually having to test in place. That's okay even after you do TDD or you know, you have test if you identify you're missing a test add a test. Nothing that says you can always go back and you know add more where you need. But you know knowing up front you got most of everything. That's what tdd, you know provides you a stronger guarantee of than just adding test willy-nilly.
[00:32:24] Leo Dion (Host): One question I have when I am right starting to write like an app is sometimes I need to like explore a particular API and just to make sure I have the functionality of that API down before I've even gotten to a point to understand what to test .
[00:32:42] So for instance you're talking like Core Motion or something like HealthKit. I need to understand how HealthKit expects me to get the data back. So I end up actually writing the code before I've even written the test. How do you explore tdd when you're like working with a brand new API that you've never even worked with?
[00:33:03] Joshua Greene (Guest): For me, there's nothing wrong with doing a spike solution. So I've never used an API before how good is my code actually going to be the first go around that, you know, I'm actually touching or using this thing. Maybe it's okay, but probably not great. Right I can go and explore this in a test app and compiling just a few classes is going to be way quicker than compiling my entire code base and just play with it.
[00:33:27] You know, maybe n after I'm done I can look back and say well this isn't actually too bad and then I might actually rewrite whatever I did in a TDD fashion and you know improve it from there or I might say, oh this code is I mean, at least I learned what I was trying to learn but is actually pretty off o it's not going in my case.
[00:33:45] I can entirely throw away a spike that takes me just an hour a couple hours to do and go back and then you know implement it in a correct TDD fashion. That's you know, one way the other way is if you do have some notion of what it does and how it's supposed to work. You can write tests in advance that has the, you know, just the behavior of how the thing works that you're trying to interact with things like Core Motion might be harder because it's actually doing asyncronous things or requiring you to move around in the real world for those sorts of bits. They are, you know, it may be that you can't do a lot of the testing upfront but other sorts of APIs like maybe making REST-ful calls. You can wait on a REST-ful call and you know, accept that I'm just going to write these tests as a temporary thing to see what it returns.
[00:34:31] There's nothing wrong with doing that and you know just using as experimentation. So for me if I'm touching a new thing. Those are my go to either use a spike solution that I may or may not keep and if I do keep I'm gonna have to write tests for it in a TDD fashion or just write tests that are temporary tests that are for exploratory only and throw them away after I'm done because they're not actually useful or that might be useful for learning but they're not useful for keeping in my project forever.
[00:34:57] Leo Dion (Host): It's almost like a rewrite like you basically explore the solution right and that kind of does the behavior more or less and then go through and then rewrite it with a little bit more architecture and tests within it. Is that what I'm hearing?
[00:35:10] Joshua Greene (Guest): You never used it before right so if you've never used an API, you're not going to know how it works or what's best to do with it.
[00:35:16] What's not to do? Your first solution probably isn't going to be great. So a lot of times it's worthwhile to throw out whatever you're doing because it's more of a learning thing than it is anything that's going to truly be worthwhile to put in production
[00:35:29] Michael Katz (Guest): 100% for me. One of the things that we have in our workspace is we have a couple of playgrounds where we actually import the local frameworks for our project. So if we wanted to like test how something new interacts with the code, we've already built we have that ready to go. And so you can spend some time in the playground adding a code calling a new API seeing how that goes. Obviously it doesn't really work for Hardware dependent things but for learning Core Location or you know something with SQL some like that there's definitely work there.
[00:36:00] Leo Dion (Host): Is there anything else you wanted to talk about when it comes to test driven development before I ask more about WWDC?
[00:36:07] Michael Katz (Guest): I think it's becoming more and more industry standard especially and I'm sure it's been in the server-side world for a long time. But in terms of the app world, I know I've had conversations with my executives and they start talking about it in a meeting and it usually takes a couple of years before they learn about whatever the cool new thing is.
[00:36:27] I think going forward we're going to be living in a world where this is more natural and each year Xcode gets a little bit better in terms of supporting the test, especially in terms of test running and test performance. One thing we do really touch on is how you some kind of continuous integration going which I think is it's not essential because TDD is just a methodology but in practice if you have a server that's always running these tests it makes it easier to catch things and make sure that there are issues with merges or multiple developers working on something not everyone may run all the tests before they submit.
[00:37:01] Leo Dion (Host): Yeah, let's talk a little bit about continuous integration. What have you done as far as that's concerned? Like I've done like open source projects on GitHub and I pretty much run Travis CI for doing my you know, simple essentially running my tests and testing out on different operating systems and different devices.
[00:37:21] What are some ways that you've done continuous integration? How have you seen benefits from that.
[00:37:25] Michael Katz (Guest): We use Jenkins. So we have a local server that actually we have I don't know how many Mac Mini nodes that run our tests and we use is a product called FastLane, which I think is pretty common in the iOS world.
[00:37:39] It's kind of like the. It sits on top of xcode build and allows you to basically specify the configuration for building and testing and you can run multiple tests and upload iTunes connect and things like that as part of the service but the important thing there is we have probably about 50 Engineers that contribute to our code base and so it's essential to make sure that what we have is always in good working order.
[00:38:04] But if you're working at a small team or you're working by yourself Travis or Circle CI are good solutions. Xcode server is what Apple keeps pushing. I haven't seen anyone use that in practice, but it has all the Integrations with Xcode so if you're able to use it and you're just building an iOS or macOS app and I think it's going to work out pretty well for you because it knows all about Esco tests and code coverage and things like that.
[00:38:30] Leo Dion (Host): Yeah, I've worked with xcode server. I've actually set up a virtual machine to run whatever the feud like, I'll probably set up one in a week or so and using Catalina and then put Xcode server on it. It's pretty decent like there's a lot to it some tweaking and things that you have to consider when you set up your project. But yeah, it works pretty decent.
[00:38:48] Joshua Greene (Guest): Yeah, so I basically used all of them myself and I'd say given the choice between using nothing or picking at random even right the picking at random is gonna be better than using nothing at all. Especially, you know, the more engineers you have the bigger bang you actually get for wanting to use continuous integration and sure at some point it's just strictly required frankly.
[00:39:10] You know, like my team is pretty large, but if you have maybe one or two developers. Especially if you're following TDD, I don't know they're strictly required for for that size per se because you do want to be constantly just running all over the test. Right? But once you reach a certain point where you have to ensure, okay, the projects is just so large or you got so many engineers that you really need to have some safeguards and checks in place other than just you know, Cowboy. Everybody agrees this is the right way to do it. Continuous integration definitely is a requirement. You know and Xcode server is a good one will be you know, especially if you have a Mac Mini or something around or haven't done it with the Virtual Machine although the now I want to try so that's a good suggestion but it's an easy thing.
[00:39:52] Leo Dion (Host): Especially if you're doing anything or if you're going to be trying new stuff, you probably want to put Catalina on a VM instead of running it on your main machine that's living on the edge. If you want to do that
[00:40:02] Joshua Greene (Guest): The Mac minis are still pretty cheap so our group uses just a small fleet of Mac minis as well there, but the cheaper route is going the VM for sure. So I guess maybe parallel or something depending on how you're running it and I suppose maybe have to buy the parallelization software. I don't know.
[00:40:20] Leo Dion (Host): Yeah, great guys. Thank you so much for coming on the show before we close out. You want to talk a little bit about your book again.
[00:40:28] Michael Katz (Guest): It's an iOS test-driven development by tutorials available now at raywenderlich.com we have an early access that has the first third of the book and the whole book should be out in the fall. It covers the whole TDD from starting from nothing. We have a whole big piece networking because there's almost every app has that and It's tricky to test we have chapters on how to work with a big legacy app especially where legacy means no tests or insufficient testing.
[00:41:00] It goes through each of these things step by step. So you follow along. So unlike other books that may give a more theoretical approach emerges say here's what your code should look like. This is very step by step you follow along going through adding the tests adding the test targets, the whole shebang.
[00:41:17] Joshua Greene (Guest): I'd say we've done the hard work for you going through the Apple apis and you know have a very experienced developers that you know Mike and I've used this for several years. Mike has used this for several years here. You don't want to learn all this stuff by yourself. It'll take longer. It'll be harder to do if you want to get up to speed and get running quickly.
[00:41:36] It's the best way just get the book save yourself some time. Save yourself some sanity happened to not go through those Apple API to yourself and get running quickly and get up to speed quickly.
[00:41:47] Leo Dion (Host): Yeah, that's fantastic. I'm really looking forward to take a look at this book and just learning some new ways to do TDD when it comes to Xcode and iOS.
[00:41:57] Before we close out, do you guys have any thoughts on some of the new stuff from WWDC and how it pertains to test driven development?
[00:42:06] Michael Katz (Guest): Unfortunately, it's about six hours from the what's new in testing in Xcode. So what I've seen so far as just been what they've covered in the what's new in Xcode and the State of the Union, but it looks like that there's some new methods for.
[00:42:21] We've had the measuring blocks before that measure the time your test takes I know it looks like you can also measure memory usage, disk usage, and a bunch of other things which is really great. They're not really like for test-driven development. But when you're building reliability tests and performance tests, and with those are going to be pretty handy and then there's this new thing called test plans which basically.
[00:42:42] Takes all the management of tests from the scheme into a separate top-level object in your project and you can rerun your tests against multiple configurations. So you can run all your tests in English and then run all the tests again in French or whatever in one click. They also have four different test ordering and different runtime checks and all that so it takes one manual step out of the loop, but I wouldn't necessarily think it's super revolutionary. What do you think Josh?
[00:43:10] Joshua Greene (Guest): I'm totally with you as far as the new features. They look pretty cool for Xcode goes. I also wanted to point out that with the SwiftUI they've really gone very much so protocol oriented development.
[00:43:22] That's great news for things like testing and test driven development here anytime you can put in a protocol in place you can insert a mock very easily into whatever set up it is so I'm very excited to see how they've really architected SwiftUI and it looks like just testing is going to be a lot easier compared to some of the previous solutions where you've got concrete class and just figuring out how you're going to mock that out, you know via subclass or whatnot may have been very difficult with pure UIKit.
[00:43:51] It looks like SwiftUI is gonna help us quite a bit in the testing realm. So I'm very excited to dive into it and learn everything about you know all the cool stuff that they've done and cool stuff to make developers lives easier with testing.
[00:44:04] Leo Dion (Host): You're talking about the difference between like UIViewController, which is this class.
[00:44:08] You have to subclass as opposed to like the SwiftUI View protocol which you can pretty much mock up any way you want to right?
[00:44:15] Joshua Greene (Guest): Exactly yeah, the whole UIViewController thing, there's so much concrete stuff in that it's difficult to truly create great mocks with it. I mean you can, you know, create a mock by subclassing uiviewcontroller, right.
[00:44:28] But had that been a protocol, I mean you can combine protocols so much more easily combined functionality using protocol oriented development. But creating mocks the same thing, you can just conform to that Mark and just Implement whatever message you actually care about and it's just much more elegant solution a much more, you know architecture friendly solution going with the protocol then something concrete.
[00:44:51] So it's a very good job on Apple's part and you know it's my take on it so. I haven't gone to see all of the defaults implementations of stuff they give you but you know, hopefully it's saying and you know, from what I have seen it looks very well done.
[00:45:04] Michael Katz (Guest): One of those things with SwiftUI is there really is from sample code encouraging people to build out separate model classes, which also makes it easier to test.
[00:45:15] Michael Katz (Guest): Yeah with the previews and you can have being able to put multiple previews in the assistant.
[00:45:20] I think I'll make it great for testing not really testing but just visualizing the different conditions before you even get to the point of completing your code, but the at state you can have a separate model class be bound to your view as you to put that logic of loading data and changing data and so on outside of the view of other people couple and their view controllers the setting of The View State and then the model state so it's exciting.
[00:45:45] Joshua Greene (Guest): It looks like they're kind of pushing Mike component-based development too you know, I've seen like so many developers where you need a custom UI tap controller or something, right. So you go and instead of right and something your own which may be in the right solution you go and you try to reuse Apple's and then you're trying to hack internal methods and so forth and it's just hard to do with all the Swift you I think they provided a same base and said look you need new components. You can create something. I'm excited for testing because the idea being you could split those off into their own library. Here's this component that I need or set of components that I need and just pull them in as you need and ensure that they're tested really well. So again, it just looks really well done. So the team did a really good job as far as everything I've seen so far.
[00:46:25] Leo Dion (Host): Well, thank you so much guys for coming on the program if folks want to get ahold of you. What's the best way?
[00:46:31] Joshua Greene (Guest): Sure so if you have questions about the book, we actually have a forum on raywenderlich.com. We actively monitor that as authors of the book.
[00:46:41] So if you have questions ask them there and we will actually answer you for anything that you've got. If you want to reach us personally jrgdeveloper@gmail.com by email or Twitter @grg_developer
[00:46:56] Leo Dion (Host): well, thank you so much for coming on and maybe we'll talk again later about Swift UI and how that's going to improve test-driven development.
Thanks to our monthly supporters
- Bertram Eber
- Edward Sanchez
- Satoshi Mitsumori
- Steven Lipton
Creators & Guests
Host
Guest
What is Empower Apps?
An exploration of Apple business news and technology. We talk about how businesses can use new technology to empower their business and employees, from Leo Dion, founder of BrightDigit.