1 00:00:03,218 --> 00:00:05,498 Welcome to another episode of Empower Apps. 2 00:00:05,498 --> 00:00:07,218 I'm your host, Leo Dion. 3 00:00:07,238 --> 00:00:09,278 Today I'm joined by Rachel Brindle. 4 00:00:09,338 --> 00:00:11,978 Rachel, thank you so much for taking time to come on 5 00:00:12,455 --> 00:00:12,965 Yeah, 6 00:00:13,015 --> 00:00:17,575 before we get into Swift Testing and all the new stuff with testing. 7 00:00:17,605 --> 00:00:19,285 I'll let you go ahead and introduce yourself. 8 00:00:20,167 --> 00:00:22,057 Yeah, so I am Rachel. 9 00:00:22,157 --> 00:00:24,887 I am a principal engineer at Autodesk. 10 00:00:25,287 --> 00:00:30,147 And I have been involved with testing, test-driven development and 11 00:00:30,147 --> 00:00:32,387 such, basically my entire career. 12 00:00:33,197 --> 00:00:37,997 A lot of fairly deep knowledge of like how to go through and test things, as 13 00:00:37,997 --> 00:00:43,077 well as mentoring others in learning how to test code, how to write better 14 00:00:43,077 --> 00:00:46,257 tests that aren't brittle and won't break all the time and so forth. 15 00:00:47,045 --> 00:00:47,535 Awesome. 16 00:00:48,255 --> 00:00:51,525 So what year did Swift Testing come out? 17 00:00:51,575 --> 00:00:57,375 That was, It shipped in Xcode 16 last year, but it was open 18 00:00:57,375 --> 00:00:59,490 sourced in September, 2023. 19 00:01:00,153 --> 00:01:00,543 Okay. 20 00:01:00,693 --> 00:01:02,893 So you did a lot of work with XCTesting. 21 00:01:02,913 --> 00:01:07,643 You have a few libraries you actually built, for XCTesting, to help folks. 22 00:01:07,693 --> 00:01:12,043 What did you think about the process as far as Swift Testing is concerned 23 00:01:12,523 --> 00:01:14,503 and how we got to where we are now? 24 00:01:15,003 --> 00:01:19,083 How we went from XCTesting to Swift Testing. 25 00:01:19,293 --> 00:01:22,053 And what do you miss? 26 00:01:22,303 --> 00:01:25,063 what are you happy that we got rid of Things like that, 27 00:01:25,113 --> 00:01:29,733 So Swift Testing is obviously, 'cause it's only like publicly, it's only 28 00:01:29,733 --> 00:01:33,903 like a year and a half old, though I'm, I know for a fact that like, 29 00:01:34,353 --> 00:01:36,243 it didn't just pop outta nowhere. 30 00:01:36,913 --> 00:01:39,433 they've been working on it since at least macros were a thing. 31 00:01:40,183 --> 00:01:43,363 Swift Testing, if it's still like no more than, it can't be 32 00:01:43,363 --> 00:01:44,473 older than like three years. 33 00:01:45,163 --> 00:01:45,763 It is. 34 00:01:46,168 --> 00:01:48,618 Nowhere near as mature as XCTest is. 35 00:01:48,658 --> 00:01:55,408 XCTest has been, was introduced in iOS seven, iOS eight era. 36 00:01:55,748 --> 00:02:02,478 But even still it dates back to much older Objective C focused testing APIs 37 00:02:02,478 --> 00:02:04,218 like syn testing kit and so forth. 38 00:02:04,668 --> 00:02:06,678 And those date back all the way to the nineties. 39 00:02:08,058 --> 00:02:09,078 I was gonna ask about that. 40 00:02:09,108 --> 00:02:10,443 'cause like XCTest, I would assume. 41 00:02:11,253 --> 00:02:16,223 Like you said is from quite a while since Objective C. 'cause I can't even imagine 42 00:02:16,223 --> 00:02:18,918 what they used for testing before that. 43 00:02:19,790 --> 00:02:24,220 Yeah, it was, I don't think Apple had, so it was initially started as 44 00:02:24,280 --> 00:02:26,340 some, third party, like predating. 45 00:02:26,430 --> 00:02:28,890 It might even predate like Apple buying next. 46 00:02:28,890 --> 00:02:30,180 I don't that. 47 00:02:31,455 --> 00:02:33,525 That I, I don't quite know the history there. 48 00:02:33,975 --> 00:02:40,165 But it was, there was a send testing kit and like a bunch of like other 49 00:02:40,705 --> 00:02:42,565 third party stuff is what people used. 50 00:02:42,565 --> 00:02:47,305 At some point in the two thousands, apple started shipping or like including 51 00:02:47,305 --> 00:02:53,055 send testing kit with, Xcode so that you didn't have to download that, And 52 00:02:53,265 --> 00:02:57,365 they started to just refine that over time and eventually they rewrote it 53 00:02:57,365 --> 00:02:59,715 with XC test and, and shipped that. 54 00:03:00,303 --> 00:03:08,273 Okay, so Swift tests, like the big thing I was gonna talk about is what you miss from 55 00:03:08,273 --> 00:03:13,073 XCTesting, but then like a part of that is just what we could do with Objective 56 00:03:13,073 --> 00:03:15,053 C that we can't get away with in Swift. 57 00:03:15,053 --> 00:03:18,023 So there is the open source XCTest. 58 00:03:18,053 --> 00:03:23,653 it's Swift CoreLibs XCTest and that is written entirely in Swift and 59 00:03:23,653 --> 00:03:27,443 that enables that has been around since Swift package manager was open 60 00:03:27,443 --> 00:03:29,423 sourced back in like 2015 or so. 61 00:03:30,383 --> 00:03:35,373 And that is not feature complete with like the private XCTest from Apple. 62 00:03:35,373 --> 00:03:37,913 But it does a lot of what you can do with XCTest. 63 00:03:38,323 --> 00:03:43,943 XCTest is really much better for building testing tools on top of, like you can 64 00:03:43,943 --> 00:03:48,923 dynamically create and register tests at runtime and XC test, which is not a 65 00:03:48,923 --> 00:03:50,633 thing you can do in Swift Testing at all. 66 00:03:51,623 --> 00:03:55,643 I know that there are people on the Swift Testing team interested 67 00:03:55,643 --> 00:03:58,343 in enabling something like that, but it's just not there yet. 68 00:03:59,293 --> 00:04:02,773 Though reading between the lines, they're definitely moving to enable 69 00:04:02,773 --> 00:04:05,613 that feature in, Swift Testing. 70 00:04:06,873 --> 00:04:09,773 And then there's other minor stuff like, performance tests. 71 00:04:09,823 --> 00:04:15,103 The XCTest has the measure API, so you can Figure out how long it takes 72 00:04:15,103 --> 00:04:18,923 to run a bit of code so you can see like, oh, do I have any performance 73 00:04:18,923 --> 00:04:21,023 regressions in this area and such? 74 00:04:21,473 --> 00:04:26,353 And you can kind of do that with Swift Testing, but for a bunch of 75 00:04:26,353 --> 00:04:30,893 reasons, Swift Testing's, concurrent test runner being a major issue there. 76 00:04:31,293 --> 00:04:32,313 it's not great. 77 00:04:32,523 --> 00:04:35,343 And of course there's not really any equivalent to the measure API. 78 00:04:35,568 --> 00:04:40,568 So what did you really like about Swift Testing and where it's headed 79 00:04:40,568 --> 00:04:42,988 right now compared to XCTesting? 80 00:04:44,053 --> 00:04:47,953 So the thing that I really like about it first, the DSL of the 81 00:04:47,953 --> 00:04:51,608 experience of actually writing tests is much nicer than with XCTest. 82 00:04:52,088 --> 00:04:56,498 The macro or just being able to specify like the attest macro and 83 00:04:56,498 --> 00:05:01,548 then just like a bear function, is great, especially in Swift six two. 84 00:05:01,618 --> 00:05:08,488 I think it's se 4 51 ads just like you can, I think it's bear function. 85 00:05:08,538 --> 00:05:12,418 You can use a backtick and then put an arbitrary string or nearly arbitrary 86 00:05:12,418 --> 00:05:15,688 string between back ticks and like that's the name of your function. 87 00:05:16,408 --> 00:05:19,718 So you can have spaces commas and punctuation in there. 88 00:05:20,378 --> 00:05:24,068 that will render nicely as a full human readable string so you don't have to 89 00:05:24,068 --> 00:05:27,248 do this weird thing where like you have attest and then whatever human readable 90 00:05:27,248 --> 00:05:33,128 string and then a non-human readable, but like valid function identifier afterwards. 91 00:05:33,911 --> 00:05:36,581 Okay, so now you could just make the function whatever you want, 92 00:05:36,581 --> 00:05:37,721 and put whatever characters you 93 00:05:37,748 --> 00:05:38,228 yeah. 94 00:05:38,408 --> 00:05:39,068 That's crazy. 95 00:05:39,335 --> 00:05:43,565 That's useful for non-testing stuff, but mostly it's useful for testing. 96 00:05:44,315 --> 00:05:46,415 That's like reading the proposal for that. 97 00:05:46,745 --> 00:05:48,725 That's like the main motivation they cite. 98 00:05:49,493 --> 00:05:49,783 Yeah. 99 00:05:50,033 --> 00:05:50,333 Yeah. 100 00:05:50,593 --> 00:05:57,233 So yeah, like the, going back to other things that I really like about, Swift 101 00:05:57,233 --> 00:05:59,463 Testing I like how it's much more open. 102 00:06:00,478 --> 00:06:03,898 Both like it's developed mostly in the open. 103 00:06:03,898 --> 00:06:06,748 there is a testing work group, which I'm a member of. 104 00:06:07,238 --> 00:06:08,063 And it's. 105 00:06:08,948 --> 00:06:12,828 It, follows a form of the Swift evolution process. 106 00:06:12,858 --> 00:06:15,798 Like testing proposals are in the Swift evolution repo. 107 00:06:16,338 --> 00:06:21,118 But the proposals are formatted slightly differently and such so 108 00:06:21,118 --> 00:06:24,418 I'm really liking that there's a lot more community engagement in 109 00:06:24,448 --> 00:06:25,678 the development of Swift Testing. 110 00:06:27,121 --> 00:06:27,481 Yeah. 111 00:06:27,481 --> 00:06:31,291 And I think that's the biggest advantage is the fact that it's open source and 112 00:06:31,291 --> 00:06:35,751 that we can build a whole set of tools around that and help it facilitate its 113 00:06:35,751 --> 00:06:39,001 growth, what point do you think it'd be important for a team to migrate 114 00:06:39,031 --> 00:06:41,161 from XCTesting to Swift Testing? 115 00:06:42,358 --> 00:06:45,678 Really that depends on the team, there's a ton of tests that are 116 00:06:45,678 --> 00:06:50,978 written right now that cannot be easily migrated or like can't, it's 117 00:06:50,978 --> 00:06:54,998 not gonna be an easy process to migrate it from XCTest to Swift Testing. 118 00:06:55,496 --> 00:06:56,801 Do you have an example? 119 00:06:56,828 --> 00:07:01,498 So like anything that accesses like, or modify some like shared global state, 120 00:07:02,038 --> 00:07:05,608 For like one specific test, you might change it to some value. 121 00:07:05,608 --> 00:07:07,438 And then another test wants it to be some other value. 122 00:07:07,928 --> 00:07:12,848 because Swift Testing runs everything concurrently in process, then 123 00:07:12,878 --> 00:07:15,998 you're gonna run into issues if those two tests are ever actually 124 00:07:15,998 --> 00:07:17,018 like running at the same time. 125 00:07:17,861 --> 00:07:19,181 Should you be doing that in the first 126 00:07:19,453 --> 00:07:20,013 I mean no, 127 00:07:20,064 --> 00:07:20,354 Okay. 128 00:07:20,451 --> 00:07:20,661 Yeah. 129 00:07:20,661 --> 00:07:20,931 Yeah. 130 00:07:21,171 --> 00:07:23,091 In a way it forces you to be cleaner. 131 00:07:23,091 --> 00:07:26,661 Like you'll end up replacing that type of thing with a task local or whatever. 132 00:07:27,291 --> 00:07:30,951 And that also brings along all the niceties of like, you're now going 133 00:07:30,951 --> 00:07:33,951 to force your tests to actually like build under the Swift six 134 00:07:33,951 --> 00:07:35,301 language mode and all that such. 135 00:07:35,901 --> 00:07:40,881 But there are plenty of tests that exist right now that work perfectly fine in 136 00:07:40,941 --> 00:07:45,536 the world with XCTest where it's only one test is running at a time in the process. 137 00:07:46,896 --> 00:07:49,806 That are going to be difficult to migrate to Swift Testing. 138 00:07:50,504 --> 00:07:50,794 Okay. 139 00:07:50,916 --> 00:07:55,026 And then the other benefit is it'll be slightly easier to read stuff. 140 00:07:55,426 --> 00:08:01,076 You'll get faster runtime of tests because they can execute in parallel but 141 00:08:01,126 --> 00:08:05,266 you're gonna get slightly slower build times because macros take a bit of time. 142 00:08:05,296 --> 00:08:09,646 Even tool chain macros, they just add a little bit more time to the compile time. 143 00:08:10,306 --> 00:08:12,766 automation tests still aren't a thing and so forth. 144 00:08:13,566 --> 00:08:19,486 But yeah, if you have working XC tests, there's no reason to, unless 145 00:08:19,486 --> 00:08:23,546 you really want to, there's no reason to go rewrite them into Swift Testing. 146 00:08:24,174 --> 00:08:24,594 Maybe. 147 00:08:25,611 --> 00:08:27,941 So like that's the approach to that idea is new components 148 00:08:28,001 --> 00:08:29,201 are written in Swift Testing. 149 00:08:29,601 --> 00:08:33,651 But if it's an existing component that already has tests written for it, then 150 00:08:34,191 --> 00:08:35,931 I'm just gonna keep it in XC test. 151 00:08:36,244 --> 00:08:39,784 Do you think macros have a place in filling the gap when 152 00:08:39,784 --> 00:08:41,164 it comes to what's missing? 153 00:08:41,164 --> 00:08:42,064 In Swift Testing? 154 00:08:42,826 --> 00:08:43,426 Yes and no. 155 00:08:44,106 --> 00:08:48,036 So well for I guess to correctly answer, yes, they have a gap. 156 00:08:48,326 --> 00:08:48,986 But like 157 00:08:50,996 --> 00:08:55,406 they are very painful to write and maintain as I'm sure you don't need to 158 00:08:55,406 --> 00:09:00,086 be reminded, I looked at how, the expect and require macros and Swift Testing 159 00:09:00,086 --> 00:09:05,606 are implemented and like mad respect to the people on the testing team who wrote 160 00:09:05,606 --> 00:09:07,796 them, because I do not know if I could. 161 00:09:08,589 --> 00:09:11,709 And there's also the other stuff of like, until very recently you had to 162 00:09:11,709 --> 00:09:15,319 rebuild sw, Swift, Swift syntax for like. 163 00:09:16,159 --> 00:09:19,369 Third party macros and such, that isn't really a thing now that we 164 00:09:19,369 --> 00:09:25,149 have like the cast with Swift syntax as of like 16 four or code 16 four. 165 00:09:25,839 --> 00:09:29,259 But like there are, there actually are places for macros. 166 00:09:29,289 --> 00:09:32,979 Every so often someone will post in the Swift forums, the community 167 00:09:32,979 --> 00:09:37,524 showcase about a library that uses macros to generate test doubles. 168 00:09:38,904 --> 00:09:40,624 Conforming to the protocol specified. 169 00:09:41,074 --> 00:09:43,654 So a test double is just kind of a stand in for the real thing. 170 00:09:44,372 --> 00:09:44,662 Okay. 171 00:09:44,662 --> 00:09:51,212 So you might have your type that has some methods or whatever and you have 172 00:09:51,212 --> 00:09:52,802 a protocol that defines that type. 173 00:09:52,902 --> 00:09:56,157 You might then use one of these tools to. 174 00:09:56,997 --> 00:10:01,147 Create a test double that automatically conforms to and implements all of 175 00:10:01,147 --> 00:10:05,087 these methods such that you can then either just directly pass it 176 00:10:05,087 --> 00:10:06,437 in and it'll work in the thing. 177 00:10:06,437 --> 00:10:09,367 Or you can modify behavior or you could even examine how 178 00:10:09,367 --> 00:10:10,867 it was called and so forth. 179 00:10:11,687 --> 00:10:14,417 that's really cool to see and that saves a bunch of boilerplate. 180 00:10:16,232 --> 00:10:20,252 Yeah, so that's one of the main examples of using macros to fill in the gap and 181 00:10:20,252 --> 00:10:25,082 do things that Apple probably isn't gonna do, at least not with Swift Testing. 182 00:10:26,259 --> 00:10:27,754 At least not in the next decade. 183 00:10:28,564 --> 00:10:31,594 Yeah, you had this like marketable macro, which is really cool. 184 00:10:31,714 --> 00:10:35,644 I just started looking at that, because like that's one of the things people 185 00:10:35,644 --> 00:10:40,384 love about Objective C is everything is dynamic so you can go in and swizzle 186 00:10:40,384 --> 00:10:45,094 and do all sorts of crazy stuff, whereas Swift is super strict about that stuff. 187 00:10:45,932 --> 00:10:51,642 Like for good reason, like, outside of using tests to like mock out 188 00:10:51,642 --> 00:10:55,502 behavior, we might get the into this in a, in a little bit, but like this 189 00:10:55,502 --> 00:10:59,282 is part of what made testing UI kit a lot less painful was being able to 190 00:10:59,282 --> 00:11:00,632 swizzle out a whole bunch of stuff. 191 00:11:01,002 --> 00:11:04,182 Which you can't even do any of that with Swift test with Swift UI. 192 00:11:04,722 --> 00:11:07,212 And like Swift UI is also not untestable. 193 00:11:08,069 --> 00:11:08,369 Yeah. 194 00:11:08,399 --> 00:11:09,449 Well, let's talk about that. 195 00:11:09,499 --> 00:11:13,519 That's one of the biggest complaints is how SwiftUI is testable. 196 00:11:13,879 --> 00:11:15,859 What does that mean exactly? 197 00:11:15,909 --> 00:11:22,319 Yeah, so you can write tests for Swift UI using Swift UI but you can 198 00:11:22,319 --> 00:11:23,759 only write automation tests for it. 199 00:11:23,759 --> 00:11:30,429 So that's like the XCUI test stuff that, works by spinning up your app in 200 00:11:30,429 --> 00:11:34,299 the simulator and then having another process that actually runs the tests that. 201 00:11:34,974 --> 00:11:39,664 Uses IPC to interact with the app using the accessibility system. 202 00:11:40,282 --> 00:11:40,502 Yep. 203 00:11:40,744 --> 00:11:43,984 and out of the box you can only like, just interact with the 204 00:11:43,984 --> 00:11:45,304 UI and examine stuff in the ui. 205 00:11:46,384 --> 00:11:51,884 You can't like go in and examine oh, did this insert something into, did 206 00:11:51,884 --> 00:11:55,064 this actually insert something into the database or is the UI just lying to me? 207 00:11:55,694 --> 00:11:58,364 Out of the box, it's going to make like every single network request 208 00:11:58,364 --> 00:12:00,644 and so forth, which it's not really. 209 00:12:01,119 --> 00:12:07,339 But there's a bunch of third party stuff to make, automation tests, or like to 210 00:12:07,889 --> 00:12:11,369 fake out network requests and so forth and like enable you to better inspect 211 00:12:11,369 --> 00:12:12,869 stuff going on at automation tests. 212 00:12:13,427 --> 00:12:18,232 Yeah, well, like one thing I hear a lot about is snapshot testing as a way, which 213 00:12:18,262 --> 00:12:20,182 I think is from the point free team. 214 00:12:20,224 --> 00:12:22,834 so there are several ways you can do snapshot testing. 215 00:12:23,329 --> 00:12:29,279 There's an, older, API from I think Uber was the first who made it iOS snapshot 216 00:12:29,279 --> 00:12:33,839 test case both of these, the 0.31 does a little bit more, you have a little 217 00:12:33,839 --> 00:12:35,369 bit more options to actually inspect it. 218 00:12:35,859 --> 00:12:41,789 it works by just rendering the UI into an image and then basically working 219 00:12:41,849 --> 00:12:45,239 entirely as a regression test of like, you render the eye into an image. 220 00:12:45,239 --> 00:12:47,849 You test that and you just assert that. 221 00:12:47,949 --> 00:12:50,679 The image is the same as a previously recorded image 222 00:12:50,729 --> 00:12:50,949 Yep. 223 00:12:51,647 --> 00:12:57,167 That can work either as like, like an XCUI automation test or as more 224 00:12:57,167 --> 00:12:59,267 of a unit test, like in process test. 225 00:13:00,227 --> 00:13:03,857 And that's fine. 226 00:13:04,427 --> 00:13:06,887 Like it works for like pure regression stuff. 227 00:13:06,887 --> 00:13:09,252 There's, it's very brittle. 228 00:13:10,217 --> 00:13:15,937 Like I do not envy the people with, with snapshot tests, going into 229 00:13:15,967 --> 00:13:19,597 who are probably not having a very fun summer, having to rerecord all 230 00:13:19,597 --> 00:13:23,387 of their stuff for like the liquid glass design, language change. 231 00:13:23,807 --> 00:13:26,687 And especially like each new beta, it's gonna change. 232 00:13:27,707 --> 00:13:31,747 But even going between like minor versions of the os, there, there 233 00:13:31,747 --> 00:13:35,077 are like gonna be slight changes to how things render and so forth. 234 00:13:35,962 --> 00:13:39,712 There's even like stuff of whether when the simulator runs on like an Intel 235 00:13:40,102 --> 00:13:44,862 or like a, apple, silicon Mac you, it will render it slightly different. 236 00:13:44,862 --> 00:13:47,742 So you have to do like some sort of like fuzzy image compare instead 237 00:13:47,742 --> 00:13:48,822 of just doing a straight up, 238 00:13:49,319 --> 00:13:49,829 You're right. 239 00:13:50,082 --> 00:13:51,852 pixel by pixel compare. 240 00:13:52,664 --> 00:13:52,954 Okay. 241 00:13:53,802 --> 00:13:57,252 so like snapshot tests work, but they're extremely brittle. 242 00:13:58,392 --> 00:14:01,902 When I say that Swift UI isn't testable, what I mean is that I 243 00:14:01,902 --> 00:14:03,922 can't write a unit test for it. 244 00:14:04,522 --> 00:14:10,192 One of the things that you can do with UI kit is you can directly call the 245 00:14:10,192 --> 00:14:16,702 methods that Youi kit itself calls when it's like when you tap for that 246 00:14:16,702 --> 00:14:18,592 lead to a button callback being called. 247 00:14:19,215 --> 00:14:23,055 So it's possible for me to write a test that's just like, Hey, when this button is 248 00:14:23,055 --> 00:14:27,085 tapped I write to the database and I can actually then inspect that the database 249 00:14:27,085 --> 00:14:31,405 was written to, or that a test double that represents the database was talked to. 250 00:14:32,725 --> 00:14:35,525 and that's not something you can do with Swift UI. 251 00:14:35,645 --> 00:14:36,785 You can't simulate a button. 252 00:14:36,785 --> 00:14:38,015 Press in test. 253 00:14:39,233 --> 00:14:42,833 could you use something like an inspect and push the button? 254 00:14:43,370 --> 00:14:48,140 Yes, there is a third party library called a view Inspector. 255 00:14:48,620 --> 00:14:50,845 I'll add it to your, show notes To its credit. 256 00:14:50,845 --> 00:14:55,005 It does a really good job of allowing you to inspect and test like, Swift 257 00:14:55,005 --> 00:14:57,115 UI stuff, Swift UI views and so forth. 258 00:14:57,665 --> 00:14:58,685 But it's incomplete. 259 00:14:58,685 --> 00:15:02,585 there's a bunch of APIs from Apple that it doesn't have good inspection for. 260 00:15:02,595 --> 00:15:07,615 it uses the mirroring API to be able to inspect things. 261 00:15:07,615 --> 00:15:10,105 And then it has to know about specific. 262 00:15:10,480 --> 00:15:16,430 Like sub components within to, or like to go like inspect, oh, this was the, UI 263 00:15:16,460 --> 00:15:18,590 button or the, sorry, the Swift UI button. 264 00:15:19,030 --> 00:15:20,260 tap handler and such. 265 00:15:20,360 --> 00:15:27,060 It doesn't automatically support new Apple provided, Swift UI views by default. 266 00:15:27,170 --> 00:15:30,050 It does work with like your own third party views 'cause. 267 00:15:31,085 --> 00:15:32,735 But it works. 268 00:15:32,735 --> 00:15:36,725 It's also brittle, it's vulnerable to internal changes to in Swift UI of 269 00:15:37,103 --> 00:15:38,333 That's exactly what I was gonna say 270 00:15:38,383 --> 00:15:39,373 that happened last year. 271 00:15:40,363 --> 00:15:41,983 So, yeah. 272 00:15:43,028 --> 00:15:46,388 really what I would love to see is first party support for that. 273 00:15:46,628 --> 00:15:51,828 I know for a fact that every time, 'cause I bring it up every year and a WW TC lab 274 00:15:51,888 --> 00:15:58,008 that both the Swift UI and testing teams get that very frequently as a request. 275 00:15:58,068 --> 00:16:02,118 But for various reasons they haven't been able to deliver that. 276 00:16:03,110 --> 00:16:04,820 How would you picture something like that? 277 00:16:05,240 --> 00:16:10,580 Like you just create a Swift to eye view and then some sort of 278 00:16:10,580 --> 00:16:12,460 select, like how does web do it? 279 00:16:12,460 --> 00:16:12,850 Right? 280 00:16:12,880 --> 00:16:18,070 Web would just be like, oh, grab the button using a selector, and then push. 281 00:16:18,710 --> 00:16:19,040 Yeah. 282 00:16:19,040 --> 00:16:21,470 So that's exactly what I'm thinking. 283 00:16:21,710 --> 00:16:25,670 Like, something that I've been working on as like a blog post is, 284 00:16:26,030 --> 00:16:27,590 exploring what that might look like. 285 00:16:28,215 --> 00:16:32,965 Through the lens of like, this is what we might try to do for UI kit and 286 00:16:32,965 --> 00:16:35,035 then because we can do it for UI kit. 287 00:16:35,635 --> 00:16:39,295 I like the approach, the interacting through the accessibility system 288 00:16:39,685 --> 00:16:41,305 that automation tests take. 289 00:16:41,915 --> 00:16:44,375 'Cause that already exists, that forces you to actually 290 00:16:44,585 --> 00:16:46,145 add accessibility, support, 291 00:16:46,483 --> 00:16:47,108 I was gonna say that. 292 00:16:47,255 --> 00:16:48,035 And such. 293 00:16:48,435 --> 00:16:50,595 I think that'd be a really good thing to do. 294 00:16:50,695 --> 00:16:54,865 plus it's also fairly, like black box, like you could call in an 295 00:16:54,865 --> 00:16:59,155 automation test, you can call a tap on an element that cannot be tapped. 296 00:16:59,185 --> 00:17:02,275 It's non-interactive and nothing will happen. 297 00:17:02,365 --> 00:17:04,975 Like you won't get a compile error. 298 00:17:05,185 --> 00:17:07,645 It's just that nothing will happen and your test will silently fail. 299 00:17:08,095 --> 00:17:10,585 or maybe it won't fail depending on what your next assert is 300 00:17:11,185 --> 00:17:13,045 that's something that I think is. 301 00:17:14,260 --> 00:17:17,560 Really powerful as a direction to go maybe not exactly mirroring the, 302 00:17:17,610 --> 00:17:19,410 public API of automation tests. 303 00:17:19,770 --> 00:17:23,610 But that's a great starting point of let's interact through the accessibility 304 00:17:23,610 --> 00:17:28,590 system to be able to like pull on, pull an element that should exist. 305 00:17:29,200 --> 00:17:32,505 and then like be able to interact with it, test or tap on it, 306 00:17:32,505 --> 00:17:33,645 swipe on it, and so forth. 307 00:17:34,803 --> 00:17:36,213 Yeah, that makes complete sense. 308 00:17:37,143 --> 00:17:40,023 So I wanna jump back, talk a little bit more about Swift Testing. 309 00:17:40,023 --> 00:17:44,793 So one of the things I really liked about it is the way that you can organize 310 00:17:44,793 --> 00:17:49,963 your tests, without needing to do certain things with classes and stuff. 311 00:17:50,713 --> 00:17:56,623 Kind of explain the whole suite testing thing and what are some things developers 312 00:17:56,623 --> 00:17:59,533 should be thinking about when they write their tests and how they organize them. 313 00:17:59,815 --> 00:18:03,865 Yeah, one of the really cool things about Swift Testing is the fact that 314 00:18:03,865 --> 00:18:06,745 you can nest sweets as much as you want. 315 00:18:07,505 --> 00:18:13,805 you can have zero sweets, so just bare functions that are with that test macro. 316 00:18:14,385 --> 00:18:16,980 You can organize, you can have like a single layer of sweets. 317 00:18:16,980 --> 00:18:18,335 You can have as many as you want. 318 00:18:18,995 --> 00:18:25,065 There is, you do kind of run into the issue where, if you have properties on 319 00:18:25,065 --> 00:18:29,175 like the parent suite, a child suite cannot reference it because they're 320 00:18:29,175 --> 00:18:32,835 structs and they can't know about, they don't, they're not getting like 321 00:18:32,835 --> 00:18:36,405 an a reference to like an instance of the parent struct and so forth. 322 00:18:36,405 --> 00:18:39,985 So you can't you have that limitation. 323 00:18:40,175 --> 00:18:43,795 I'm hoping that once apple. 324 00:18:44,380 --> 00:18:49,630 Provides like more APIs for like dynamically adding tests and so 325 00:18:49,630 --> 00:18:52,870 forth, dynamically that there'll be community solutions to that. 326 00:18:52,870 --> 00:18:57,240 Something like what the quick framework provides for XCTest. 327 00:18:57,340 --> 00:18:59,380 Could you give like a use case of that? 328 00:19:00,008 --> 00:19:00,668 so 329 00:19:00,970 --> 00:19:03,295 a parent suite info or metadata. 330 00:19:03,748 --> 00:19:04,348 Yeah, yeah. 331 00:19:04,618 --> 00:19:07,883 So that, that's something like, Even just as simple as like having shared setups. 332 00:19:07,883 --> 00:19:13,223 So you only have to create the object being tested like in one place in the, 333 00:19:13,223 --> 00:19:18,173 like in it for the, in like the parent or whatever, the outermost suite. 334 00:19:18,823 --> 00:19:24,163 And then later on, like you might have like a child suite that is just calling a 335 00:19:24,163 --> 00:19:27,013 method on that object that's being tested. 336 00:19:27,743 --> 00:19:28,013 That's. 337 00:19:28,713 --> 00:19:29,463 One way to do that. 338 00:19:29,463 --> 00:19:34,683 I also like to nest suites, or I would like to nest suites for like, 339 00:19:34,763 --> 00:19:40,433 conditional and such, a child, one suite for if this conditional is true. 340 00:19:40,433 --> 00:19:42,353 Another for if it's false and such. 341 00:19:43,073 --> 00:19:49,423 Because then you also get like the nice fo code, code voting capabilities well, I 342 00:19:49,423 --> 00:19:52,423 don't really care about the positive case right now, so I'm just gonna fold that 343 00:19:52,423 --> 00:19:53,698 up and only look at the negative case. 344 00:19:53,913 --> 00:19:54,303 Like what, 345 00:19:54,363 --> 00:19:59,363 what I use a lot, and we'll get into traits later, but I have to have, I'll 346 00:19:59,363 --> 00:20:03,473 have a suite that I do a lot of full stacks with stuff and so I'll have 347 00:20:03,523 --> 00:20:07,123 something that's only supported on Apple platform, so I end up having to do like 348 00:20:07,903 --> 00:20:13,783 pound if can import or something like that to kind of distinguish between the two 349 00:20:14,323 --> 00:20:15,493 SOI, for instance. 350 00:20:15,523 --> 00:20:20,413 And then I have to create a variable and then use that as a global variable 351 00:20:20,413 --> 00:20:22,423 that says whether it's available or not. 352 00:20:22,423 --> 00:20:25,273 which it's kind of a mess, but it works. 353 00:20:25,523 --> 00:20:28,093 We'll talk about traits more but back to like sweets. 354 00:20:28,483 --> 00:20:30,813 What do you think, what are some tips you have as far as how 355 00:20:30,813 --> 00:20:32,343 you should organize your tests? 356 00:20:34,540 --> 00:20:38,500 Yeah, I personally tend to just do one layer of sweets just because I would 357 00:20:38,500 --> 00:20:41,590 like to nest them for conditionals, but doesn't really work out well. 358 00:20:42,190 --> 00:20:46,830 So I just do one layer and then either organizing them with tags or using. 359 00:20:47,395 --> 00:20:49,395 good old Mark comments. 360 00:20:49,878 --> 00:20:52,513 Like I usually just do, I'm still stuck in the way of like. 361 00:20:53,248 --> 00:21:00,658 I'll have a test suite lined up with a type essentially and do it that way. 362 00:21:00,688 --> 00:21:04,168 Is that, do you feel like that's still a good way to go about 363 00:21:04,265 --> 00:21:05,045 that's how I do it. 364 00:21:05,865 --> 00:21:08,825 I'll have like a suite that's like, my object tests 365 00:21:09,228 --> 00:21:09,618 Okay. 366 00:21:10,035 --> 00:21:10,215 I'll 367 00:21:10,218 --> 00:21:13,428 And it's good for like code coverage too, because then you could 368 00:21:13,428 --> 00:21:15,138 tell, okay, what is this testing 369 00:21:15,188 --> 00:21:15,478 Yeah. 370 00:21:15,903 --> 00:21:18,003 But you could also do that, without the suite. 371 00:21:18,053 --> 00:21:23,023 It does make it nicer in terms of getting the, test diamond, on the 372 00:21:23,023 --> 00:21:27,323 left side of the editor because then you can just select everything there. 373 00:21:27,323 --> 00:21:29,863 But if you didn't want that, You could just have them all on top of 374 00:21:29,863 --> 00:21:33,253 the file and then have the file be your top level organization for it, 375 00:21:33,585 --> 00:21:34,005 Got it. 376 00:21:34,093 --> 00:21:36,223 but then you don't get like shared setup and so forth. 377 00:21:36,955 --> 00:21:38,005 Right, exactly. 378 00:21:38,885 --> 00:21:42,365 What are some other traits that people should be aware of? 379 00:21:42,628 --> 00:21:46,858 I would recommend making like a custom trait that automatically does the, like, 380 00:21:46,888 --> 00:21:50,848 oh, if Mac, if iOS, if Linux or whatever, 381 00:21:51,465 --> 00:21:52,665 How hard is it to do that? 382 00:21:52,738 --> 00:21:53,548 it's not that hard. 383 00:21:53,678 --> 00:21:57,658 you can, create a type that conforms to, you'll probably want to do 384 00:21:57,658 --> 00:21:59,518 both test trait and sweet trait. 385 00:22:00,038 --> 00:22:04,558 and if you do conform to sweet trait, You should implement the is recursive 386 00:22:04,558 --> 00:22:08,518 and then just have that be true is recursive means that for a sweet 387 00:22:08,518 --> 00:22:14,768 trait, it also applies that trait to every single like child of it, all the 388 00:22:14,768 --> 00:22:16,478 tests, all the sub suites and so forth. 389 00:22:16,525 --> 00:22:17,815 Yeah, that makes total sense. 390 00:22:18,278 --> 00:22:21,698 you can just have it say like, this also is a disabled trait, or like, 391 00:22:21,698 --> 00:22:23,348 this marks the test as disabled. 392 00:22:23,738 --> 00:22:24,638 A fun project for me 393 00:22:24,638 --> 00:22:27,218 yeah, but it shouldn't take that long to do. 394 00:22:27,895 --> 00:22:28,255 Okay. 395 00:22:28,348 --> 00:22:32,148 There are the, comment in bug traits, which just allow you to 396 00:22:32,148 --> 00:22:34,428 associate like text information. 397 00:22:34,953 --> 00:22:36,423 With a tester suite. 398 00:22:37,110 --> 00:22:37,400 Okay. 399 00:22:37,493 --> 00:22:44,563 There is the serialized trait which forces the test covered by that suite 400 00:22:45,143 --> 00:22:49,333 or the like parameterized test to run serially, so not concurrently, 401 00:22:50,115 --> 00:22:50,505 Okay. 402 00:22:50,773 --> 00:22:51,703 That's kind of weird. 403 00:22:51,793 --> 00:22:53,983 'cause that only applies to like those specific tests. 404 00:22:54,013 --> 00:22:59,053 if you have like two like sibling tests that both have like the serialized trait 405 00:22:59,593 --> 00:23:04,243 and you run them, but like the parents, the containing suite isn't serialized. 406 00:23:04,663 --> 00:23:08,683 If you run those, then like those two tests will actually run concurrently 407 00:23:08,683 --> 00:23:10,483 or like in parallel with each other. 408 00:23:11,413 --> 00:23:13,963 Should you be doing serialized tests? 409 00:23:14,740 --> 00:23:16,030 it's a thing that exists. 410 00:23:16,080 --> 00:23:18,885 Especially in a post Swift six world, like 411 00:23:20,007 --> 00:23:23,367 I haven't run into like a specific use case that needs it. 412 00:23:23,877 --> 00:23:27,147 I am a hundred percent certain that that use case exists, 413 00:23:27,147 --> 00:23:28,797 otherwise it wouldn't exist. 414 00:23:29,125 --> 00:23:29,475 right. 415 00:23:29,817 --> 00:23:33,857 But like every time that I've used that I've reached for it, it turns 416 00:23:33,857 --> 00:23:37,547 out that actually what I should have just done is made whatever the 417 00:23:37,547 --> 00:23:41,607 thing that was forcing the thing to be serial to actually make it. 418 00:23:42,312 --> 00:23:45,462 Be thread safe, like change a global variable into a task 419 00:23:45,462 --> 00:23:46,512 local or something like that 420 00:23:47,100 --> 00:23:54,000 so I did want to cover your article, which I'll put in the show notes, but 421 00:23:54,100 --> 00:23:58,870 what are some highlights about, what's new this year that folks should really 422 00:23:58,870 --> 00:24:01,120 start looking at using in their tests? 423 00:24:01,792 --> 00:24:07,882 Okay, so the major new change is exit tests, which doesn't apply to iOS. 424 00:24:08,842 --> 00:24:10,162 on the server side team? 425 00:24:10,639 --> 00:24:14,029 Yeah, service side, even Mac development windows and such. 426 00:24:14,509 --> 00:24:16,999 They allow you to verify that. 427 00:24:17,389 --> 00:24:20,629 Code exits properly as you expect 428 00:24:20,679 --> 00:24:21,159 code. 429 00:24:21,937 --> 00:24:23,137 returns an exit code. 430 00:24:23,137 --> 00:24:25,117 It throws 431 00:24:25,309 --> 00:24:25,559 Okay. 432 00:24:25,572 --> 00:24:26,262 or something like that. 433 00:24:27,044 --> 00:24:31,064 So you can basically say, Hey, make sure the signal is the right integer. 434 00:24:31,114 --> 00:24:32,044 Yeah, so you 435 00:24:32,317 --> 00:24:33,847 standard error context. 436 00:24:33,877 --> 00:24:34,087 What? 437 00:24:34,087 --> 00:24:35,197 How do you check that? 438 00:24:35,247 --> 00:24:35,847 so, 439 00:24:36,157 --> 00:24:39,427 The result has STA standard error content as a property. 440 00:24:39,559 --> 00:24:41,659 the example I wrote could have been better, 441 00:24:42,287 --> 00:24:42,617 Now. 442 00:24:42,677 --> 00:24:45,197 'cause I didn't understand what result was and I was like, I'm 443 00:24:45,197 --> 00:24:46,457 looking at this, and I'm like, okay. 444 00:24:46,757 --> 00:24:50,327 So result is basically some sort of structure, which has all that metadata 445 00:24:50,579 --> 00:24:51,299 Yeah, so 446 00:24:51,349 --> 00:24:51,769 okay. 447 00:24:51,932 --> 00:24:56,842 it has the, if you tell it, if you tell your exit test to grab 448 00:24:56,842 --> 00:24:58,057 the standard error, then it will. 449 00:24:58,882 --> 00:25:01,642 Then that, standard error content will be non nil. 450 00:25:02,242 --> 00:25:06,232 If you specify with standard output, it will also be a non nil thing. 451 00:25:06,812 --> 00:25:10,842 Those are just an array of, I think it's U eight but an array 452 00:25:10,842 --> 00:25:12,012 of, yeah, it's array of U eight. 453 00:25:12,012 --> 00:25:16,182 It's array of bites that if you wanna, you then have to like convert it to a 454 00:25:16,182 --> 00:25:18,762 string using one of the methods there. 455 00:25:19,642 --> 00:25:21,862 If you wanna do a string comparison. 456 00:25:23,072 --> 00:25:26,392 there are like some weird beta bugs in there right now where it will 457 00:25:26,392 --> 00:25:31,042 like, capture a lot more of the like standard error than you need. 458 00:25:31,042 --> 00:25:35,152 And that's just kind of how the, posik spawn, API 459 00:25:35,214 --> 00:25:35,689 Oh, really? 460 00:25:36,119 --> 00:25:38,824 So yeah, it's. 461 00:25:39,779 --> 00:25:41,099 But it works really well. 462 00:25:41,129 --> 00:25:43,679 I'm really impressed with how this works. 463 00:25:43,779 --> 00:25:45,639 you can't do this with XC test at all. 464 00:25:46,209 --> 00:25:48,399 Like Apple's never added support for this. 465 00:25:48,399 --> 00:25:52,779 So if you wanted to verify that your function had a precondition or 466 00:25:52,779 --> 00:25:56,109 such in it, and that precondition worked as you expected, you had 467 00:25:56,109 --> 00:25:58,119 to go way outta your way to like. 468 00:25:58,614 --> 00:26:02,514 Construct something to do that, which no one has ever done in their life. 469 00:26:02,922 --> 00:26:03,212 Yeah 470 00:26:03,564 --> 00:26:05,574 so it's really nice to have this. 471 00:26:06,177 --> 00:26:08,097 that sounds like a server side thing. 472 00:26:08,454 --> 00:26:11,569 Yeah, so really, I think I want it to come to iOS. 473 00:26:12,379 --> 00:26:17,119 the sandbox in iOS is a thing and that's gonna be difficult. 474 00:26:17,602 --> 00:26:19,312 what's the use case for iOS? 475 00:26:19,312 --> 00:26:23,332 Because like if the, the app shouldn't crash, I mean, it should just never crash. 476 00:26:23,382 --> 00:26:23,622 Right. 477 00:26:23,622 --> 00:26:27,522 But you do have APIs where it's like, Hey, I expect, for 478 00:26:27,522 --> 00:26:29,622 example, array subscript, right? 479 00:26:30,072 --> 00:26:31,302 So, or like subscript an array. 480 00:26:31,302 --> 00:26:36,542 If you pass negative one then, or if you pass like an out of bounds index, then 481 00:26:36,575 --> 00:26:37,325 fatal error, 482 00:26:37,442 --> 00:26:38,132 a fatal error. 483 00:26:38,852 --> 00:26:40,682 And like that. 484 00:26:41,807 --> 00:26:45,167 Whether or not it should or, or should return nil or do or like throw 485 00:26:45,167 --> 00:26:48,167 an error or something that you can actually handle is another question. 486 00:26:48,167 --> 00:26:51,977 But that sort of thing exists and we should be able to like handle 487 00:26:51,977 --> 00:26:54,497 that test case just to have like that coverage that it's working. 488 00:26:54,905 --> 00:26:56,075 I was about to ask that. 489 00:26:56,105 --> 00:26:59,965 'cause a thousand years ago when I did a, an episode with on this very 490 00:26:59,965 --> 00:27:04,735 subject, I was asking like, how do you test for fatal error or precondition 491 00:27:04,735 --> 00:27:06,385 or assert or any of that stuff? 492 00:27:07,075 --> 00:27:10,405 And the way we've had to do it is basically have a way to 493 00:27:10,405 --> 00:27:12,385 like overload it essentially. 494 00:27:13,135 --> 00:27:14,755 But we don't have anything like that. 495 00:27:14,755 --> 00:27:19,105 And this, which what it sounds like is this would work in that case, but then 496 00:27:19,105 --> 00:27:20,840 at the same time it wouldn't work on iOS. 497 00:27:21,020 --> 00:27:21,520 Is that correct? 498 00:27:21,922 --> 00:27:27,112 Yeah, so the way that I would approach this is I would, I would either just 499 00:27:27,112 --> 00:27:31,072 not, or I would like put in like an inject, like a, an injectable using 500 00:27:31,072 --> 00:27:36,272 like dependency, injecting like a shim that just in actuality it calls, as 501 00:27:36,272 --> 00:27:39,302 you were just saying, like it just calls precondition and actuality. 502 00:27:39,302 --> 00:27:41,912 And then in test we can examine that it was called that. 503 00:27:42,722 --> 00:27:47,072 For this I might either write, like if it's a cross platform app, like 504 00:27:47,072 --> 00:27:53,082 I wanna have it run on Mac OS and iOS then I would just have the. 505 00:27:53,982 --> 00:27:58,822 Have it be covered under the test for macOS with like a pound if or 506 00:27:58,822 --> 00:28:01,882 like if os macOS stuff covering that. 507 00:28:03,412 --> 00:28:06,832 There's a few things that are promising that isn't in there yet. 508 00:28:06,982 --> 00:28:11,112 The thing that I didn't write that I'm most excited for is a thing 509 00:28:11,112 --> 00:28:12,682 called, issue handling traits. 510 00:28:13,612 --> 00:28:18,252 And these allow you to filter or modify issues. 511 00:28:18,832 --> 00:28:22,553 So like if you have a pound expect that fails, like it. 512 00:28:23,182 --> 00:28:25,672 Pound expect true, equal, equal false. 513 00:28:25,912 --> 00:28:27,602 That's, guaranteed failure there. 514 00:28:28,022 --> 00:28:33,772 Then it allows you to, specify a trait that allows you to modify the issue. 515 00:28:33,772 --> 00:28:36,652 You might wanna convert the, severity of it. 516 00:28:36,712 --> 00:28:37,972 you might wanna just entirely 517 00:28:38,125 --> 00:28:38,665 Yeah. 518 00:28:38,755 --> 00:28:39,265 Okay. 519 00:28:40,312 --> 00:28:42,172 That's really powerful. 520 00:28:42,172 --> 00:28:43,822 I don't think it's gonna happen in Swiss six two. 521 00:28:45,580 --> 00:28:48,505 But you think it's possible in the next six 522 00:28:48,555 --> 00:28:49,825 Six probably six three. 523 00:28:49,825 --> 00:28:51,745 It's actively being discussed. 524 00:28:52,385 --> 00:28:52,756 Yeah, 525 00:28:52,806 --> 00:28:58,521 there's other stuff there that's, exciting, from what it enables, like, 526 00:28:58,561 --> 00:29:02,911 range confirmations, which were merged in, Swift six one and have been out 527 00:29:02,911 --> 00:29:05,101 since Xcode 16, three earlier this year. 528 00:29:05,801 --> 00:29:10,141 This is a thing that they view as necessary for automation tests. 529 00:29:10,788 --> 00:29:15,108 So range confirmations is basically just, you expect the callback to be 530 00:29:15,108 --> 00:29:17,838 called within some amount of range. 531 00:29:18,258 --> 00:29:24,218 And the example given in the proposal is using an automation test of 532 00:29:24,218 --> 00:29:25,988 like, oh, you have a button and. 533 00:29:26,723 --> 00:29:32,753 You can expect it to be tapped, clicked once, but what if the user 534 00:29:32,933 --> 00:29:36,173 actually interacts with the app at that point, like a human operator on 535 00:29:36,173 --> 00:29:37,343 the simulator or something like that. 536 00:29:37,763 --> 00:29:42,533 Like in that case, you'll get a Furious tap event and it'll be or you'll 537 00:29:42,533 --> 00:29:45,223 get like a tap event that the test wasn't expecting and then you'll get a 538 00:29:45,223 --> 00:29:47,333 failure that you don't really care for. 539 00:29:47,783 --> 00:29:53,403 There's also, in that same vein, attachments, were added, I'm just 540 00:29:53,403 --> 00:29:54,423 jumping around in this article. 541 00:29:54,473 --> 00:29:57,593 So is, is attachments like any sort of like metadata 542 00:29:57,593 --> 00:29:58,998 you want to attach to a test? 543 00:29:59,256 --> 00:29:59,646 Yeah. 544 00:30:00,406 --> 00:30:03,556 So this will be, this is already pretty useful to have as like. 545 00:30:04,211 --> 00:30:08,231 I can attach like a generated JSON payload in the test. 546 00:30:08,231 --> 00:30:14,121 And then for debugging reasons, I want to inspect what this JSON was so that I can 547 00:30:14,661 --> 00:30:18,831 more easily see like, oh, this is actually an issue with my production code, or this 548 00:30:18,831 --> 00:30:20,511 is an issue with the test, or so forth. 549 00:30:21,301 --> 00:30:21,691 But. 550 00:30:22,351 --> 00:30:26,071 They're also really useful in automation tests where they're used for storing 551 00:30:26,531 --> 00:30:31,461 screenshots of what the app was doing, at the time, or even videos and such. 552 00:30:31,461 --> 00:30:35,631 I don't remember if the current code does this, but I know that 553 00:30:35,631 --> 00:30:38,781 Xcode 26 supports recording video 554 00:30:39,171 --> 00:30:42,831 Like failing automation tests, which is really cool. 555 00:30:43,161 --> 00:30:43,491 Yeah. 556 00:30:44,583 --> 00:30:47,453 That's a thing that is probably not necessary, but makes the 557 00:30:47,453 --> 00:30:51,673 experience of debugging automation tests much easier, especially when 558 00:30:51,673 --> 00:30:52,633 these things are running on ci. 559 00:30:54,913 --> 00:30:58,393 both of those are things that I'm really excited for in terms of what they enable. 560 00:30:58,973 --> 00:31:00,533 Let's talk about it. 561 00:31:00,633 --> 00:31:03,053 Actually before we get into that, you wanna talk a little bit more 562 00:31:03,053 --> 00:31:07,313 about specifically your proposal regarding polling confirmations? 563 00:31:07,670 --> 00:31:08,210 Yeah. 564 00:31:08,215 --> 00:31:11,805 one of the libraries that I maintain is called Nimble. 565 00:31:12,225 --> 00:31:18,295 Nimble is a library that I believe, Swift Testing took a lot of, or at least 566 00:31:18,295 --> 00:31:20,305 the pound expect and pound require. 567 00:31:20,575 --> 00:31:24,955 APIs took a lot of inspiration from, I haven't asked to confirm this 568 00:31:25,505 --> 00:31:31,275 but that provides a, syntax for, it's also inspired by ARS Spec. 569 00:31:31,275 --> 00:31:33,195 So, or Nibble is inspired by ARS Spec. 570 00:31:33,195 --> 00:31:34,905 So like there's that too. 571 00:31:35,385 --> 00:31:40,095 It provides just a syntax for that's better than the XCT assert, in my opinion. 572 00:31:40,555 --> 00:31:46,450 For comparing v. Asserting that some behavior happened, something happened. 573 00:31:47,060 --> 00:31:50,910 one of the features of Nimble is something that I call pulling expectations, 574 00:31:51,600 --> 00:31:57,260 which is like the form is, expect something to eventually something else. 575 00:31:57,740 --> 00:32:01,980 So the idea of a polling expectation is that it will 576 00:32:01,980 --> 00:32:03,600 run the thing that you expect. 577 00:32:03,600 --> 00:32:06,930 the value being sent into expect is actually a closure. 578 00:32:06,930 --> 00:32:12,880 So it'll run that closure many times pulling it continuously until it passes 579 00:32:12,910 --> 00:32:14,350 the matcher, in this case, equal two. 580 00:32:15,530 --> 00:32:18,740 So in this case, or in the example I gave, that would always fail, but if you gave 581 00:32:18,740 --> 00:32:22,150 it like a method and then that method might change, how its return value is 582 00:32:22,150 --> 00:32:25,280 over time then that might eventually pass. 583 00:32:26,090 --> 00:32:30,080 This is something that like I think is really necessary 584 00:32:30,140 --> 00:32:32,310 for Swift Testing, to support. 585 00:32:32,730 --> 00:32:32,915 You can. 586 00:32:33,930 --> 00:32:37,440 Kind of do it as a third party, but as I found it actually implementing this 587 00:32:37,800 --> 00:32:44,790 as a pitch there are problems with the more naive solution or approach to this. 588 00:32:45,890 --> 00:32:50,610 So yeah, in the article I give an example of like, all right, you call 589 00:32:50,660 --> 00:32:52,790 some method on a background task. 590 00:32:53,450 --> 00:32:56,620 And then in the test you wanna say like, I want to. 591 00:32:56,980 --> 00:33:02,460 Confirm that this eventually, you know, the method being run in the background 592 00:33:02,510 --> 00:33:03,920 that it eventually does its work. 593 00:33:04,670 --> 00:33:06,800 And that's precisely what this is doing. 594 00:33:07,460 --> 00:33:11,510 like in your example, it's like it keeps calling raised dolphins until the 595 00:33:11,510 --> 00:33:13,205 account gets up to one, Is that correct? 596 00:33:13,332 --> 00:33:17,242 So what happens is that raised dolphins is called once but. 597 00:33:18,472 --> 00:33:18,982 As part of 598 00:33:19,100 --> 00:33:19,520 Okay. 599 00:33:19,622 --> 00:33:23,242 yeah, as part of, and as part of like the internal implementation of it, it might 600 00:33:23,302 --> 00:33:26,242 increment the number of dolphins on it. 601 00:33:26,332 --> 00:33:30,802 And so then you're just checking like that is that eventually there 602 00:33:30,802 --> 00:33:32,032 is at least one dolphin there? 603 00:33:32,082 --> 00:33:36,217 I assume it would take some sort of time interval or like duration. 604 00:33:36,487 --> 00:33:37,387 'cause yeah. 605 00:33:37,477 --> 00:33:37,807 Yeah. 606 00:33:37,857 --> 00:33:38,967 really interesting, 607 00:33:39,040 --> 00:33:39,310 yeah. 608 00:33:40,775 --> 00:33:46,145 That's been like a thing of, that's the, time interval thing of like 609 00:33:46,145 --> 00:33:50,525 when to cut this off has been the most eye-opening thing about this 610 00:33:50,635 --> 00:33:51,115 right? 611 00:33:51,412 --> 00:33:55,792 Swift Testing submits every single one of your tests to be run at once, 612 00:33:55,822 --> 00:33:58,862 and it submits it all to the Swift concurrency shared thread pool. 613 00:33:59,402 --> 00:34:03,842 So on your system, you might have that shared thread pool, might have like 614 00:34:03,842 --> 00:34:11,192 10 threads available, but if you have like 300 or so tests, you know, to run, 615 00:34:11,732 --> 00:34:14,882 Then it's gonna run those all and they might all be running concurrently, or 616 00:34:14,882 --> 00:34:18,452 at least all spend some amount of time on that, shared thread pool before, 617 00:34:18,452 --> 00:34:20,012 a wait causes the next one to run. 618 00:34:21,252 --> 00:34:24,802 And so that prevented an issue with the timeout. 619 00:34:25,262 --> 00:34:28,342 Or the desire to have this automatically timeout after like one second. 620 00:34:28,852 --> 00:34:34,422 If you have 300, 3000, you know, 10,000 or whatever tests to run on it. 621 00:34:35,082 --> 00:34:40,482 And then your test gets run, you then like await to as part of this. 622 00:34:41,382 --> 00:34:42,612 There's no guarantee. 623 00:34:42,642 --> 00:34:45,822 And in fact, the more tests you have, the more likely, and especially in 624 00:34:45,822 --> 00:34:50,482 like resource constrained environments like ci there's no guarantee that 625 00:34:50,482 --> 00:34:54,802 you'll actually get that callback within that, like one second timeout. 626 00:34:55,672 --> 00:34:59,772 So on like extremely large on extremely large test suites, this will. 627 00:35:00,232 --> 00:35:04,012 Polling confirmations as I originally implemented it, I have fixed that now. 628 00:35:04,472 --> 00:35:08,072 Will be increasingly flaky and unreliable. 629 00:35:08,572 --> 00:35:08,962 Okay. 630 00:35:09,172 --> 00:35:10,552 That makes sense. 631 00:35:10,594 --> 00:35:12,004 Yeah, disappointing. 632 00:35:12,054 --> 00:35:12,474 Yeah. 633 00:35:13,374 --> 00:35:17,304 before we close out, a lot of people are curious about like CUI tests. 634 00:35:18,064 --> 00:35:24,544 They're still very much tied to the old Objective C APIs and things like that. 635 00:35:24,544 --> 00:35:28,174 Where, what do you see as like the future of that as far as 636 00:35:28,294 --> 00:35:30,274 being more Swift friendly, 637 00:35:30,324 --> 00:35:32,394 I mean, it's gotta be like Swift Testing support. 638 00:35:32,724 --> 00:35:37,374 There's a lot of, it is a lot of those APIs are very extremely typed. 639 00:35:38,134 --> 00:35:42,724 So ideally there'll be some way to make them. 640 00:35:43,339 --> 00:35:45,289 Not be as much there is. 641 00:35:46,249 --> 00:35:52,009 I don't really see how, I guess there is the, what is it, the dynamic callable or 642 00:35:52,009 --> 00:35:58,669 like the dynamic property accessor from a while ago that could be used to at least 643 00:35:58,669 --> 00:36:01,789 make it feel more Swifty, even if it doesn't actually provide any guarantees. 644 00:36:02,939 --> 00:36:06,119 that might be it in terms of like making it feel more like idiomatic Swift. 645 00:36:06,879 --> 00:36:10,989 But like I think what they're more interested in doing is 646 00:36:10,989 --> 00:36:15,534 enabling, like using the automation framework with Swift Testing. 647 00:36:16,034 --> 00:36:16,324 Okay. 648 00:36:16,917 --> 00:36:19,887 Because one of the things that they did, I don't remember whether it was 16 three or 649 00:36:19,887 --> 00:36:25,817 16 four, but they pulled out or Xcode 16, three or four they pulled out the, audit. 650 00:36:26,297 --> 00:36:31,367 They made a new XC automation framework, which is separate from XC test, which 651 00:36:31,367 --> 00:36:34,577 is another one of those reading between the lines of like, they're clearly 652 00:36:34,577 --> 00:36:38,717 going in the direction of enabling automation tests with Swift Testing. 653 00:36:39,767 --> 00:36:42,887 What is the automation framework exactly? 654 00:36:43,214 --> 00:36:46,104 It's all of the XCUI test APIs. 655 00:36:46,614 --> 00:36:51,324 So it's like the XCUI element, so you can get an element by 656 00:36:51,324 --> 00:36:53,164 the, accessibility identifier. 657 00:36:53,744 --> 00:36:56,974 it's like the XCUI application and interacting with it through there. 658 00:36:57,784 --> 00:37:01,969 Is that used outside of testing for like Mac OS apps. 659 00:37:02,676 --> 00:37:03,306 No, it's not. 660 00:37:04,086 --> 00:37:06,606 It's entirely meant for use with automation tests. 661 00:37:07,689 --> 00:37:10,059 Because I know like there's apps that I'll always ask for, 662 00:37:10,119 --> 00:37:13,119 like accessibility, access. 663 00:37:13,219 --> 00:37:17,429 That is a different, that is also it's a different automation library from Apple. 664 00:37:17,849 --> 00:37:18,329 Great. 665 00:37:19,019 --> 00:37:23,099 so that is much more of like just being able to interact 666 00:37:23,099 --> 00:37:24,179 with the system through the. 667 00:37:25,109 --> 00:37:26,939 Automation A PII believe. 668 00:37:26,999 --> 00:37:32,009 I wouldn't be surprised if XE automation is fundamentally using 669 00:37:32,009 --> 00:37:33,809 like those same APIs under the hood, 670 00:37:34,142 --> 00:37:34,712 Under the hood. 671 00:37:34,712 --> 00:37:36,962 but I don't have any way to verify that. 672 00:37:37,262 --> 00:37:40,652 Anything else you wanted to talk about, especially with dub or 673 00:37:41,072 --> 00:37:42,992 testing before we close out? 674 00:37:43,492 --> 00:37:46,722 Yeah, The other stuff I'm really excited for all of the, concurrency 675 00:37:46,722 --> 00:37:48,072 improvements with Swift six two. 676 00:37:49,062 --> 00:37:53,772 I've been diving into those and just been really excited for that. 677 00:37:53,892 --> 00:37:58,512 The defaulting to running everything or constraining everything to the main actor. 678 00:37:58,754 --> 00:38:02,324 As opposed to non isolation is something that I'm excited to see. 679 00:38:03,054 --> 00:38:05,214 Though it is right now very buggy. 680 00:38:05,844 --> 00:38:08,814 But you know, it's also just happened. 681 00:38:08,814 --> 00:38:10,974 It's early in the beta season. 682 00:38:11,394 --> 00:38:12,564 We'll see how it turns out. 683 00:38:13,584 --> 00:38:16,074 Well Rachel, thank you so much for coming on the show. 684 00:38:16,324 --> 00:38:18,784 it's great to have you on and talk about this stuff. 685 00:38:19,364 --> 00:38:20,714 Where can people find you online? 686 00:38:20,997 --> 00:38:22,347 yeah, I am. 687 00:38:22,427 --> 00:38:25,007 my blog is at rachel brindle.com. 688 00:38:25,517 --> 00:38:29,937 I am on, Mastodon at unata@hackderm.io. 689 00:38:30,487 --> 00:38:35,257 And I. Instagram, though, that's mostly flying stuff now. 690 00:38:35,567 --> 00:38:36,557 at Rachel Brindle. 691 00:38:37,307 --> 00:38:37,597 Yeah. 692 00:38:39,054 --> 00:38:39,984 Thank you again. 693 00:38:40,084 --> 00:38:42,994 we'll put links to the proposal, your blog post, all that 694 00:38:42,994 --> 00:38:45,664 stuff in the show notes below. 695 00:38:46,174 --> 00:38:51,614 People can find me on Mastodon at Leo G. Dion at c Im I'm on 696 00:38:51,614 --> 00:38:53,864 Blue Sky as well at Leo g Dion. 697 00:38:54,464 --> 00:38:55,574 Take a look at me there. 698 00:38:55,664 --> 00:38:56,894 Links are in the show notes. 699 00:38:56,924 --> 00:38:59,504 If you really enjoyed this episode, please like and subscribe. 700 00:39:00,084 --> 00:39:03,174 Early access is to those who joined Patreon, so thank you 701 00:39:03,174 --> 00:39:04,824 for all the Patreon supporters. 702 00:39:04,824 --> 00:39:05,724 It's super helpful. 703 00:39:06,324 --> 00:39:09,534 Thank you so much and I look forward to talking to you in the next episode. 704 00:39:09,564 --> 00:39:10,164 Bye everybody.