Join us as we talk about our build system move to an Orka cluster consisting of 8 Mac Studios, our ongoing "Ready for Swift 6" project, to get an update on documentation coverage across the package ecosystem, and to hear our regular package recommendations.
News
- Plotting a Path to a Package Ecosystem without Data Race Errors
- Switching to ephemeral macOS build runners
Packages
- swift-testing-revolutionary by Kohki Miki
- Soto by Adam Fowler and the release blog post
- SwiftSessions by Alessio Rubicini
- Tabular by António Pedro Marques
- CoreXLSX by Max Desiatov
Creators & Guests
What is Swift Package Indexing?
Join Dave and Sven, the creators of the Swift Package Index open-source project, as they talk about progress on the project and discuss a new set of community package recommendations every episode.
Are you following the Euro, by the way, the football?
I'm not a big football fan. I did watch the last half and the extra time and the penalties of the
Switzerland game, though, which was very exciting. And I'm not a football fan, but I believe we don't
have a great track record at penalties. I think we did exceptionally well in that Switzerland game.
I was going to say England winning on penalties is quite the occasion.
It's unheard of, right?
Yeah, practically. And we're set up for my adopted country, France, possibly going up against
England. Who knows? Although going past Spain...
Oh, really? Is that what's next?
Well, it's not next.
I have no idea what the schedule is.
France have to win against Spain, which is a tough one. And England will have to win against,
and I'm blanking who their next opponent is. So yeah, there's still steps to go.
But it might be a nice, interesting final.
No, I've been watching more Wimbledon than I have football. But I think all the British names are
even knocked out of Wimbledon by this point at this stage. So that's a shame.
Yeah, the French as well, I believe. Yeah, just the last one dropped out yesterday. And
I've been seeing a lot of rain in SW19.
Yes, yeah, there has been a lot. In fact, I think I was reading the other day that they were going to
consider reducing the length of some of the matches to try and fit it all in this week. So
I don't know whether that's gone ahead or not, but that would be, I think,
unprecedented if they do that.
But how would they do that? Like up to four games only or super tie breaks?
What's the solution there?
Yeah, I think tie breaks after two matches were on the table.
Oh, wow. Okay.
So I don't know what the, sorry, not matches, sorry, two sets.
Interesting. Well, let's see how it goes.
But I'm not sure what the conclusion of that was. But you didn't tune into this podcast to learn
about football or tennis, I hope, because you're in the wrong place if you have a football and
tennis talk. What we can talk about this week is two blog posts, actually. So we published,
first of all, a blog post on SWIFT.org about the Ready for SWIFT 6 project that we've been
working on this summer. And I won't go into huge detail about that because we've talked about that
at length on this podcast. But there is a blog post there that gives a bit of an overview about
what we're doing and how we're representing that data on the package index package pages and things
like that. And so that's up on the SWIFT Language blog, if you haven't yet checked out that page or
are interested in some of the work that went in behind the scenes on that.
And then the second blog post is one that we published just yesterday, which is about a new
expansion to our build infrastructure that we use to do all of our compatibility build testing
and documentation building and all of that. So we have, since the beginning,
used a mixture of machines because we have Mac machines, which you need for performing
the Apple platform builds, Mac OS, iOS and friends. And we run some Linux builds using
Docker on Microsoft Azure, which is kind of separate, but all part of the same build system.
And up until quite recently, we've had several Mac mini machines, just kind of bare metal,
manually configured Mac mini machines that we've been running on. And they have been,
I mean, they've been amazing. We started off with two Mac minis at the beginning, and
they ran the initial set of builds, plus hosted the entire website at that point, didn't they?
Yeah. And they were Intel Mac minis, right? Because that was actually quite a bit slower
compared to the first M1 Mac mini we got once those came out.
Yeah, we had the most overloaded web server ever when we first launched that build system,
because it was trying to serve the web page and constantly process builds in the background.
And over time, we have separated that out. The web server is no longer on
that infrastructure anymore. That's running in Azure too. And the Mac mini machines became
dedicated build machines that do nothing but build packages. And that was great. That was a
great step forward. But of course, there's a couple of restrictions with that. First of all,
it's all manually managed. So maintenance each week was making sure that the machines were still
stable, because running hundreds and thousands and tens of thousands of builds per week
on the same machines can be a source of instability. And making sure they're up to
date with updates and all that kind of maintenance stuff that you have to do.
But also, there were some limitations in that system. Because we were running each of those
compatibility builds on the same machine as other compatibility builds, there was no kind of
isolation between those environments. So we never hit insurmountable problems with this.
But it was always a worry that either someone would find some way to break out of the Swift
compiler sandbox and be able to do something to the Swift compiler that would disrupt or damage
the machine. And thankfully, we never had that situation either. But again, that was always a
worry. But secondly, just build environment isolation is a nice thing to have. To know that
your build is not going to be impacted by something that some other build did at some point.
So we had a really great system. And don't get me wrong, the system worked incredibly well
for years. But it was never ideal. And in a conversation where we were talking to Macstadium
about capacity, we ended up talking about their Orca system. So their Orca system is a
Kubernetes system for virtualizing macOS. So I think it's Orchestration...
What's the... Do you remember the abbreviation, the acronym?
>> For Kubernetes? >> No, for Orca.
>> Oh, the acronym. Oh, I'm sorry. No, I don't know. I don't know what Orca...
Right, that isn't actually just the whale, is it?
>> No, no. Well, I mean, it is a whale. It's VM Orchestration with Kubernetes on Apple.
So they have this orchestration system for automating the creation and deletion of
virtual machines on native Mac hardware in a kind of automated way. And so we ended up talking about
that system with them and whether we could switch to an ephemeral build system, which is
that every single build that we attempt gets a completely fresh VM, starts the VM up,
executes the build, and shuts the VM down at the end of it. And that gives us total
predictability because you're starting from the same exact place on every image every single time
because the changes are not persisted back to the image. And it also gives us complete isolation,
both from the impacts of other builds doing things to affect this build, but also of malicious builds.
They would have to not only break out of the Swift compiler, but they would have to break
out of a virtual machine as well, which I think is almost impossible. Never say anything security
wise is impossible, right? But I think it's as good as. Yeah, absolutely. And I think one huge
advantage, which we didn't have to do it often, but sometimes we had to run builds in the actual
environment. And that was always, you know, it meant to SSH into one of the builders, pick the
right one, check out stuff there. And, you know, you always sort of kind of worried, well, is this
going to be different because I'm now doing it in a different directory than normally?
You know, it's this here, we can just spin up the exact same copy of a machine,
connect to it and run the build just as it normally would. And you can have really high
confidence that you're actually looking at the same thing and debugging the same thing.
And that's just a big help. But I think if you ask me what's the biggest advantage is beyond,
you know, the maintenance that you already talked about, this debugging is actually our
capacity management, because we now have the situation where every machine in the cluster
can run any build. And previously, that was not the case. Every physical Mac that we had set up
had a specific macOS version, and then one or two Xcode versions. I think that's sort of the maximum
we ran with. But we sort of had to configure them in certain ways. And not every Xcode version can
run on every macOS version. So we had to be, we had to choose what we deploy. And we never had a
great mix where every Swift version was equally available across all machines. There was always
one machine that was handling, like two Swift versions, where the other two were only handling
one version. And you can imagine if lots of builds are coming in, that one machine will be struggling
while the other two are underutilized. And that's great right now, because as soon as something
shows up that needs building, any machine can participate. And not only that, it also means that
in case we need more capacity, it's really easy to just add it in general, and it'll benefit all
Swift versions. So we never need to worry about specifically setting up certain Swift versions
and their availability anymore. And that's just a huge relief. And you see the benefit as we
right now do our Swift 6 reprocessing. So over the summer, we're continuously hitting the system
with lots and lots of builds of a very specific Swift version. We can just run full throttle,
and we don't need to tweak anything about our setup. Like previously, we put up a couple of
extra Swift 6 builders to help us deal with that immense backlog as soon as the new Swift 6 version
came out. Now we don't have to, we just have everything available and we can just run with it.
And that's just a huge relief. And that situation was quite extreme at times. Sometimes we needed
three different host operating systems to run four different versions of Xcode. So at one point,
we were running Monterey, Aventura, and Sonoma, with Monterey running one Xcode version, Sonoma
running one Xcode version, and Aventura running two. I think that was when we were running five,
six and above. So yeah, it was never even load. And now as part of this Orca system, we get to,
on a job by job basis, say, let's spawn a Swift 5.7 build system, or let's spawn a Swift 5.10
build system. And it comes up in a remarkably short amount of time. I did a little testing
for the blog post as I was writing it. And the average time to spawn one of these VMs is three
seconds, or in fact, under three seconds, it was like 2.8, 2.9 seconds, which I mean, I know it's
not actually booting the machine, it's all saved state and it's just restoring that state into
memory. But still, that is an impressive amount of time to have a fully usable, not container,
but virtual machine running macOS. Yeah, I mean, I think they're sort of very similar because,
you know, on macOS, if you run Docker, you're actually running a virtual machine.
On Linux, it's a bit different, but just in terms of how you use it, if you're not familiar with
Kubernetes and Orca is sort of a flavor of Kubernetes, I've not actually used pure,
if you will, Kubernetes. But you can think of it just as if you were running a, or spinning up a
Docker container here. That's effectively what we're doing, except it's Mac machines that are
coming up and all the rest really behaves pretty much like that. And as my understanding is also
like the Orca commands you run and the output you get is effectively like Kubernetes. So if you're
familiar with Kubernetes, Orca will be really familiar to you, I believe. It's a really nice
system. And we should talk a little bit about the hardware that's running as well, because the
Mac minis that we had previously have had a little upgrade. And we're now running on...
They've bucked up, yeah.
Yeah, they certainly have in style too. We're now running on Mac Studio machines. So we're
actually running M1 Mac Studio machines, but with M1 Ultra chips in them and 128 gig of RAM each.
So what that actually means is because there is a limitation on all virtualization on Mac,
and that is that Apple have a limitation of no more than two virtual instances of Mac OS
running on a machine simultaneously. So what we're doing is we're splitting those Mac Studio
machines straight down the middle with 10 cores per VM and 64 gig of memory per VM, which is still
an enormous amount for a Swift compiler package. Even the largest packages are not going to run
out of memory on our new VM based system. So yeah, and we have eight of those Mac Studio machines
for a total of a terabyte of memory, which is kind of mind-blowing. And I think 160 cores
of CPU processing, which gives us a total concurrency of 16 builds running at the same time.
Yeah. And that's, as I said, the real shame that we can't boost that up a little higher.
Yes. If anyone from Apple is listening, please drop us a kind word with whoever is responsible
for that saying that we would really appreciate it because it's actually, it's worth talking
about that for a second. We are able to, especially when we're doing these Swift 6
processing runs where we fill up the backlog with a hundred thousand jobs ready to process,
we are able to saturate our cluster with work to do for more than seven days of processing time.
You know, we queue up a hundred thousand jobs and it takes about seven days to clear that backlog.
And we had a look at the underlying CPU usage during that seven day period. And the underlying
CPU usage was averaging out between 30 and 40% across the entire Mac Studio cluster.
If we were able to spawn more VMs within our cluster, like more VMs per host,
then we could even that out and drastically make our build environment more efficient.
Because the build process has several stages. The build process is a Git checkout and then a Swift
compile, and then a documentation build, possibly if your package author opts in, and then uploading
that documentation to S3 and then reporting back results before kind of finishing the build.
And there are periods in that build process where the CPU is not very busy. So during the clone and
during the documentation upload at the end, the CPU is effectively sitting idle. If we could have
more virtual machines per node, we could even that CPU out significantly on the hosts and get more
work done with the same amount of hardware, which would be something that we'd love to do. So I'm
fairly confident the person responsible for that decision is not listening, but just in case they
are. - Send us the secret user defaults command to disable that limit. - Or send us the secret user
defaults command, yeah. I'm sure if you disable system integrity, whatever, you could probably
find a way to make it work, but that is not what we're doing. We are faithfully following the
license agreement that we agreed to. So the good news is that we've been doing
quite a bit of testing with this system before we put it live. And we ran a lot of the early Swift6,
ready for Swift6 build queues with this cluster before we switched over our production workload
onto that as well. And it has just been an absolute dream to work with. There was a little
bit of software development that we needed to do to get it all working. We need to
write a fairly small piece of software that orchestrates the cluster, basically. So it reads
incoming jobs that it needs to process, decides which VM type to spawn, spawns the VM,
SSHs into it, executes the build commands, and then at the end shuts that machine down
and reallocates the resources back to the cluster. But apart from that, it's been fairly
easy. I don't think we needed in the end any changes to our builder tool. Is that right,
Sven? No, in fact, what made it a bit more complicated, I think, was that for a time,
we actually ran both because we always had builds coming in. So it was really hard. It wasn't really
practical to say, well, we're not going to process builds for a few days while we sort this out.
So the way we set this up, that we actually ran both systems at the same time. And you can imagine
that's a bit more complicated because it's a very different way of bringing up jobs,
because one machine has a GitLab runner permanently running that picks up the jobs,
and the runner knows exactly what machine it's on. In the other case, the machine is ephemeral,
so there's nothing running to pick up jobs. So something else needs to do that. And we sort of
had to move the logic that sits in the runner into this orchestrator and driver. And at the
same time, not pull out the old one. So we needed to do both things at the same time while we were
transitioning over. And that was the fiddly bit that also worked out quite well. So after we
figured out how to do that, that really allowed us to take our time in transitioning over and making
sure it works as we expected. Because in a system like that, you don't really want to go back and
forth, right? You don't want to transition too fast and then realize, "Whoa, whoa, whoa, hold on,
this isn't actually working." And there were a couple of cases where it did not quite work out
all right in our first attempt. And we needed to sort of stop processing for a bit and redo some
things. This was mainly about our stuff where we didn't report the results back properly or the
queues didn't see the build failure. So they thought that we're actually okay. And stuff like
little fiddly bits that get wrong when you... I mean, everyone who's ever set up a CI system knows
exactly what I'm talking about, right? You think you have it running, then you send the job to CI,
everything's green. And then you look at the logs and you see, "Oh, it actually failed." And you
didn't catch the last build result and that sort of stuff. And when you run lots of builds,
you kind of don't want to get that wrong. And that was the fiddly part.
- Not content with setting up our own CI for our tests, we decided to set up CI for the entire
package ecosystem. - Yeah, well, actually, we're not,
we're just running builds. I mean, we're never going to run tests. I'm going to say that here
now, but just the builds is fun sometimes. - Exactly. So I think it's
a great upgrade of our both capability and also the kind of stability and isolation of our build
environment. I sleep completely at ease these days with knowing that there's zero risk of
waking up to a build machine which has been compromised.
- Yeah, that's actually a good final point maybe to raise because one of the issues we did
occasionally see that was a problem with the old system was when one of our builders fell over and
it was a specific Swift version that was only served by one builder. It could actually happen
that our queue filled up with those builds and then we wouldn't actually continue processing.
And that's actually something that can't happen anymore. And that's, I think, the key thing that
we're actually gaining operationally apart from all the maintenance stuff and debugging
that really, you know, a couple of times caused trouble with the old system.
- So we'd like to say a huge thank you to Max Stadium. They have been
supporting this project since the beginning. And if they have a very generous open source
program actually, which is available to any open source project, which is if you require a
Mac mini or something like that for your project, if GitHub Actions are no longer
cutting it for what you need to do in terms of CI, they offer an open source program to provide
that kind of resource for free. And we'll put a link in the show notes to that. But also, obviously,
it's a little bit of a different situation having a few Mac minis to eight Mac studios,
but they continue to support us by making this possible. So, you know, not to go into the full
details of it, but this would not be possible without their support. So thank you so much for
everything you've done for us. And more importantly than the supply of the resources is
every single conversation we've had with them has been met with, well, let's figure out a way to do
this. So when we started talking about ephemeral builds, it wasn't a case of, well, are we actually
going to be able to keep this within a sensible budget, but just how are we going to do it? Is it
going to be the best solution? And I've kind of worked from there. So I can't say enough good
stuff about actually both Mac Stadium and Azure, they are both exceptionally accommodating with
every request that we've made to them. Yeah, absolutely. Huge, huge. Thank you.
All right.
Right. There's another piece of news while we were talking about the build system. Maybe that's
something worth highlighting right now. Because this week we crossed a nice threshold and that's
10% of all packages are now opting into host documentation on the Swift package index. And I,
I was sort of tracking this number. So I saw it coming. There's been a really interesting
increase recently in a number of packages opting in like new packages, the ratio is quite high.
Well, higher than 10%. I mean, that's obviously how we're catching up starting from zero. But I
sort of was thinking back when we rolled out documentation hosting, I thought, well, you know,
getting a couple of percent would be nice, you know, like maybe a couple of hundred packages.
And now we're, we've crossed the 750 mark and the 10% mark. That's actually quite extraordinary how
many authors are actively opting in because this is not something we are doing right. Every package
author needs to add this SPI.yaml file to their repository. I mean, it's not a huge change,
but it is something you actually need to be doing and be looking at.
It takes a git commit.
Yeah. That's the thing. Yeah.
Anything that requires any kind of git commit is asking the package author to do something
that they probably weren't expecting to do. So no, it is not an onerous procedure by any means,
but it is something that takes action rather than a passive process.
Yeah. Yeah, absolutely.
One thing that I was curious about when you told me this stat during the week was
whether it has changed recently and in terms of, because obviously when we first launched
the package index, we didn't have documentation hosting. And so I have a little quiz question
for you based on the fact of we added into our package, our add a package process, a little
call to action at the end, which is to let package authors know that they can host their documentation
with us and link to the documentation that we have on how to do that. And so what I did is I
did a little filtered query to say what the percentage of packages that have adopted our
documentation system in the last year was. What would you say the percentage of packages, given
it's 10% overall, if it's packaged in the last year, what do you think we're at?
Well, I'm slightly cheating because I do track the number of new packages, you know, that number.
So I, but you know, I, that's this from memory, I'd say maybe 25%.
No, it's not quite that. It's 16, just over 16%. There are 306 packages in the last year
that have added documentation as part of it. Although that number is slightly skewed,
I think skewed lower because of the way that our rename process works. So we track when people
rename their packages within our organization. And we also track moving packages from one
organization to another on GitHub. And every time we detect a rename, that actually shows up in our
kind of stats as a removal and an addition of a package. And so there are potentially some
double counted packages there because of that process. But I don't think that would,
I don't think it's enough to be statistically significant.
Yeah, I'm certainly probably biased because just the recent, like recent weeks, it was 30%.
It's been really high, the ratio of adoption. I'm not sure why that is. Maybe it's
WWDC was a documentation thing or something, or maybe I don't know, but it's been noticeable,
the uptick in the ratio.
I think it's, I love the fact that we're so high on hosted documentation. I think that's,
it's great to see. And it's great to see so many package authors putting the time and effort into
creating documented packages.
Yeah, absolutely. Yeah.
Let's do some package recommendations, shall we?
Let's go.
Do you want to start us off?
All right, I shall do so. My first package is a package I absolutely love. It's called
Swift Testing Revolutionary by Koki Miki. And I just love this. When I first saw Swift Testing,
I was really excited about this Swift Testing package. And we've now seen the introduction
at WWDC. I just love testing. I mean, I spend a lot of time in tests, so anything in that area
really gets me excited. But I also knew looking at our test suite, just in the main project,
we have 766 tests. And obviously, that's just tests. That's not even just assertions. That's
way more. I thought, well, that's going to be a while until we actually use that extensively,
right? Because you're not going to dive in and adopt that, cross the board and make all those
changes. But then I saw this amazing tool last week. It was announced on the Swift forums.
And it's a tool, like a build tool that you can add to your package, like the new build tool thing
that you can trigger in Xcode or via the Swift Package Manager. Or you can use it as a command
line utility. And what it does, it rewrites your tests to the new Swift testing syntax.
And what that means is it adds the @Suite annotation, converts your test class to a struct,
or optionally leaves it as a class. You can configure that. It adds the @Test annotations
to each of your test functions. And it converts your XCT test assert calls to the new #expect
expression. And that is an amazing time saver. And I actually tried this out in our repository
with the 766 tests. So this is like, it's so fast. At first I thought, well, it probably didn't do
anything because it just finished in like less than half a second. And it rewrote all the tests.
Now in our test suite, it isn't working perfectly. And I'll explain why in a second. But in
a plain case where we were using an XCTest case as the base class, it converted the tests perfectly.
And they ran and they passed. And it was excellent. It was just no notes. It just worked.
The problem it has is with cases where we use a different base class. And that's where we have
snapshot tests or database tests where we inherit from our own class and then use
objects of that class in the tests. And because it converts that, and I didn't fiddle with any,
I just used the default setting and converted it to a struct. So it converted it to a struct.
And then obviously that lost all the inheritance and the tests didn't compile because they didn't
know about the DB or the app properties. It just didn't exist on the struct. But that's
really simple to fix. Perhaps even just by using that other mechanism and making it a class with
an absolute annotation or just fixing it up. I mean, that's trivial. The main thing is that
rewrites all the tests and adds the @test annotation and it converts all your asserts.
And that's the big thing because that's hundreds and hundreds of lines of changes
that we just save. It's amazing. Yeah. This is not a tool that you're going to need to run
as part of a build process or anything that's going to need to be, that the output is going
to need to be perfect for fixing up a few edge cases after the tool has run. It's a perfectly
acceptable way of running this tool. And because once you've run it, you've run it and that's it,
it's converted, it's done. So yeah, what a great tool. I hadn't seen this. So that was fascinating
hearing about that. Great. Yeah. It's really nice. And obviously you can do it file by file. So it's
like, I'm really looking forward to this because I really want to use some of the new stuff.
And obviously that's all predicated by actually moving over. And if this allows us to move us
over fast, that's super exciting because it'll open up Swift testing much, much sooner than I
would have thought it would. I must admit, I had wondered whether we would ever actually migrate
to it because of the fact that we have so many tests already. I thought maybe we would approach
it in the case of, well, if you're writing new tests, then you write them in Swift testing.
But I'm very happy to hear that that won't be necessary. Yeah. Well, I hope we'll see what
happens, but I'm really, just this short test showed me it's probably quite viable. I'm not,
yeah, I'm just really hopeful. Did you check that the result of the conversion process was not to
XCT assert true to make sure all the tests pass? No, it's actually quite remarkable because like,
yeah, I know you're joking, but like, for instance, it rewrites an XCTest assert where
equals where you have A comma B, it converts that into a hash expect A equals equals B. So it, I
mean, obviously uses Swift syntax to actually understand your test syntax and intelligently
rewrites that. So you get a lot of mileage out of this. Just if all it did was rewrite the
assert macros, the exert, you know, well, they're actually macros in the
original Objective-C, I think in Swift their functions, but these, I still, in my head,
I still think of them like as macros. They, it's just, if it just rewrote those, that would already
be a huge time-saver because that's obviously the most lines in that diff is the asserts and then
tests, you know, the individual tests. And then I think we have like maybe 15
test classes. So if, even if it didn't, those that would be trivial, right? 15 things you can,
you can manually convert it's the others that are important and those, those seem to be coming
across really well. Right. So yeah, big thumbs up. So from a brand new package like that one
to my first recommendation this week, which is to, for a package that has been around for seven
years. And it's actually a package we use in the Swift package index. And it's by far not a new
package, but it is one that received a major version upgrade this week. And it is Soto by,
well, the Soto project is the official organization name, but it's primarily written by Adam Fowler.
And I was looking through packages and there were actually a huge number of new packages this week.
But this one, the major version struck me because it is a full conversion to away from event loops
and into a Swift concurrency. So that's a great step forward for the package. But I should also,
I should also explain what the package does for those people not familiar with it.
It is a Swift language SDK for all of the Amazon web services APIs. So Amazon are constantly adding
and changing their APIs, I'm sure. And this effectively is guaranteed to always be up to
date with those because it is kind of generated from their API definitions. So this is very
unlikely to be used from like an iPhone app or anything like that, or a watch OS app. I don't,
I don't start up EC2 instances from my watch very often. But it's certainly if you're doing
any kind of server side Swift, then this will likely be already in your project. And so it's
worth a note when it does something significant, like upgrade itself completely from event loops to
Swift concurrency. And I just thought that I looked through, I was actually kind of surprised
that in the 40 something episodes of this podcast so far that we hadn't yet talked about Soton.
And I thought it was time. Oh, we haven't.
That's, that's interesting. I checked the show notes and it's not there yet. Have you
also checked its old name because there was a rename it was called something AWS
kit or something before. Oh, I don't know whether I did. I don't quite recall its original name,
but obviously it was called something AWS and that that needed to be changed when AWS introduced
their own. We have nothing in our, in our list of packages that we've talked about on the podcast
that mentions either AWS or Soto. So remarkably I don't know how we haven't talked about it because
it is, it is one of those packages that I thought would definitely be on this list, but it's, but
it's not anyway, worth a mention Soto by Adam Fowler for all of your AWS needs.
So my second pick is called Swift sessions by Alessio Rubicini. And that's also a package I
saw on the Swift forums. And Alessio also points to an introductory blog post, which explains this
package in a little more detail. And there was a really nice line in that, in that blog post,
it says, so this package is about Swift sessions. It's about session, so-called session types,
which is apparently like a computer science concept. And he writes, this is a concept.
If you've never heard about session types, that's normal. This is a concept born and explored
in recent decades. This is where I expected like weeks or months, but he says decades. So
we, I certainly missed it in recent decades. I'd never heard about session types.
It's new to me too.
Well, there you go. We, we kind of missed it in recent decades. The, the blog post does a really
nice job explaining what it is. Nonetheless, it is, it seems like a complicated topic, but in a
nutshell, what it does, it is using type checking to, to assert or implement a message exchange and
make sure that the bits line up like the parts line up. So imagine you have a side A, and this
is sort of taken from the blog post. You have a side A that sends an integer across the wire,
then expects to get a bool back, and then the communication ends. So that's side A, sending int,
getting bool, and then end. And side B has, has the, the message receiving an int,
sending a bool, and then it ends. And you can see how these line up, right? Sort of like,
you know, imagine like saw teeth in a, in a gear cogs and they sort of, you know, you have the
exact right alignment of sending, receiving int, sending and receiving bool, and then
both expect to end. And this is something that, that this package allows you to encode in the
type system. So at compile time, it'll be ensured that your messages line up and, and, you know,
the, the orders also, and not just the types line up, but the, the order, the sequence order of your
message exchange is, is asserted at compile time. And that, that seems like a really nice
guarantee to have, you know, and this package makes that possible. One note, if you use this,
you, you probably want to make heavy use of type aliases and use type inference, because
otherwise you will get hurt by all the pointy angle brackets in the type signatures. It has lots
of, lots of those going on, as you might imagine. But it's one of these cases where the type system
really opens up opportunities or possibilities to use the compiler to check things for you that,
that weren't possible before. And I find that really interesting to see these kind of new
packages pop up that, that make use of these things. That's great. So there you go. That's
Swift Sessions by Alessio Rubicini. My final package for this time is a package called Tabula
by António Pedro Marques. And this is, well, it's a package. Let me tell you the stuff. Let me start
with what the package does, and then I'll tell you why I picked it. The package is a spreadsheet
reader and writer. So you can combine it with another package called CoreXLSX by a core office
organization, which actually looks from the author information, it looks like it was written by
Max Desiatov, who is now at Apple and who, but before that had his fingers in all sorts of open
source packages. And the combination of them allow you to read and write data from spreadsheets.
So that in itself is the kind of thing, this is, you know, it's fairly niche
things I want to do. Not every application is going to want to read and write
spreadsheet files or Excel files. But it reminded me of a project that is both probably the very
worst code I ever wrote as a developer, but also in its own way, the best code that I ever wrote
as a developer. So this is way back in the 16-bit era when I was working for a software company that
wrote HR software, so job evaluation and kind of pay modeling software and that kind of thing.
And we had this job evaluation system where you would put definitions of roles into this system.
And then as a team of management consultants that we had in the company would work out within that
business, what were the important aspects of that role that made it more responsible and made it
effectively a ranking of jobs from low to high across an organization. And it was used in lots
of equal pay disputes and things like that, where you could prove that the job requires this rather
than the person doing it requiring something. So it was a great piece of software, but the
management consultants who came up with the rules on a company by company basis determined what was
important in that company. What they did was they loved using Excel spreadsheets to basically make
these rules. And what we would then have to do is take those Excel spreadsheets and convert them into
code. And that was okay. I mean, it was never terrible work, but it was fairly tedious work.
And it was also prone to mistakes. So when you're reading from an Excel spreadsheet with an
extremely complicated formula behind it, the potential for errors there is very high.
So we did lots of testing to make sure we got it right and that kind of stuff. And
when we were moving this to work on the web, because originally this was just desktop software,
we came up with this idea that actually maybe we should just open the Excel spreadsheets,
poke a value in, let Excel do the calculation and read the value out of the other side of it
every time we needed to do these calculations. So one of the first pieces of web software that
I ever wrote was a 16-bit Visual Basic COM component, which was a component object model,
I think it was, that allowed Visual Basic to interact with ASP, not ASP.NET, just ASP,
which was the original Microsoft scripting language, very primitive web scripting language
that was, but it could call out to a COM component that then executed a command line
16-bit Delphi application that could read and write the Excel spreadsheet. So it would poke
a number into the Excel spreadsheet, read the number out back out of the Excel spreadsheet and
leave a file on disk that the COM component then picked up and sent back to the database and stored
in the database. So you too can build your own Excel Rube Goldberg machine with Tabular.
- And that's how it was done back in those days.
- I mean, that was it. And what was incredible was you'd look at the processes running on this
web server and to see, you know, 14 different copies of Excel all in memory doing calculations
as the web server fielded requests. And the reason I say it was both the worst,
I think it's quite obvious why I'm saying it's the worst bit of software I ever wrote.
But the reason it was the best bit of software I ever wrote was because it worked flawlessly
for years and there was never a problem with it. It never got it wrong, it just worked perfectly.
- Exactly.
- And we can't ask for much more than that, right?
- It is sometimes extraordinary to what lengths organizations go to, especially banks. I worked
at a few banks. They'll get a system running once and then they'll go to extraordinary lengths to
keep that system running, no matter the cost. They'll just build anything around it to just
keep that one system going that managed to get this thing running in the first place and just
keep chugging away with it. It's sometimes quite remarkable.
- Exactly. But yeah, I mean, it was great. And who knows, maybe it's still running today.
I don't think it is, but who knows? So I think we should probably wrap it up there, I think.
And we will be back in three weeks with another episode with some more packages for you to have
a look at and news from the Swift Package Index. But until then, I will say see you in a few weeks.
See you next time. Bye bye.