Join Dave and Sven, the creators of the Swift Package Index open-source project, as they talk about progress on the project and discuss a new set of community package recommendations every episode.
Well, Happy New Year Sven.
Happy New Year.
Did you have a good break?
Oh, absolutely. Yeah, good break. Very relaxed. Not quite as cold as in Germany or in other
places. I've seen it's been really cold in Sweden. There's someone I'm following on
Mastodon and he's been subjected to minus 40 degrees Celsius.
Okay, that's a lot colder than it is here too.
Yeah, no, I can't stand the cold and just reading that gave me the shivers.
It's not very warm here. It's going to hit two degrees above zero on Friday this week,
but it's certainly not minus 40.
Yeah, no, that's just way too cold.
Did you have a good break as well?
Yes, I did. Thank you. Nice to see family, friends,
spend some time away from a computer a little bit, which was always nice to do, but ready
to get back to it and raring to go for 2024.
Excellent.
And maybe we should start there actually. So we should maybe talk about
some of the things that we hope to get to in 2024 with the package index.
Yeah, that sounds like a plan. We used to post
like things way back and we held that up for a couple of quarters, but then due to the way we
organise stuff that sort of didn't, it wasn't really feasible anymore. It didn't make much
sense anymore, but I think on a yearly basis, it's a nice thing to maybe look forward what
we're planning to do.
Yes. And of course, all of the things we're about to talk to today can change.
And will.
And will. Exactly. Yeah, exactly.
So, yeah, I think just in terms of I briefly thought about what's the rough plan. And I think
one of the things that I'd like to continue working on, and we started doing that a bit is
dependencies. We'll also talk a bit about that later on in the news section. We've started
tracking dependencies quite early on by just capturing the resolve dependencies,
which is the full dependency tree you get in a Swift package after package resolution. So that's
the transient dependencies included and the test target dependencies. That's literally everything
your package touches. And that's been in our database for quite a while,
but that's also not the best thing to report really for people who are inspecting a package.
You don't necessarily are interested in test dependencies or all the dependencies. You sort
of maybe want the first and direct dependencies and that sort of thing. We're starting to pick up
and we have those in the database as well now, the direct dependencies. So we can report in a bit
more detail what we've got, but there's a lot more work to do here to also prune up the test
dependencies and all that. And I think that's going to be really exciting because it's one of
the things that will give us a bigger ecosystem wide overview, just to the things you can do then
by looking at what packages are sort of central. They have lots of dependencies, packages that
depend on them. Is that dependency? Yeah, that's dependence. Now that's dependence or whatever,
dependence. That's the one. I'm kind of imagining we'd be able to maybe draw some graphs or
something around like these webs and see packages that are very cool. And we've talked in the past
about when we talked about the NPM ecosystem and the crates, the Rust ecosystem. There's some
statistic and analysis that they've done in their ecosystems that would be interesting to do in
Swift, in the Swift ecosystem. I'd really love to do that, try and replicate that, see if we see
some of the same problems there with the circular dependency graphs, which were kind of weird.
I wonder if there's something like that in Swift as well. All that sort of stuff falls
under that umbrella that I'd really love to focus on in the coming year.
And I know you've started doing a little bit of this already, not so much on the dependencies
display in the package index, but so building on some of the work that we already have in the
project for getting that list of complete, that total list of every type of dependency that we
currently have. There was a nightly process, or there still is a nightly process that runs
through GitHub actions, which goes and tries to find any dependencies that are dependent of,
sorry, that depend, no. I got it wrong. It finds any packages that are not in the package index,
which are depended on by other packages. And that process used to take about eight or nine
hours to run something like that every night. Yeah, yeah, exactly. It had increased to,
I think, nine and a half hours now, even because we've grown since we started doing this.
And yeah, this is a long one. And I know that we've, well, you just put some work into,
and we've just put into production a version of that script that now uses the dependency work
that we do in the main package index to shortcut all of those searching for dependency steps and
just read the dependencies out of our application. And that took the job from nine and a half hours
down to certainly less than half an hour, right? Yeah, it's like five minutes or something like
that, because it's effectively, we were previously crawling every package, downloading the package
manifest and doing a package resolution to get the package resolved out of it, which is something we
already do continuously in the Swift package index. So we have those things in a database,
and this was effectively just ripping that out, asking the database for everything in one go.
And that query is even faster. I mean, that query literally runs in 50, 100 milliseconds,
and it's the downloading of the data that maybe takes maybe half a second. And the remaining time
is then to validate these packages. But we have a very fast way now to find out which packages
are being referenced but aren't in the index yet. And that was really, really helpful to get that
nightly process under control. It also helped us find packages that we didn't actually find with
the other process in the past, because the way the other process worked was to download the
manifest of the main branch, of the latest version of the main branch. But what happens then is that
we have packages in the index that are old, that don't have new revisions, and they never moved
forward. And they were actually referencing, they had all the manifests that were referencing
package versions that weren't, that were only live at the time, but aren't live anymore.
This sounds probably way more complicated, and it's hard to follow. But the thing is,
we're now unearthing packages that are being referenced that we didn't actually see before.
So if you are following our Swift package updates, Mastodon account, you will see loads
of new packages appearing. And those are the packages we're now finding and adding to the
index because we've made that change. And we've also changed, we're now accepting forks as that
people are depending on, and adding them to the index, because they are packages like any other
package, and they're being used. So they should actually be in the index, so we can properly
account for all the dependencies we've got and have them in the index.
And this is the point I was going to make, and it's the influx of these additional dependency
packages has really tripped me up this week and made... So when I'm preparing for this podcast,
I go through the list of new packages to see what's interesting and what's been added.
And with all these, quite a significant number of packages, more than 100 packages being added
by this script, I made that job much more difficult than usual, because it's a whole
load of forks and packages that I just depended on that have maybe been around for years and years
and years that we might not feature on a podcast like this, but as you say, should be in the index.
And one thing that, in the context of what are we going to work on this year,
one thing that this has really highlighted for me is that we quite desperately need a feature
which shows which packages are forks of other packages. And if we have the other package in
the index, then we should also refer to that package because it's quite... In fact, I was
tripped up myself by a fork of plot, John Sundell's plot library, which is an HTML generation library,
which we use in the package index. And I was going through the list of packages and I saw plot go up
and I thought, "That's strange. We've definitely got plot." And I opened it in my browser and
looked at it and it took me a second to realize that it was my fork of plot that...
Was it your own fork? Yeah. I saw that flow come through.
And that deserves to be in the index, but it's also... It should be really obvious. Yeah,
it should be really obvious that that is a fork and not a brand new project.
Well, if any fork deserves to be in it, it's your own, right?
Well, it takes my total number of packages in the index up to two. We've got left pad,
which is the test package that we have. And now a fork of somebody else's work.
Nice. So yeah, that feature, we just need to
add a little bit of extra metadata to those fork packages. So just be careful if you see
new packages that you think should already be in the index, it may be a fork right now,
because we're not very clearly displaying that. There's a couple of things that I'm
interested in working on in terms of broad kind of themes for this year.
One is I'd love to do some more work on the package score feature. So we've talked about
this in previous episodes of the podcast and in blog posts recently as well. We have a
internal, in fact, not just internal anymore, it's now publicly available package score. And
you can see the score for any package that's in the index. But the list of kind of factors that we
use to score each package could be improved. And there's a thread open in the discussions
forum on the package index GitHub project, which we'll put a link to in the show notes.
If you have opinions on that, there's already a discussion going there.
I would like to do some more work on that this year, and kind of bulk up the significance of,
not the significance, because it won't be any more significant than it is at the moment,
but bulk up the kind of the evaluation of packages to promote better packages in the
index. So that's one thing I want to work on this year. And I think the other is something we've
been kind of discussing a little bit is over time, I don't know whether you remember, actually,
I know you will Sven, but I don't know whether anyone else remembers that when
the package index first started, it was entirely a metadata, metadata only information on the
package page. So we didn't actually display the readme file until quite a while after we launched.
And instead, it was it was purely metadata, it was it was a lot of the metadata that you see today,
even build compatibility and those results on the package page, and then a link to
the GitHub repository. And of course, over time, we and in fact, it was never our intention not
to have the readme file on there, it was just a matter of, you know, getting things done.
And obviously, we did add the readme file. And here we are today. But over time, that metadata
section has grown and grown and the compatibility matrix takes up quite a lot of space. And it
pushes that readme too far down the page. So one of the things that we've been chatting about this
morning, actually, is whether we can look at that page design again, and maybe think about trying to
pull that readme file back up the page a little bit, we're not going to get rid of the metadata,
there's definitely going to be metadata at the top, because that metadata is really important.
But it would also be good to have at least the first few lines of the readme visible
on a reasonably sized screen, which is currently not the case.
Yeah, definitely. I think the there's ways to condense that down a bit, I think,
especially going forward, we might see fewer Swift versions that we need to display,
you know, that sort of stuff. And the compatibility matrix, as you mentioned,
might be might need to shrink down a bit anyway. It's we got all these Apple platforms, maybe
there's something we can do there. It's it's, it would be nice to pull up the readme a bit more.
Yeah, agreed.
But of course, I'm sure we'll do a whole load of other things as well.
Yeah. Just to mention, there's there's no year without new Swift versions. So that's
certainly something we'll be looking at trying to add those as early as possible.
Who knows when Swift 6 will come out if it's this year. Rest assured, we'll be looking at
compatibility info there because we we know that Swift 6 will have will be potentially
have sourcing compatibility. So it's it is possible to have syntax changes there.
That's allowed. So that's allowed. That's something that is that is in the cards,
I should say. And we'll definitely make sure that we test this early so people can
can maybe use us there as the if they haven't set their own CI up or or just want to rely
on another service to have a look there and give an overview. So we're definitely trying to be
prompt with testing for Swift 6 as much as possible.
So when we've been doing a little bit of planning already with this and talking about Swift 6 and
you know, there's a Swift 6 strict compatibility mode, which is the concurrency checking that's
coming in that's potentially source breaking in terms of like there were things that with Swift 5
it would let you do and in Swift 6 it will it will strictly stop you doing some of those things.
And there's two modes you can run the Swift 6 compiler in, which is with strict compatibility
testing for concurrency issues on, which I believe is going to be the default for Swift 6.
And then Swift 6 in Swift 5 mode, which is if you want to use the Swift 6 compiler,
but you still need Swift 5 concurrency checking, then you can switch the Swift 6 compiler into
that mode, which we're not going to test for explicitly. But what we will be doing
is testing the latest version of Swift 5 at that point as well. So you'll have Swift 6
compatibility with strict mode on and then 5.9, 5.10, whatever it is at the point that
Swift 6 comes out, we'll be testing the Swift 5 mode. Have I got that right?
Yeah, exactly. I think there's, we don't expect there to be a significant difference or any
difference at all between Swift in 5 mode versus the latest 5 version at that time. So that's why
we're not really going to, we're not planning to spend effort on running those two modes in parallel.
Exactly. So, but of course we don't know, I don't know whether the public, the schedule
for that has been announced yet, but I'm not aware of it. So it's something that we will do
as and when those versions come out. Yeah. And well, I think there's no new
Apple platforms to plan for this year, right? Vision OS just came out and is...
Well, the good news is we already support it. Yeah, we already do. And I'm not aware of any
rumors about new Apple hardware or OS releases that we might be planning for. So I think on
that front, we're well covered at least. There is no new Apple hardware until the
day that Tim Cook announces it, right? It doesn't exist until he steps on stage.
That's how that works. Yeah, exactly. Yeah. I think you're right.
I think Vision Pro and obviously the Vision OS SDK, which did get announced, I think yesterday,
the release date for that, which I think is February 2nd, maybe, that we already support
in our compatibility matrix. What we will do is we'll update the version of Xcode that we use to
do that compatibility checking to 15.2, which again, I believe was released yesterday,
but really no changes that we need to do there. Yeah, exactly. And my understanding is the
515.2 beta one, I believe that we were using already had the... My understanding is how the
Vision OS SDK version that is actually shipping. So there won't be like the compatibility right now
should be up to date already, even when we change to the newest 515.2 version, that should all
remain the same, fingers crossed. That's great.
There is another bit of news we may want to briefly talk about. And that's a really interesting
post I saw coming through this week on Mastodon. And that's a blog post about
when everything becomes too much by Frost Abukadi. Did you see that, Dave? Did you have
a chance to look through that post? I did. You sent me a link to this earlier
today. And I did. I did give it a read. Yeah. So this is really fascinating. And
what a mess. What a mess. Yeah. And this is not about pointing fingers or just, you know,
laughing about what's going on in NPM. This is just general ecosystem stuff. And NPM
is pretty much, you know, it's a huge ecosystem, like orders of magnitude bigger. And it's sort of
the things they hit, they are going down paths that the Swift ecosystem hasn't even begun walking
down yet. So there's interesting stuff there. You could say they're stress testing dependency
management. Pretty much, yes. Yes, they definitely are. And some of this is inherent in how
the ecosystem works. I think it is fair to say that node packages are smaller. You know,
they bundle up smaller chunks of functionality into smaller packages. And that by nature then
have way more dependencies and all that. And that has a knock-on effect to how that works. But
that is just one aspect of it. And the other is just more exposure to more people that try
things. And what this is about is that someone created an NPM package called everything,
which, well, depends on everything. They have effectively made this package depend on every
package in the NPM registry. And just think about what that means. If someone went on to do NPM
install of that package, they would effectively DDoS themselves because it would start downloading
every package. And this just sort of spiraled out of control because it compounded with another
problem that happened in 2016. And that was the LeftPad incident.
Ah, LeftPad gets a second mention of the episode.
Yeah, second mention. There you go. So in 2016, the NPM world broke because the author of this
LeftPad package removed it at the time. And this was actually a package that was used in lots and
lots and lots of other packages. And that led to problems because then they wouldn't build anymore.
Lots of CI systems, is my understanding, started breaking and failing because they couldn't resolve
that package at build time and all that. And as a consequence of that mess at the time,
they established a rule that you can't unpublish a package once another package depends on it.
And now you can imagine what happened once everything started depending on every package
in NPM. That meant that no package could actually be unpublished anymore. So it was
effectively blocking everyone from unpublishing their packages after this. And just imagining
about how, you know, one thing that was made or one rule that was implemented to fix one thing
can then completely backfire if another thing goes horribly wrong or, you know, another,
you could call it an abuse case happens and sort of messes everything up. I don't want to be in
the shoes of people trying to manage this whole mess, but it must have been really horrific.
I think they found a solution in the end by just pulling the whole org down of the people who
published this. The people themselves actually were apologetic when they realized what they've
done. They want to do the right thing and unpublish the package, but they couldn't
themselves anymore because they'd been blocked out of unpublishing by, I guess, someone using
their package or something like that. I don't remember what the actual effect there was,
but it was just a big mess, but quite interesting as a look ahead.
I think it said in the post that the solution was to make the repositories, the everything
repositories private. And I think that's the workaround that they got to, because you can't
stop somebody taking a repository private. So they must have had to deal with that situation.
And I think that's what let them unwind this, if I read the post correctly.
But I mean, I do feel for the person who did this, because I genuinely believe that it was
done as a little bit of a lighthearted joke, or certainly its intent was a lighthearted joke.
But what a mess. Yeah, I mean, it's, can you imagine sitting there and suddenly realizing,
you know, it's like when that production incident happens and you suddenly realize you are connected
to the wrong database when you drop the tables. That's the sort of feeling that must go through
you. I mean, unless you really do it with the intention of breaking stuff. But if you didn't,
and I don't think they really did, then that is rough.
And it's the kind of thing, I mean, certainly,
Swift is a very different dependency environment than NPM. And we don't have this kind of situation,
or even really the potential yet for this kind of situation. But I think it's good for us to be
aware of what's happening in these other dependency environments. And certainly this one,
when, as I read more and more of this post, I thought, "Oh, this is, oh yeah, this is really,
it's really, really bad." I would encourage you to read the post. It's really good. Yeah.
- Yeah, we'll add a link to the show notes for people to read up on this.
Shall we do some packages or...
- Yeah, we could do some packages. Actually, I have a bonus extra little package,
which is not really a package recommendation this time. Well, but it's not in my normal theme of not
recommending a package. There may be one of those coming up later though. This was something I
spotted as I was doing a troll through the RSS feed. And I really liked the marketing approach
that this package has, which is to have no readme file whatsoever. So there is no information about
what this package does in the readme file at all. The name of the package is VGSL and the package
description says everything you really need to know. It's a very good Swift library. So that's
all you need to know. You should import this package immediately because it's a very good Swift
library. So, but no, I do have three actual recommendations as well. And the first of them
is by Daniel Lyons and it's called Plus Night Mode. And one thing that I... So I bought a new
watch this year in September or October, whenever they came out. And I don't know whether it's
something specific to the new watch. I think it may be to do with a new watch face, but when the
ambient light in the room is dark, the face and everything on it turns red, which is a really nice
way to not strain your eyes in the evening, or if you wake up at night or something like that and
try and look at your watch. And it's a... So everything goes kind of monochrome, but not
black and white, red and black. And this package by Daniel is a Swift UI view modifier, which you
can add to the bottom of any view hierarchy. So your entire navigation stack, for example,
could have this view modifier on it called Observing Night Mode. And you can pass a switch,
a Boolean in there to turn night mode on or off, which is something you'd have to track yourself.
So this would be an app specific feature. And it basically turns every view in that view hierarchy
into a red and black monochrome view, no matter what the original appearance of it was. And I
think that's lovely. I think, first of all, there's a couple of nice things. I think it's a
great feature. But secondly, I love the kind of cascading nature of Swift UI view modifiers in
that if you add them to the top level, then you can just do something like this, which would be
very difficult to manage in UI kits. I mean, you could still do it in UI kit, but it would not be
as easy as adding one view modifier in a UI kit. So I like that. And I thought it was worth mentioning.
- Yeah, I love that about Swift UI as well. I think the redacted stuff works the same way,
right? Where you can stub in, I don't know how best to describe it, redacted UI elements,
like text is not displayed as text, but as placeholders, like rectangles and stuff. I think
that's really nice and really powerful that you can just tack this on and everything just changes
into a different mode. I think that's really fascinating and super useful.
- I hope apps take advantage of this because I'd love to see this feature in apps that I use.
- Yeah, I have the same. I love this about the watch where it turns red. It's really nice.
Right. My first pick is called package-benchmark by Joachim Masila.
And this is a great package when you're looking to benchmark your code.
What it effectively does, it gives you a function or instrumentation to run a closure that gets
benchmarked. And it supports many, many metrics like CPU time, both like actual CPU time or
work clock time, throughput, memory allocations, threads, like how many threads you're using,
disk stats, all sorts of things. There's an amazing range of metrics you can track.
And I've been following this package for quite a long time, but I've always delayed talking about
it because I wanted to really give it a try. And it was clear from looking at the readme that this
is super comprehensive approach to benchmarking, but yet you'd also need to invest a little time
to understand what it reports and why that is important. That sort of held me back until I
actually sat down and tried using it. And it's not because the package itself is making this
complicated. It's because the topic is actually complicated and more complicated than you might
think. And that these details aren't ones that you should just gloss over and just take an average or
minimum when you benchmark stuff, because that's the thing you sort of reach for. I certainly,
whenever I benchmarked something in the past, you run it a few times, you look at it, well,
there's not too much jitter. I'll take the best one. You never really know what you're doing.
Typically when you measure, you do it a few times, you maybe calculate an error rate based on the
standard deviation, that sort of stuff. The thing is, when you do this, there are certain assumptions
that go in when that is actually valid and when it isn't. And mostly, almost always when you're
measuring benchmarks, these assumptions are wrong because of the way the distributions are shaped,
you can't actually calculate a standard deviation. And it doesn't make sense. And the framework
really leans into that. So what it does when you run a benchmark, it gives you so-called percentiles.
So it gives you a range of percentiles and measurements for those. So what that means,
a percentile is, let's say, percentile 50 means you're halfway through your distribution.
Everything is at least as fast as this and not slower. And the higher percentiles, for instance,
P100, 100th percentile means this is the slowest result you got. And the first one, which is called
P0, is the fastest result that you've got. So effectively, what this does, it gives you
certain points along the curve of your distribution. And it really drives home
the point that you shouldn't be thinking of your distribution as like a normal shape,
you know, like normal distribution. But it's really, really different. Just forget that
this is a normal distribution, because it won't be. And that's what I really liked. It's not
something you just drop in and then get numbers out and you walk away with the numbers, because
it gives you many numbers. And you really need to understand what you're trying to do.
And especially what you're trying to measure, what your requirements are for your system,
how fast it is actually supposed to be, and how fast is the slowest result supposed to be. And,
you know, it's making you think about the thing you're actually trying to achieve.
And I found that really interesting. And I had a couple of questions to Joachim
ahead of time, because I wasn't sure what this actually means and how to interpret the results.
And he was super helpful, gave me some pointers. And we'll add these to the show notes. There's an
interesting talk about by Jill Tenney, I think is how you pronounce his name,
about why normal distributions, these are normal distributions, why you should be looking at
percentiles and what you should be looking for when you do run these kinds of tests. And yeah,
I think this is really great. This is one of these packages where you start looking at the package
because you think it's useful. And then you end up going down a rabbit hole and learning really
a lot about a domain that you didn't know a lot about. And that really gives me, just looking at
the package and the documentation of it and the breadth of it, you know, like these, all these
different metrics and the care that has been taken to give you controls and knobs and, you know,
allowing you to set these thresholds, make it configurable, gives you really high confidence
that this is a really worthwhile thing to invest time in and use. And I know I've seen this pop up
in a few places, this package. It's been being used in quite a few projects to measure benchmarks.
So I think it's really fascinating. And give this a look if you want to benchmark stuff,
I think this is the perfect place to start. - That's great. Are we going to use it with,
because we have query performance tests in our code base, right?
- Yes, we do. I might try that. There's some, another place where I want to use it first,
but it's certainly something I'd be interested in doing more because right now we're doing really
the bare minimum of just, effectively we're not as much benchmarking, we're sort of snapshotting,
we're baselining, you know, we're trying to establish how fast is this if I run it once,
and then we record that time and we effectively see if we get slower. So we're not really ensuring
that we're hitting a certain speed, we're tracking how we're getting slower over time.
The way I like to look at it is that it's more of an indicator than an actual test. It is,
you should be aware that things have got slower. - Yeah. I think if I was asked what's this for,
I'd say this is to see if we suddenly have a query that runs two times slower, because that we would
see and that would be important. And this has actually happened. And that's what this is for
really to not accidentally, because SQL queries are really easy to just tweak a little bit and
then have huge performance regressions that you don't notice, especially you won't notice it if
your dataset is small. And what we're actually doing is we run it against our staging database,
which has a comparable dataset to production. So we would see if a query is really bad all
of a sudden, we would see that by looking at the, we're not looking at the runtime,
we're looking at the cost, which also isn't perfect because the cost is an estimate,
it can be wrong, but it is a, in my testing, it has been a fairly decent indicator of what's
actually going on. So this turned into a much longer packaged recommendation than usual.
But we haven't finished yet. Cause I was going to say the only, the only slight downside to that is,
is that because that CI step is not a mandatory CI step. What we quite often do is something will
slip by just a few milliseconds and it will start the test failing. And it takes us a few weeks to
maybe get around to updating those benchmark scores. And so if anything doubled the query
performance in those few weeks, we'd never know. Well, actually we would, we'd never know until
we came to update the benchmark scores. And at that point we'd realize, and there we go. But,
but, but yeah, but I mean, who's perfect, right? Yeah. I mean, that's the thing. This is hard to,
to automate because A, we want to, don't want to be running this all the time.
Like auto updating, right? You need to review these new numbers, right? You need to put them
in somewhere. We need to baseline somehow. And because Swift package manager doesn't have the
same affordance to do these, what's it called in Xcode where you can, you can baseline tests,
right? You can have performance tests in an Xcode and baseline them automatically because we can't
use that due to the way we're running tests. We sort of have to do that bit manually. Otherwise
that would be really great if we could do that and then just review the changes and approve them.
That would be ideal. Right now we can't, I mean, the best thing we could probably do is,
is write a, like the Pointfrico guys have done this auto snapshotting thing we could, but you
know, I've already spent way too much time instrumenting this. I think I'm not gonna,
I'm not gonna take it even further. This is, I do this once a week and that's, that's fine. So
it's not too much work at the moment. So my next package recommendation is by
Navin Chauhan. And this is something that I never expected to see in a Swift package. And I was,
I was kind of delighted when I saw it. It's called Swift Gopher. And it is a, a less than
one month old implementation of the Gophers, both server and client written in Swift. Now
I'm pretty confident that there are people listening who will not remember Gopher.
Do you remember it Sven? I do remember it. I saw it early on,
but I actually never used it. I know it's one of these, not ethernet, internet protocols, you know,
like FTP, HTTP, that sort of stuff. But I don't, what did Gopher actually do? It was, I don't
remember. I also, so I did, I think I did use it like once or twice. I remember on the old
Unix machines at university. I think there was still a couple of Gopher. I think Gopher was
mainly over by the time I even got access to the internet. So when I was at university, which is a
very long time ago, the internet was really only just becoming something that you could get access
to even in academical institutions. Certainly there was no home internet access at the time.
And Gopher was kind of dying at that point. So it was really early. But what it was,
was it was a precursor to the web and you could Gopher into a server. And it was like a step
up from Telnet. So Telnet, you basically got a remote console on the server. That's what Telnet
basically did. And Gopher was a way that you could fetch information and then post information back
to it in a text-based interface in the very early days of the web. But it's not dead apparently,
because Navan has written both the client and the server and you can run a Swift Gopher server
and you can connect to it with the Swift Gopher client and you can fetch information from your
Gopher server. It's a bit like these very early BBSs. I have no idea how that actually used to
work. So I do remember those. So that was really one of my first experiences with the internet,
which was the Telnet BBS systems. The only one that I remember by name was a Telnet BBS called
Skynet, which is a very original name for a BBS. And you would connect effectively using,
you'd effectively get a kind of shell into the server and it would be this BBS system that you
could leave messages for people, you could read messages, you could send messages, you could post
on public forums, I think, but they were very, you know, it was a very different to anything you'd
see today, but it was all through a kind of text, a terminal style window. And I remember the speeds
that we were able to have at the time, you could see the terminal screens drawing character by
character. So it was, it was early, early days of the internet. Well, Gopher, I didn't have that in
the cards as a package. I was pretty confident that that would be all mine. Yeah. Well, this
is one of my traditional recommendations, which is not a recommendation. Oh, is that what it is?
Although I do think we should, we should maybe consider supporting Gopher for the package index.
So we don't neglect those people who still want to use a Gopher client to fetch package information.
You can fetch your package of the day. Exactly. My second pick is called WebSocket Actor System,
and it's by Stuart A. Malone. And that's, I loved seeing this. So I've been following
Apple's distributed actors package with interest for quite a while now. But in the past, I've,
I've really struggled. I've tried a few times, but I've struggled with creating a proper example
that would work across nodes. So the repository comes with, with an example, tic-tac-toe the Apple,
Apple's distributed actors package, and I've played around with it, but I've never really
gotten it to work properly in a, in a local setup. And, and Stuart has really done all the hard work
here and created a package and then also shipped an example and explained how to use it. We can
set up a distributed actor system based on WebSockets here to communicate between nodes,
where nodes really is just another distributed actor instantiation. So it's really fascinating
because from a dev point of view, you're effectively just calling into an actor and
you don't even realize or know whether it's local or in another process or on another machine
across the network. And that's the beauty of this, of these distributed actors. It abstracts away
all of this and, and all of this message passing and, and encoding decoding is, is handled by the
library which is really nice. And Stuart has chosen WebSockets here as a transport and I think
JSON under the hood as a, as a exchange format, but you actually see none of that. So you really
just have actors and you, you know, you, you connect and then call functions with try await,
you know, like normal async functions on the actor that you've just resolved across the network
potentially. Now I haven't tested, I've actually used this in a little example to see how it works,
but I haven't tested what this means in terms of efficiency, resilience, error handling,
that sort of stuff. It does claim to automatically reconnect after failure. So it has some
affordance for this. And the other thing that's really nice about this, as opposed to a normal,
traditional HTTP rest API, for instance, is that you can call methods, actor methods on
the service you connect to, but the other service, you know, can also do this. So it's actually
really nice if you want to have a server that calls back to you as a client, you know, imagine
it's like a push mechanism, right? Whereas normally you talk to an API, you'd have to
pull the API to understand if something's there for you, you know, if something has something to
respond. But in this case, these two nodes are really peers. So if there's something happening
on the server, it can actually call into the client and say, look, this just happened, you
know, handle it. So that's really nice. And that might open up use cases that are quite powerful.
So, yeah, I really wanted to highlight it because I'm really interested in the distributed actors,
and I haven't seen a lot of packages actually using it, or even like proper applications using
it. And maybe this package WebSocket actor system can be a building block for someone who's doing
something with it. I have sort of a couple of ideas how we might actually be using it
in the future. That's why I'm super interested in this whole domain. And I've really loved seeing
this package come around. Are you ready to share those ideas? Well, I mean, it's no secret we're
running a build system, right? And right now our build system is... I figured that would be it, yeah.
Yeah, I mean, this because we have components that talk to each other.
And we're currently using GitLab as our queuing system. And if we ever have need to change that,
I'm sort of toying with the idea that rather than building some sort of HTTP-based system,
maybe this would be a good use for actors where the actors are the builders and they report back,
that sort of thing. I have to talk to people who know more about these kinds of systems if
that's actually a good idea, but it strikes me as something that might work really well for us here.
I don't know. I think these things make me overly nervous because they feel... There's no judgment
here on the actual reliability of this because I'm completely uninformed on the actual reliability.
But my perceived reliability is... I'm suspicious. Yeah, I mean, this is why I said I haven't tested
this in terms of efficiency, resilience, and reliability. I think that's really
a key thing to understand. But it's also true that distributed actors is based on tried
technology. So there are actor systems out there already. And Conrad, his last name escapes me
right now, at Apple, who's working on these systems, he has lots of experience with this.
And this is Apple's adaptation of actors. And this is not entirely novel, is what I'm trying to say.
Like, this is not completely uncharted territory. This is Swift's take on a system that exists. I
think Erlang has this in a certain sense. There is certainly a Swift spin to it. But I think we're
not, you know, like complete greenfield here. I think there's lots of reasons to approach this
with interest and see how it actually works and understand, especially if this is a good fit,
right? Because I think it's also important to stress that distributed actors aren't intended
to replace every distributed system, right? There are reasons why HTTP is really good. Because I
think if one reason I can think of where you probably wouldn't want to replace it, if you
need caching, that sort of thing like this or proxying, you know, like HTTP as a protocol
allows you to do lots of things where your services don't even know that they're happening,
because it's just a plain standard protocol. You couldn't, I think, insert anything like that into
a distributed actor system and expect it to still work, right? They really rely on talking to the
other end of the system that is as the same part of the system. There's no expectation of this
being proxied or load balanced or anything like that, is my understanding. Again, I might need to
talk to people who understand this way better than I do, to really get a feel for this.
We just continue with our reputation as the best research podcast in the world.
And with that, I think we should probably wrap it up for this episode. We will be back in a
couple of weeks with some more news and some more package recommendations. But until then,
we will say goodbye for now. Yep. Goodbye. See you in two weeks. All right. Bye-bye.