- 🛡️ Security releases
- Proposal accepted: cmd/fix: automate migrations for simple deprecations
- ⛓️💥 Go Supply Chain Attack: Malicious Package Exploits Go Module Proxy Caching for Persistence
- 👊🏾Ghettoize Hook👊🏾
- Interview with Thorsten Ball
Creators & Guests
What is Cup o' Go?
Stay up to date with the Go community in about 15 minutes per week
Show is supported by you, our listener. Stick around till after the news to hear more about that. This is Cup of Go for February important happenings in the Go community in about fifteen minutes per week, sometimes more, sometimes a lot more. I'm Jonathan Hall.
Shay Nehmad:And I'm really hoping that today we can keep it short enough, because other than news, we have a pretty cool interview coming up. How are you, Jonathan?
Jonathan Hall:I'm good. How are you, Shay?
Shay Nehmad:I am excited. I have started the process of selling all my earthly possessions in anticipation of the move. So I'm trying to sell, like, my car and my bike and my furniture. I already sold my guitar, sold my standing desk, which you are, familiar with, selling my gun. If if any of these, interesting, products sound enticing to you and you live in Herzliya, please buy them for me, like, now, like, soon.
Jonathan Hall:I'm doing the opposite. I'm buying all sorts of stuff. We've been furniture shopping, ordered a bed, ordered nightstands and sofas and all sorts of things.
Shay Nehmad:Yeah. Yeah. Yin and yang. Alright. Have you gotten a bed or just a box with a bed shaped hole in it?
Jonathan Hall:We've ordered a bed frame that will take about thirteen weeks to be built, custom built and and made for us.
Shay Nehmad:Ah, damn.
Jonathan Hall:The mattress should be here on Monday. So I'll be sleeping on a mattress on the floor for a while.
Shay Nehmad:Well, you're sort of, like, starting like a peasant and then moving to a custom made bed frame. That's, literally things that only kings had.
Jonathan Hall:Very cool. Is it?
Shay Nehmad:I've always slept on IKEA furniture in my life, I think.
Jonathan Hall:Well, we have plenty of that coming too. The bed is not actually from IKEA this time. Anyway, we do have some news to talk about.
Shay Nehmad:Yes.
Jonathan Hall:We're not just talking about buying and selling things. We'll also be talking about some security releases, lots of security related things actually in this episode. We'll be talking about the live episode that's coming up next week, and we're doing an interview with Thorsten Ball. So stick around. He, is a lover of Go who's been forced, dragging and screaming to write Rust for the last, several months.
Jonathan Hall:Although, I think he kind of enjoys that too. Anyway, stick around the interview to hear his hot takes or not so hot takes about Go and, AI and Rust and other things like that.
Shay Nehmad:Cool. Let me take the reins. Go 123.6, 1 20 2 12, and 124 r c three, well, that's hard to say, have been released. They all basically include the same security fixes. Two security fixes, both of which I think are interesting enough to talk about.
Shay Nehmad:So, yeah, you should upgrade, but let's discuss the content as well because it's, pretty nice. If you're on 01/22 or 01/23, just upgrade the miner, you should do it anyway. And if you're one of the few kind souls that's actually trying out the release candidate and you're, like, CI pipelines or whatever, make sure to, you know, upgrade your your CI pipelines to the third release candidate. The third release candidate is the last one. Right?
Shay Nehmad:That means that one twenty four, like, should be released pretty soon.
Jonathan Hall:I don't know if that's official. I think it frequently is, but I don't know if there's, like, a a rule that after three, you have to do the real release.
Shay Nehmad:Of course, there is. Third third time's the charm. Okay. Anyway, so two security fixes. The first one is arbitrary code execution, during build.
Shay Nehmad:So, you know, how we have to compile our software in order to ship it to someone? So, normally, what people find is that compiling on platform on multiple platforms can sometimes be kinda hard, but in Go, usually it's pretty easy, too complicated. Yeah. So I think it's very rare for me to see issues that are related to build and that are platform specific. I've I've certainly never experienced anything like that just, you know, sometimes I mistakenly build something for Linux and then ship it on Mac or something like that.
Shay Nehmad:But in in this case, when you use, when you build on Apple, there's, usage of things like executable path, loader path, like any special values, for Sego, for like the LD flags for Sego, you can get just arbitrary code execution. So if you have, like, bad flags or someone gives you a package with bad flags, they can just run code on your machine. This is only for, Go one twenty four r c two. So this is not, like, impacted the previous versions. It's a new thing.
Shay Nehmad:Funnily enough, the person who discovered this particular issue is a Juho Forzen from Mattermost, which we had on the show. Yeah. Here's a snippet of that interview.
Juho:Has to be with you. So I work at a company called Mattermost. We are a Go software company, and we do a lot of different kinds of things with with the Go. I work as a security engineer there. And, as part of the role at at MetaMost, I've worked with the Go internals for for a few years now.
Shay Nehmad:And other than, Juho's, vulnerability, there's a timing side channel attack in an elliptic cryptography package. However, it's, like, only on specific architecture, like PPC. And a very specific PPC. Yeah. 64 little endian.
Jonathan Hall:Which, like, three people in the world have. I think fewer people use that than use Go on Windows.
Shay Nehmad:I don't I I don't know, like, what maybe it's, like, embedded devices, but then why it's 64 bits? I don't know. I don't know. Anyway, the interesting part about this one is just the nature of the attack. I really really love side channel attacks.
Shay Nehmad:If you know if you don't know what they are, it's basically there's like a cryptography mechanism or like any whatever mechanism that doesn't give you any direct signal back about whether you're successfully, like, hacking it or not, but you can use, different artifacts of, like, timing, how long it takes or temperature, how hot the device gets or whatever to understand if you're doing a good job. Good way to explain it, I think, is when you're trying to pick a lock, you don't actually see, like, the pins inside the the lock, right? You're trying to put your lockpick in and you don't get immediate until you finish all the pins, you don't get feedback whether you succeeded or not. Right? But you can listen to hear if, like, the thing got caught on the on this on the metal there.
Shay Nehmad:Right? Mhmm. So it's sort of the lock is side channeling information whether you successfully picked it or not. So in this case, it's a timing side channel where you can, understand what is the private key based on how long it takes to generate different parts of it or something like that, but they don't believe it it's it actually allows you to recover the private key, but they solved it anyway. Alright.
Jonathan Hall:So those six people.
Shay Nehmad:Yeah. Actually, you know, the the people who use these architectures probably, like, hold the nuclear codes or something.
Jonathan Hall:So so I I think this architecture was was a transition transitory? What's the word? Transitory architecture specifically designed to make it easier to port Linux software from x 86 to PowerPC, because normal PowerPC is big endian, but I guess x 86 is little endian. So by doing this PowerPC little endian thing, it made it easier to port software for Linux to the new this new architecture. And I I don't know how that's really useful because I would imagine you wanna continue the the transition and go to Big Indian later on.
Jonathan Hall:But, yeah, it's a very, very niche piece of hardware for a very specific use case. I would be surprised if anybody's using this for anything serious anymore. I don't I don't I don't think this this hardware has been manufactured for ten or fifteen years or more.
Shay Nehmad:I still think, at least pining before, like, all the machines to make them still work is is, nice.
Jonathan Hall:Oh, sure. I'm I'm not at all, disparaging the people who who solve this bug.
Shay Nehmad:Yeah. For sure. For sure. I just I don't remember what's the guy's name. I think retro Ahoy or something like that.
Shay Nehmad:But just today, at lunch, I watched, like, a fifteen minute video of how many Amigas did the Commodore company sell, like, overall in the world, and he's just, like, digging into old shareholder reports he bought off eBay just to try to I don't know. People have nostalgia for old hardware. Like, when I see an old CRT monitor, the big ones, you know, the Mhmm. Yellow plastic, the, like, yellow grayish plastic thing with a curved glass on it, I get I get, like, it's cool. Definitely happy that, people work on these hardwares.
Shay Nehmad:Not sure it's super critical for Go one twenty four r c three, but sure, guys. At least it's it gave us a chance to talk about the side channel attacks, which are always Here we go.
Jonathan Hall:Well, kind of like the way PPC 64 LE was a transitory architecture, I wanna talk about something else that's related to transitions, and that's a new proposal that's been accepted. It's a way to automate migrations for simple deprecations in code. So I'll use an example that's that comes from, I think, Go 1.16, because I think it's easy to understand. Prior to Go 1.16, we had this IO utils package that had things like read all and had a few different things in it.
Shay Nehmad:Yeah. Whenever I use that, Versus Code, like, strikes it out and says, hey, this is deprecated.
Jonathan Hall:Use something. It was deprecated in Go 1.16, and and the things were moved to various other places, mostly into into the IO package proper, a few things to other places. Basically, the Go team decided, you know, having a a package just called Util is kinda silly. You know, the the same thing that you learned back in, you know, twenty years ago, that having a package called Util or helpers or stuff is kinda silly. The Go team finally caught on to that.
Shay Nehmad:I don't wanna date your example. Twenty years ago, I was, like, starting to learn English because I was at, like, I don't know, third grade. So just remember and I assume, like, half our listeners are even younger than me. So probably so.
Jonathan Hall:Anyway, the point is these transitions from IOU till dot readall, for example, to IO dot readall is a pretty straightforward transition. Wouldn't it be nice if there was a tool that would do that sort of thing for you automatically? Well, now there will be because this proposal has been accepted. So basically what you can do, and this works not just in the centered library, you can do this in your own code. You can put a special comment, basically a go directive in your source code when something is deprecated to tell this tool how to rewrite it to the new version.
Jonathan Hall:And then if they if you run, I think it's called go fix is the command. It will it will just go through your your project and do those changes for you automatically.
Shay Nehmad:Is that a good feature in your opinion?
Jonathan Hall:I I think it's a good idea. Yeah.
Shay Nehmad:For a standard library stuff, sure. But for any library stuff?
Jonathan Hall:I mean, if you trust the library to be using it, you probably trust it to tell you how to upgrade itself too.
Shay Nehmad:So I'm wondering, like, if the case is that it's easier to theoretically, it's easier to deprecate stuff and, like, build unstable interfaces, you're sort of disincentivizing library developers to, like, think about their API longer and develop stable APIs. Can they because they can just be, like, oh, whatever. Run a new version. You have to run go fix.
Jonathan Hall:I I can see that argument. But there's enough times. I mean, I I could say this is the is the author of an open source library. There are enough times where, legitimately, deprecation is called for. But I I think I I I would love the opportunity to to be able to tell people here's the the quick and easy way to update.
Jonathan Hall:You know, I mean, a really a really simple example, even simpler than what I did is one where just one function is replaced by another one. And we already have the comment deprecated use blah instead. If you put a go fix in front of that, it'll automatically just rename to use the new function now.
Shay Nehmad:How does the directive work? The I utill example is a good one because you have to change the imports as well. You can't just call a new function.
Jonathan Hall:So yeah, they talk about that in the in the discussion on the proposal. It may add and remove imports as needed, if that's, if that's appropriate, depending on on the change.
Shay Nehmad:Cool. And writing the go fix, like, directives, is it like a super cryptic language or does it look like sane?
Jonathan Hall:Good question, Shai. So basically, what you do to make this work is you add a go fix inline, directive before the function that that has been deprecated or or in the go doc, the function has been deprecated, and it will essentially replace calls to that function with whatever is inside that function. So if you're if you're deprecated function is called read all and it just does say return IO dot read all, then when you put go fix inline in the go doc for IO util dot read all, the go fix tool will just do the inline for you basically. So, it will then call the the new function for you. So you you wouldn't wanna do this on, like, a big hairy function that that does something crazy.
Jonathan Hall:If it's a 50 line function, you wouldn't want to put go fix inline because then you're going to copy and paste those 50 lines all over someone's code base. Yeah. But for short things where something, you know, the order of function or order of function arguments changes or the function name changes, things like that, it's a really easy way to do that for you.
Shay Nehmad:Cool. It's another, command, like, to remember, but it doesn't seem like you have to run it in every CI or anything like that.
Jonathan Hall:Oh, no. I would run it It's one
Shay Nehmad:of these things that you do manually.
Jonathan Hall:Yeah. I would do it manually, probably once every few months or once every time I upgrade a major library or a version of Go, for example.
Shay Nehmad:Cool cool cool. New new thing. New good thing in the in the standard library. In the standard, tools, I mean. The standard tool chain.
Shay Nehmad:Yeah. Yeah. Like Jonathan said, we have a lot of cool security things for you. One of them is a blog post from a company called socket.dev socket security socket.dev is their, domain. They do a supply chain, security, which is, like, there's a lot of companies that do that.
Shay Nehmad:Funnily enough, like, when you look at their page, one of the things you find is socket versus sneaks, socket versus dependabot versus s m grep, indoor labs, blah blah blah. All these companies, what they do is they try to prevent vulnerabilities from entering your code through your dependencies. So you don't wanna import a library with security bugs. Now, libraries that you import or dependencies that you have can have we can have two types of security issues. Right?
Shay Nehmad:Intentional or unintentional. The case we they they published a blog post about, I think it's super interesting, it's also very simple to understand, is a malicious package that's just based on, like, type squatting. Right? Typosquatting. Typo squatting, basically, you have a package in a name that looks like the thing you'd like to import, but it's not actually it.
Shay Nehmad:In this one, I think the interesting thing they did is they the typo is not in the package name, it's in the organization on GitHub. So very very hard to see because if you look at code examples, like, at at your code, it's not like it's the package name is different from official documentation. So your code snippets look the same, but the, organization on GitHub. So just the import line is different. In this one, it's the BoltDB database.
Shay Nehmad:Have you used the Bolt, DB? Do you know, like, what database it is?
Jonathan Hall:I know what it is. I've not used it.
Shay Nehmad:What is it?
Jonathan Hall:It's a I believe it's Go native, key value store.
Shay Nehmad:And it's, like, super widely used. It's not niche at all. It's used by, like, a ton of, organizations, Shopify, whatever. And the malicious package is bolt d b dash go slash bolt and it has this one has malware in it. What happened is they created it a lot of years ago, it looks exactly like the same thing, so if you install it and by the way, I think it should even work.
Jonathan Hall:They they effectively forked the the real code and added their their thing with a near identical organization name, right?
Shay Nehmad:Yes. And it was uploaded on November 2021, which is super scary to, like, read. They they uploaded the package and GitHub tag is pointing to a clean legitimate version, something that looks like a fork. It's not a fork, it's a mirror, so you can't see they were like linked to the original, right? But But it looks like a clean legitimate version on GitHub, but on the Go when you run Go mod install, it's not because in Go mod, it's cached as the previous tag.
Shay Nehmad:So they changed the tag, but only on GitHub and they didn't notify Go mod and because go mod has no other way to know, go mod points to the old tag which is malicious. Again, it it gives the attackers access to run arbitrary commands on your machine. Not great, especially like when you're building DBs and things like that. Go module, you might ask the question, wait, why didn't the Go mods update when I changed the tag? Right?
Shay Nehmad:But it makes a lot of sense that the Go the module is immutable because otherwise, like, you could build the same package and get two different things, like, you really wouldn't want people to be able to upload any version. That's where you have the lock files, etcetera. But, yeah, it's, there's a function that attempts to continuously go to a hidden domain and execute some shell commands without any validation, with weird ass names like underscore r and a pill init, like, just very clearly shitty names.
Jonathan Hall:I'm taking that personally. Those are names I use every day in my code. Yeah. By by the way, the repository the entire organization on GitHub is now gone. I guess GitHub decided that they didn't like hosting.
Shay Nehmad:Yeah. So Kirill, I don't wanna say the name wrong, but Kirill Boichenko published this, like, two days ago, so I assume they they're jumping on it at this point. Yeah. The code itself, the malicious code itself is also, interestingly obfuscated, like, the IP addresses and the ports are saved in constants called max memory size and actually saved as numbers and max index and max port and things like that. But they just transform it and like, you know, replace it with dots so it so it actually becomes the IP address and the port of the CNC server.
Shay Nehmad:Very, like, if you go read the, blog post on saga.dev, it's linked to the show notes, you can look at, like, all the ICOs. I assume, if you use the product like the saga.dev it will report it for you, but since they published it publicly, I assume dependable and others will do the same. At least I hope they they will. It's a cool attack, like, I don't wanna praise it because it's a bad thing. I think it it abuses Go mod interestingly and I'm wondering how many more cases of this actually exist or maybe you can even target specific organizations, right, that you know that use a specific dependency.
Shay Nehmad:So if you knew Uber is using a specific Go dependency like Zap, right, their log library, maybe you can create Zap with double p and hope that one of them installs it one day. And all you need is once if you wanna infiltrate the specific organization. So a really cool, blog post, and security work done by, Socket Security here. I really liked it. Well done, Kirill.
Shay Nehmad:Very cool stuff.
Jonathan Hall:Try it. I get so bored with my logs sometimes. Do you know anything I can do to improve that?
Shay Nehmad:Well, someone posted on our Slack a pretty interesting solution. I guess if your logs were more sassy or ghetto, you'd be happier. Right?
Jonathan Hall:Yeah. It would help.
Shay Nehmad:So someone wrote a log hook. This is, this is, funny. I don't know if I'd actually ever use it.
Jonathan Hall:I know I would. As it is. Yeah. Because it uses it requires loggers, which I no no shade against loggers. I've I've used Loggers.
Jonathan Hall:I have some patches in Loggers. But now that s log is in the center library, I never intend to use Loggers again.
Shay Nehmad:So other than the fact that, uses Loggers, what is it? It's a hook to for your logs that goes to a local Ollama server, so basically a local LLM model, and asks it to, ghettoize your logs, which is absolutely hilarious. They have, like, ghettoize dot new ghettoize, function, which asks, like, basically, it's all a wrap around take the log line and, around this prompt. How would you make this, log entry more like a ghetto sentence? The output should be already valid to be the log entry.
Shay Nehmad:Don't give me options and choose one. Feel free to use emojis. So pretty good. And it uses Ollama, which I've been meaning to have you, like, played around with Ollama? No.
Shay Nehmad:I've deferred it again and again. I was like, I have to find something to do with a local LLM, like, because I use LLMs quite a lot, but I I have them everywhere. Like, I have them on Versus Code and I have them on GitHub and I have them on, like, ChatGPT, which I pay for, etcetera etcetera. And I was, like, ah, I need to run local models. Like, what will happen when I'll, like, be on a long flight?
Shay Nehmad:Right? I need I want the model on my machine. So this, project gave me the excuse to finally install Ollama and start playing with it. Manuel Martos, thank you very much for the project, which is very interesting. I really like his, his GitHub page.
Shay Nehmad:It's, like, the most setup thing I've ever seen. It has, like, GitHub stats, GitHub trophies, languages I'm currently learning, contact information, pretty busy cool stuff. And, this project is just kind of funny. If you do use Logres, I don't know if anyone is. If you do, can you try it and show us the results, please?
Jonathan Hall:I was a little surprised that there weren't some examples on the on the readme.
Shay Nehmad:Maybe we should, like, fork it and and add support for, standard library as well.
Jonathan Hall:For s log?
Shay Nehmad:S log has hooks. Can I hook into s log?
Jonathan Hall:Why not? It's called the handlers, but, yeah, it's the same concept.
Shay Nehmad:I'll I'll definitely do it at some point. Speaking of things I'll do at some point, let's talk about, the episode next week. Let's move to our ad break and talk about next week's episode, and you should stick around because we're not the only doing ads, we have a really interesting interview with Thorsten Hall.
Jonathan Hall:So we have a live episode coming up in one week. One
Shay Nehmad:week from now, it's our hundredth episode.
Jonathan Hall:Yes. We will do a live
Shay Nehmad:stream on all the platforms. We're testing it, like, right after we're finishing this recording. But just mark it in your calendars. It's gonna work somehow. There's gonna be some link or some service you could click on, join our episode.
Jonathan Hall:We will announce that link on our Slack channel. And our Patreon group. And our Patreon group. So if you're not a member of either of those, you have a couple days to go sign up, and then, you'll get notified.
Shay Nehmad:We'll probably spam people on social media as well, like LinkedIn and Twitter, but just if we're, really extra. We're planning a live episode February 13 at eighteen hundred UTC. So it should be, like, early enough for American, listeners just, like, start your day with, joining your podcast and for European listeners, it's like evening, you're off work, should be a good time for everybody.
Jonathan Hall:And if you miss it, of course, we'll also, post a recording to our regular, podcast feed and probably on, I mean, on YouTube and maybe a few other places.
Shay Nehmad:And just make sure to put it in your calendar if you do wanna join. We would really appreciate it.
Jonathan Hall:And bring in bring a news item or or blog post related to Go that you wanna share with us. Or a friend.
Shay Nehmad:Or a friend.
Shay Nehmad:Who wanted to introduce to your show and you're like, have you listened to that podcast I've sent you? And they're like, nah, man. I don't know. I don't have the time. Like, join the live episode.
Shay Nehmad:That'll be speaking about our communities, so we have the Slack group, Slack channel. You can join the Slack channel on the go for Slack. It's a cup dash o dash go kebab case, or you just talk about the show and some random stuff. Or if you wanna extra support the show, you can join as a Patreon. It's $8 a month and it helps us support the show financially.
Shay Nehmad:This is a hobby. We do it for fun and to learn, but it does cost a bit of money. We need to buy boxes of, microphones and then buy the actual microphone a week later. We need to pay for editing to, you know, bleep out all the whatever. I actually use the curse words this time, Filippo.
Shay Nehmad:Sorry. So you actually have to bleep them out instead of me just saying bleep these words out, etcetera, etcetera. So if you wanna support the show, come join as a Patreon member, or if you don't feel comfortable giving, money, that's also fair. Maybe rate the show on your podcasting app of choice, Spotify or Apple Podcasts or whatever, or share the show with a coworker or a friend or a co student so, more people hear it because that's always fun. Anything I'm forgetting?
Jonathan Hall:You can also give us money in exchange for swag if you really wanna do that too.
Shay Nehmad:Yeah. We're this is not really supporting the show. It's more just fun stuff because I think on average with shipping, it's like we're totally breaking.
Jonathan Hall:We're breaking. Yeah. But still, it's nice to have a little, rooster mug or t shirt, and then people would say, what? Where'd that t shirt come from? And you can tell them about the show that way.
Shay Nehmad:For sure. If you're planning to go to any of the conferences usually, we talk about conferences on the show. This week, we, like, skipped the conference.
Jonathan Hall:And Foz didn't just happen, and that was completely off my radar because I'm not not in that part of the world anymore.
Shay Nehmad:That's we're not giving the European vibes anymore
Jonathan Hall:for the show. That's right.
Shay Nehmad:Gofer Khan Israel is coming up soon, but I'll mention it next week. The Swag Store is also, open and the best item that you didn't mention is the hoodie. Oh, yeah. That has become a daily wear for me, especially now that it's so cold here in Israel and raining. We also wanted to mention a cool new service by a friend of the show, Josh Bleaker Snyder.
Shay Nehmad:If you have a merge conflict, how about you go to merdenicht.ai,merde.ai, and have the AI fix it for you. Not gonna say anything else if you wanna go and check it out, check it out if you struggle with merge conflicts. Thanks, Josh, for being a friend of the show and developing all these awesome, services. That does it for the ad break. Let's jump to the interview.
Shay Nehmad:Don't forget to come to our live episode next week. So, Jonathan, we've been doing this show for, like, two years now, and I think it's time we we find, like, a better name.
Jonathan Hall:Okay. We
Shay Nehmad:have Capa Go. Capa Go is nice, But maybe we can do something I'm imagining old timey sign, Hole and Nechmad. Nechmad and Hall. What do you think?
Jonathan Hall:That's gonna be hard for me to pronounce. Even though I lived in Holland for a while, that that sound just isn't natural for me.
Shay Nehmad:Well, it's not like we can do Hall and Ball. Oh, wait. Thorsten, why don't you tell us how we pronounce your last name?
Thorsten Ball:Hello. You know, the the last name is not the problem usually. Ball is pretty easy to pronounce the first name. Thorsten is hard for people. And only this week I've realized that I can you know, in the past, I said, like, think of Thor and then at Sten, you know, Thorsten.
Thorsten Ball:And this week I realized, no. Like, people know how to pronounce Thomas in English, and it's just that with a different ending. So
Jonathan Hall:Yeah.
Thorsten Ball:I don't know.
Shay Nehmad:So think about it, Hall and Ball, if, if you ever wanna take the show on the road. Thorsten's easy, but maybe that's just because I played a lot of Skyrim.
Thorsten Ball:Oh, I see.
Shay Nehmad:Yeah. So other than, potentially renaming our podcast, why don't you tell the people about yourself?
Thorsten Ball:Yes. So thanks for having me. My name is Torsten. I am software engineer, occasional writer. I guess for this podcast, I'll do the Go focused introduction.
Thorsten Ball:I've been writing Go since one point zero came out in 02/2011 or '12, something like that. I I don't know. And then I joined source well, in 02/2016, I published a book called writing and interpreting Go, which some of you might know followed up in 02/2018 with writing a compiler in Go. I joined Sourcegraph in 02/2019 and worked there for the next five years, also working mainly in Go. And then this is where it gets spicy here.
Thorsten Ball:Last year, I left Sourcegraph or say, you know, one year and a few months ago, I left Sourcegraph and worked at you started working at Zed, and I started writing Rust full time every day. Yes. After, you know, three years and multiple attempts of using Rust in my spare time, NextivaGo, I did it professionally now for a year. But now I'm going back to Sourcegraph after this year, to see if Rust. So right now, in fact, I am unemployed this week.
Thorsten Ball:I'm starting the new job next week, so I can basically say anything I want right now because it's just my opinions and not those of my employer. Yeah. So that's my intro.
Jonathan Hall:Very cool. I have to first off say it's it's really nice to hear that even though you're doing rest now, you're still willing to talk to us from your Gophers.
Thorsten Ball:Man, I still like Go very much, and I missed Go. I I really missed a lot of aspects of Go.
Jonathan Hall:We'll talk about that in a little bit. Go ahead, Shay.
Shay Nehmad:Yeah. I wanted to start other than all these things, you also have a substack called, register spill.
Thorsten Ball:That's right.
Shay Nehmad:Which is how I actually sort of got to know you. I think I I, like, saw the books before, but I I have to admit I didn't read them. And you have, a specific newsletter you put out called Joy and Curiosity. And I've been scrolling through Substack because I've been trying to replace social media with, like, better things. I just saw, like, you know, people trying to prep you for interviews and, like Yeah.
Shay Nehmad:Yeah. Obviously, like AI generated slop that that nobody wrote and nobody's gonna read and it's just gonna, like, generate numbers. And then I stumbled on joy and curiosity and it really it really caught me. The the way I see it, maybe you explained it a bit differently, but it's sort of a list of, links that you found interesting that almost necessarily aren't the main context, the main focus, the main job. They're almost necessarily these like cool random things that you would like send in in the random underscore tech channel in your Slack or like in your Telegram group, with your with your co students or wherever you are where you have a cool link.
Shay Nehmad:It's not directly what you're doing right now, but it's just so cool that someone built it about, like, tech. Someone built something really cool or someone wrote something really cool. You have to put it somewhere. And you sort of aggregate all these cool links you find, and just put it in a weekly newsletter. And instead of calling it the tech roundup or blah blah blah, you call it joy and curiosity.
Shay Nehmad:So before we dive into the, like, specific links and how do you collect them and what do you like about it and what's coming this week, Why joy and curiosity? Why that name?
Thorsten Ball:Good question. So, yeah, first of all, thanks for the appreciation. I
Shay Nehmad:appreciate it. Everybody listening, go sign go, like, go follow if you have Substack, which I
Thorsten Ball:Yeah.
Shay Nehmad:I don't know if I recommend. I don't know if I recommend, but you can do it in the email newsletter as well.
Thorsten Ball:There's also an RSS feed and whatnot. Yeah.
Shay Nehmad:Yeah.
Jonathan Hall:Yeah.
Thorsten Ball:And it's it's a normal email newsletter. Yeah. So so the, you know, the background is the newsletter is called register spill, and I started it because like you, you know, when Twitter morphed into x and every day people were talking only about social media and they were switching to this platform or doing this other thing or quitting it altogether. I realized that I kinda want a place that's not that noisy or a place where I can just write stuff. So I started writing register spiel with the idea being that once a week, I just sit down Sunday morning, it was in the first half year, and just write for now what's on my mind and, you know, spilling the registers.
Thorsten Ball:And then over time, it slowly transformed into, you know, now it has, like, thousands of subscribers. And then I added this special edition that comes out every week of the same newsletter called Join Curiosity. What was your sorry. What was your you had some good way to phrase it. What was the question you just had?
Thorsten Ball:The way the way the white process is.
Shay Nehmad:The tech roundup, Thorsten's awesome links. You won't believe which links Thorsten found this week.
Thorsten Ball:Yeah. The so okay. This is how I wanted to tie it together. I just said this. I wanted to quit social media because, not quit it, but I think there's a big problem with social media in that people often just it seems to come easier to people to share negative stuff or stuff that they don't like.
Thorsten Ball:And I've become super conscious of this, and I really try my hardest not to do it. Sometimes I slip and then I delete a post after ten minutes, which sounds incredibly weird to somebody who's not, you know, addicted to social media. But what I wanted to do was, you know, spread positive things and spread curiosity in the thing that we all do every day, like programming computers. And the reason why I started to do it was that while writing register spill, I realized that, you know, it's hard to come up with a topic every week. That's certainly one thing.
Thorsten Ball:And then one trick that I found for me was that, you know, people have been saying this to me for a few years. They they're like, oh, you you're like, your passion is inspiring, or you get other people excited about this stuff. And I can't help it. Like, I just get excited about computers and programming, all of this. So then when I started to think about what should I write about in my mind, I went, look.
Thorsten Ball:Like, if I went out for beers with the buddy, like, what would I talk to them about if they were also programmer? And then I just started to write about this. Like, this was my mental hook. And then I realized that I also want to share this other thing. Like like exactly like you said.
Thorsten Ball:Like, I would send friends these links in, you know, iMessage and whatnot or WhatsApp. I said, look at this. Or I would tweet it out. Right? And people appreciate this or thought it was interesting, so I started to put it in a newsletter.
Thorsten Ball:And then it started with me just putting it at the bottom of another, you know, mini essay, whatever you wanna call it. And then I made it the main thing every week. And, yeah, joy and curiosity, I think it's this. It's a good combination to have, you know, enjoying what you're doing, being curious about it. And I think that's kinda what I want to foster.
Thorsten Ball:And that's also, you know, why I find this stuff interesting because it does give me joy. And I do find like I also realized that the reason why I'm still doing this, and I I don't know how long you guys have been programming, but I noticed that when you start out programming, in the beginning, you kinda assume everybody's the same roughly or has the same motivations when you have no four other junior developers. You all wanna become a senior engineer and whatnot. And then after a few years, in my case, I saw people that have been programmed as long as I have kinda drop out or get bored by program because they, you know, after doing it for eight years, you just become bored of it. Like, I myself have, you know, now that I've been doing it for nearly, you know, twenty years, driving a car has become boring for me.
Thorsten Ball:You know? And if you're 20 years old and you've been driving a car for two years, you're super excited about this. And I realized that this is what other people go through with program. They kinda lose interest, and so that's fine. Like, that's completely okay.
Thorsten Ball:You know? But for me, I realize it just doesn't stop. I still find it interesting. I still find it, you know, it brings me joy, so I kinda wanna foster this and and share with others. And it seems to resonate, like, it seems to resonate with a specific group of people.
Shay Nehmad:Like, For me, it's really itching my like, I get a lot of newsletters and most of them don't last over, like, two months in my inbox. Some of them I signed up to, you know, because they're friends or whatever. But, like, ones that I signed up to actually to read, most of them, like, don't last. Most of most of them I start archiving and then the moment I realized, like, oh, I'm auto archiving. This one, I'm gonna unsubscribe.
Shay Nehmad:But this one, I'm like, oh my god, so many cool links. I've never heard that OpenOffice can't print on Tuesdays. What what the fuck? Like, how does that work?
Thorsten Ball:Yeah. I there's this internal struggle that I have from time to time where I think, you know, oh, if I wanna grow my audience, I guess I should do something that people think will be useful for the day to day job. You know? Like, if you write a newsletter that's called keeping up with the news things in AI, whatever, then maybe you people sign up because they think it's important for professional development, and maybe they could get their boss to pay for a subscription. But then I always realize that, nah, like, that's not what I'm interested in.
Thorsten Ball:Like, I would rather do this other stuff. And, I mean, you know this, I guess, but I'm often also surprised by how people respond to what I put in because it's so sometimes I just link to a tweet or sometimes just to one sentence of a tweet or a comment on Hacker News or wherever it is. Basically, just things that sparked other thoughts or that created some sense of curiosity or joy. And not to, you know, not to go too meta, but over time, I realized that in the beginning, the thought was, oh, isn't it easy to find these links? Like, people have probably seen this to go by on hacking.
Thorsten Ball:So they've probably seen this go by on Twitter or wherever. But then I realized that everybody's really unique in the way they absorb information, and that's what makes them unique in some sense. Like, all of that stuff comes in and only some of that stuff resonates. And if you kinda take that stuff that resonates with you, you will end up with a unique combination even if all the inputs are equal. You know?
Thorsten Ball:And so so I don't know. I've been leaning on this.
Shay Nehmad:It's it's a lot of fun. I I do have a question. Yeah. Jonathan and I, like, we also have a similar cadence. Right?
Shay Nehmad:Once a week? Is it a weekly thing? Now I realize it's coming on Sundays, but I don't know if it's every week. Yeah.
Thorsten Ball:It's weekly. Yeah. Yeah.
Shay Nehmad:Yeah. Yeah. So cool. So, Jonathan, we have a similar cadence. Right?
Shay Nehmad:We, collect links over the week and put them in our in our backlog. Then once a week, we do a program about it. So you can draw similarities between what we do and what this newsletter. You know what I mean, Jonathan?
Jonathan Hall:Yeah. Yeah. That's right.
Shay Nehmad:We cover three topics, like four, maybe five, and then they it's really the long shows where people don't even get to the ad break a week and that's a struggle. I'm opening this last one join well, the second to last one, join curiosity number 23. I think there are like 30 links here. How do you get so so many links and actually read them? Do you have, like, structured time every day to go hunting for links and, like, actually fostering it?
Shay Nehmad:Is it it doesn't happen, like, drive by?
Thorsten Ball:I it morph It definitely changed how I consume things. So I do have a note. I'm looking at it right now. Basically, it's just an Apple note in which I drop stuff. Right?
Thorsten Ball:Just stuff that might be interesting. And sometimes I add a thought next to it or some other thing. And right now, I have one, two, three I don't know, 15 or something, and I'll probably throw five of those away. So over the week, I just drop stuff in. And out of those 30, you just said, I think, you know, five of those might just be links to tweets or something or stuff that somebody else sent me.
Thorsten Ball:Maybe I think in that issue, maybe even added a comment like, hey. I haven't read this fully yet, but blah blah blah. And I think the rest is just stuff that I, like, just came across, you know, like a video I saw or some tweet file by the link to some other thing. And then there's two things happening, which is this newsletter, like, made me then more conscious about, you know, like, oh, I should really read this. So instead of just saying, oh, this looks interesting and skimming of it, I'm like, okay.
Thorsten Ball:Before I link to this, I have to read it. So I then make some effort to put some time aside. I also, you know, now have the Readwise reader in which I add stuff and then try to read it. So number of links also has grown, but it's not I wouldn't say it's something that I put a lot of extra effort in. It's mostly all of the reading I do anyway.
Thorsten Ball:And then just, I don't know, like, then just collect it and maybe there's one or two or three things. Like, there's been Thursdays where I'm like, I don't have anything. I gotta write this on Saturday or something and, you know, I've got four links or something. So then I would dig into my email inbox, like links that I sent myself, or I would look into reader or some other stuff. I also have, you know, backlog, like if something becomes too long, I put it on a backlist, okay, and link it to some other time.
Thorsten Ball:And then I mean, you also see me sometimes not cheat, but I I can do whatever I want in that format. It doesn't have to be up to date. So sometimes I can just pull out old links and say, I watched that video again that I've been watching for ten years. You know? Like, Rich Higgy, it's simple made easy.
Thorsten Ball:Right? And it's this. I've seen that talk five times, but then it comes up in a conversation with some other program, and I'm like, I should link to this again. Right? Because not everybody has seen it.
Thorsten Ball:And if you internalize this, that David Ogilvy, you know, marketing guy, he said, you're not advertising to a standing army, but a moving parade, meaning you're not always sending stuff out to the same people. Like, every week, different people might tune in, and not everybody has seen the same stuff that you've seen. It's totally fine to share link that, you know, some people have seen maybe, you know, like, five years ago. Like, they won't get pissed off by this. They won't say this guy have seen this before.
Thorsten Ball:Like, why does he link to this? You know? Yeah. But others might not have seen it. So there's nothing bad about being loose with what you put in.
Shay Nehmad:Cool. Really good point. So what I need to internalize is I need to watch Simple Made Easy again.
Thorsten Ball:Oh, yeah. Yeah. Yeah. It's a great talk.
Shay Nehmad:Yeah. It's a great talk. I reference it all the time. One final question about the joint curiosity. What's the, like, on the upcoming one, what's the, like, link that, you're most excited about?
Thorsten Ball:Let me see. Let me see.
Shay Nehmad:You're you're all getting a sneak preview just because you have the press, we have the press.
Thorsten Ball:I mean, I've been really into the AI stuff, and, obviously, there's been a lot of interesting thoughts on the whole deep seek release last week. And, you know, now analysis coming out, and I'm kinda I don't know. Like, I I the whole, you know, oh, it's a bluff. It's a double bluff because they pretend it's, you know, like, it costs more than they say it cost. Like, I find that stuff fascinating.
Thorsten Ball:So there's a bunch of stuff that I kinda bookmarked and read. And then the other thing that I was staring at where I wrote three lines of comments already in my notes was was somebody was comparing Wayland versus x 11 input latency on Linux. And I worked on the zed Linux version last summer on the x 11 version, and it wasn't a good time. And I read this, and I just thought the the whole I don't wanna, you know, offend anybody who's a big Linux fan here. But if you read this, the whole Wayland versus x 11 thing, and then you compare it to, say, macOS application development, it's night and day.
Thorsten Ball:Like, it's it's just there's no wonder that, you know, high quality applications on Linux are rare, you know, or say cohesive, high quality applications on Linux are rare. So, yeah, that's kinda it's this. And then I don't know. There's, Jimmy Miller wrote a great blog post that's in there this morning. Also this week, I'm unemployed.
Thorsten Ball:I have too much time this week. Yeah. This week also, there was some interesting thing with somebody, some guy who makes, like, all of these different fonts, and he had, like, a new monospace font that he said as a system font in macOS, which I thought was interesting.
Shay Nehmad:Monospace system font on Mac?
Thorsten Ball:Yeah. Yeah. Yeah. So this
Shay Nehmad:is This is this is exactly what I mean. Like, I wanna say, okay. Let's stop the recording. I have to go install it and see how it looks because I
Thorsten Ball:already have mono
Shay Nehmad:I've I already have mono space, mono space font on Slack, and Yes. That's been, like, really helpful.
Thorsten Ball:But One link above this is a link to a YouTube video called let me see what the name is. The Intel four eighty six d x two sixty six megahertz startup sounds. It's just a what is it? Fifty second video of the startup sounds this computer makes. And that brought me joy.
Thorsten Ball:You know? Like, just listening to that thing boot up and the the hard drive and, like, the beeping and
Shay Nehmad:Cool. Cool. Cool. You mentioned you were very much into the AI stuff. Mhmm.
Jonathan Hall:I wanna hear about that.
Shay Nehmad:Let's talk about that a little bit. Yeah.
Thorsten Ball:I I don't know how to kick the I don't wanna fall into the to trap here.
Jonathan Hall:I think I think there's, like, a few hot takes out there, right, that that we can either agree with or or or, debunk if we want to. So the of course, the big hot take is developers are all gonna lose their job within five years. Or some will go so far as to say everybody's gonna lose their job within x number of years. You know? All creative jobs will go and then manufacturing jobs will go because AI's will build the robots that manufacture things.
Jonathan Hall:Nobody will have any jobs anymore. So that's the one extreme. And then and then and then there's other hot take. This response to that is, no, that's all baloney. Every time we've had an advance in technology, it's, you know, cars didn't didn't, didn't reduce the number of people who who went around it, increased it.
Jonathan Hall:Right? You make the road bigger and more people drive on it. The Internet meant more people want or when we went to AWS, server sales skyrocketed because now, people are selling more servers, etcetera, etcetera. So, you know, the the flip argument there is AI is may make us more effective on a per task basis, but we're gonna take on so many more tasks and do so many more things that there's gonna be a huge shortage of development talent out there. Two strong views opposite end of the spectrum.
Jonathan Hall:Where do you fall, or do you fall off of that spectrum somewhere?
Thorsten Ball:I would say I fall right in the middle Mhmm. In the sense that
Jonathan Hall:That nothing's gonna change at all? I know that's not a new thing because I've written something.
Thorsten Ball:Yeah. Yeah. Yeah. No. What I mean is I've been you know, I worked at Sourcegraph two years ago when the whole AI thing start to kick off, when Sourcegraph also went and build an AI assistant.
Thorsten Ball:I was part of that team. And back then, I was highly skeptical because, you know, the variance was too high. Like, it was used JADGPT and it would spit out like 50 lines of Python. Impressive. You try it again the next day.
Thorsten Ball:Completely unimpressive, totally wrong. And then you had all of this hype going on, and I just didn't see it. I didn't see how it would even change things. It felt like you had an unreliable Google or Stack Overflow or something like this. And I didn't understand it.
Thorsten Ball:I didn't know looking back, I also didn't know how to use it, or I didn't understand it enough to know what to use it for. And I think in the last two years, this has changed for me by one thing, and that is the models have become better. Like, that's just that's just what it is. I think even a year ago, it was completely different. But now since the summer cloud three point five, anthropic, that's a completely different thing than two years ago, chat g p t.
Thorsten Ball:And that so that's one thing. And the other thing is that I what really changed things for me was in the last two months, I worked on the AI team at Zed on inline completion, meaning, you know, what cursor has copilot inline completion, and we built this for Zed. And to do this, we work with LLMs, and we fine tuned LLMs. And I you know, when you do this, you collect data, which is a lot of plain text. You have to label data.
Thorsten Ball:You have to generate data. You have to analyze data. I realized that LLMs are really good for that. Like, they're really, really good at taking, say, a hundred files, and you give them the option of which of these five labels would you give this file. And if you run this a bunch of times or even just once, you get pretty good results, and you can see how this is a lifesaver.
Thorsten Ball:And for me, I don't know how to put it into words yet, but for me, I started to realize that there's this mechanical nature of it all where, oh, these things are really good at taking fuzzy stuff and turning it into non fuzzy stuff, which is a big thing that we've been missing in computers. Like, you know, I mean, you can now take a screenshot of this screen and and send it to Claude and say, where should I click, you know, to to end this call or something? And it can pretty much tell you where to click. And that's huge, I think. And working with text, working with data, fine tuning stuff, playing around with inline completion, and seeing how good it can be and how much of the mechanical stuff that you do in day to day programming it can take away from you or it can help you with made me realize that, oh, like, this is I made a category error.
Thorsten Ball:I I the LLMs are not this, you know, this oracle that helps me do all of the stuff that, you know, some people say it is or it might be one day. But for me, it's just more this, oh, they're really good at working with text. And guess what? I work with text all day because I'm a programmer and, you know, code is text. And give you, like, one example here.
Thorsten Ball:You know, I I was a Vim user for many, many years, two decades, basically. And in the Vim community, people are obsessed with being really efficient with, like, navigating through text and whatnot. And there's, like, you know, how you jump, and I think it's called easy I don't know. Quick scoping and easy find and all of these things to jump to specific characters.
Jonathan Hall:Mhmm.
Thorsten Ball:And while working with LLMs, I realized that LLMs can be in the same category as these tools. If you have a small local model, it can predict where you want to jump next, and then you just have to hit tap tap tap tap. And putting the alarms not on you know, taking them down from this pedestal and putting them into the category of this is a tool that works with text, a a thing that, you know, the fit I can use the mechanics this thing allows me to have to get more productive with the thing that I do. That kinda changed things a lot for me. And I don't know how much of this made sense, but it's this, I guess, it changed things for me.
Thorsten Ball:And I see them now as really useful things to work with text, meaning they can be pretty good when writing code. And they can also be really good at generating small amounts of code. And they also allow you to turn a lot of fuzzy stuff into non fuzzy stuff. And they can be super helpful, you know? And one last thing on this, and this is not, you know, the appeal to authority, But people were super skeptic of AI.
Thorsten Ball:And I, like I said, I was too. But then you would see all of these programming legends suddenly switch over and kinda get interested in this stuff and see it or use it, you know, like Antares from Reddit. You know, John Carmack famously, right? And Mitchell Hashimoto Hashicorp, he doesn't use a language server, but he uses Copilot. Right?
Thorsten Ball:And that guy is one of the best programmers I ever seen in action, and he trusts Copilot to write Zeek for him. And that changed it for me where I start to think, okay. If these guys use it, I should take a closer look. Like, then it's not just all bullshit. Something has to be there.
Jonathan Hall:Mhmm.
Thorsten Ball:And yeah. So now, you know, will they take our jobs? Will they, will we all be out of a job in a few years? I do not know. Are they just a fancy autocomplete?
Thorsten Ball:No. You know, I don't think so. But I do think they are a, to sound super fancy, a paradigm shifting tool that will change programming. I I think programming has changed multiple times over the years. Right?
Thorsten Ball:We started with punch cards. Nobody uses punch cards anymore. Right? Why should programming stay the way it's been for the last, say, ten years? And where people write just text in a text editor?
Thorsten Ball:Everything. Right? Why wouldn't we go into this hybrid mode where some code is written by AI and people review it and other code is written only by humans and stuff is merged. Like, I I think stuff like this will happen where we change the way we write code because now we have these machines that are surprisingly good at writing lots of, you know, small amounts of code. And Mhmm.
Shay Nehmad:Let me ask you something. Yeah. The you mentioned just Vim users being, like, obsessed about the finding. I find well, I found a lot of joy. Like, I I felt smart and cool when I wrote this macro which fixes I don't remember which, Golangsi Island error, but a pretty complex Golangsi island error that does have a pattern.
Shay Nehmad:I think it was, reusing or something like that and not reassigning it. And I found a way to copy, like, the name of the function that I'm, doing and then capitalizing the first word and then putting it the end of the error and then adding a colon if it doesn't exist already in the file. And then I had it in a macro. I felt super smart, and then I did it a couple of times. It obviously took me longer than it would to, like, you know, command I on the file in Versus Code and tell, Copilot, hey, can you please rename all the error, variables to, you know, the corresponding names to avoid this lint error?
Shay Nehmad:And, you know, if you click on, like, IntelliSense right now, the first options are not the auto complete and and the previous IntelliSenses are explained with Copilot and fixed with Copilot. So it wouldn't even have to type anything. Like, it would just offer me fix with Copilot. A lot of you you speak about this with a lot of passion where the others things you mentioned enjoying curiosity are often very low level programming things, right, that feel to me that are gonna disappear. We're not gonna do cool vMacros anymore because Mhmm.
Shay Nehmad:Like there's no point. So how can you be so excited and curious, and joyful about these, like, very human program y, things that feel to me like they're drawn from, like, ages past and not looking forward, and also be very excited about this, like, Pacific Rim mind melding you and Claude going to attack the Godzilla together.
Thorsten Ball:I think the answer is that I don't see it as two opposing ends. And I once saw it like this. Like a year ago, I would have said, in one corner, we have AI slob as they call it. And on the other end, we have handcrafted software where people with care and passion, expertise and skill, you know, aligned every line. You know?
Thorsten Ball:But then what changed for me was that I started to find or I began to find the same fascination with LLMs as I had for somebody who writes VIN Macros. Right? Like, you have to have some kind of, I don't know, romantic idea of what you're doing to become fascinated by VIN Macros. Right? Because if I go out on the street and I tell somebody, hey.
Thorsten Ball:I just use, like, two t's keystrokes to do what others, you know, would use 40 keystrokes for. They don't give a damn. Right? But you do because you care about something here. Right?
Thorsten Ball:And it's not this, oh, a person is sitting on front of a screen and typing letters on the keyboard. You have some idea of, you know, what you find fascinating about this. And once I started looking to LLMs and I realized that it's this, you know, all written text on the Internet compressed into numbers, into vector space, where you can have, like, you know, one vector meaning the king, and then if you subtract the vector of, I don't know, male and you get out the vector of queen or something like this. Like, it's just marvelous that this stuff works at all. You know, it's just fascinating.
Thorsten Ball:And then when you think that you took like, they took all of the public code of whatever it was and condensed it into numbers, and now I can take my code, turn it into numbers, and shove it in there, and out come a bunch of other numbers that are a completion of the code that I just wrote. That is fascinating. That's as fascinating to me as the guy who spent ten years, you know, hand aligning his Emacs configuration to whatever it is to, you know, be aligned to the Fibonacci numbers on every tenth line or whatever. You know? Like so I get fascination from this.
Thorsten Ball:And the other thing is that a bigger, you know, equally important thing is that I don't like doing things that I think are a waste of time. And I think with the advent of LLMs, a lot of programming for me felt like a thing of the past or waste of time. Like, hey. I'm oh, I I wanna write a script where, you know, I have five CSV files in this directory, and I wanna load them all and concatenate them and output them as, like, an XML file in some other thing. And I can do this.
Thorsten Ball:Like, I can write you that script in probably three, four, five different languages. Right? But I will have to look into documentation. I will have to figure out how this library works, whether this has know, CSV in the standard lib, blah blah blah blah blah. And doing this and writing this by hand for the sake of writing it by hand just feels like a waste of time to me.
Thorsten Ball:Like, I can hit a button, and Claude can generate me that same script, then I can move on and do other interesting stuff. So, yeah.
Jonathan Hall:Thanks for sharing your thoughts on AI. I I think you have a pretty balanced, take there. It's kinda refreshing to hear when after you know, as as we're talking about on social media where everyone's always, trying to get get the last word and hot takes everywhere. It's nice to hear a balance to you. And on that note, you used to do a lot of Go, and now you're doing Rust.
Jonathan Hall:I'm curious if you would compare and contrast those, maybe in a balanced way or or or or hot takes if that's your thing. Whatever. What what are your thoughts about those two those two languages? Some said it is a hot topic out there.
Thorsten Ball:Yeah. I'll my balanced take is that I think that those values overlap with my personal values more than Rust's values overlap with my values. Okay. And specifically, I think built into Go, there's a big focus on simplicity and minimalism and and and, like, I don't know, no bullshit in some sense, rawness, whatever you wanna call it. And I've and also pragmatism and trade offs and all of that.
Thorsten Ball:And I think that resonates with me a lot. And that's also what I've been missing a lot in Rust. For example, you know, Go famously has made trade offs to not put stuff in the language that I could have put into language to keep, say, compile times down. And then people say, oh, Go should have have should have this and should have that. But I love that they have this explicit trade off.
Thorsten Ball:Like, all engineering is the managing of trade offs, I think. And they made some good trade offs, and they really care about the ergonomics. They've I think the first language had really shove this into the focus, the ergonomics of a programming language. Like, oh, you have a build and test tool. That's amazing.
Thorsten Ball:You can you can compile and run and test your code with a single tool. You have a format, a build in tool. The the format has no configuration. Right?
Shay Nehmad:Mhmm.
Thorsten Ball:That's what I love. And they made trade offs. Sure. It lacks in other areas, but they consciously, I think, made these trade offs. And if you look at their issue tracker and, like, you know, Russ Cox comments and whatnot and in Lance Taylor's comments on how they think about adding stuff to the language, you can see that these are incredibly nuanced and detailed discussions where they have a lot of thoughts on, you know, pros and cons of different things.
Thorsten Ball:And I really appreciate this. And with Rust, I found that, you know, the community, I think they they like different things that I like. It's it's a nice language. It's often also fun to write it. Sometimes it's not.
Thorsten Ball:They have some amazing features in it. I do love the enums. I do love the match statements. I do love, like, I don't know, like, how much you can do with the print line command because I'm a print line debugger. You know, there's a lot of stuff that I love.
Thorsten Ball:They have include string, which is like how you you know, to go embed stuff, but kinda nicer. I also have to admit, I do like macros sometimes, like writing my own. It's cool. Join curiosity. There you go.
Thorsten Ball:Macros. But for example, whereas Go would say, you know, a little copying is better than, you know, adding a dependency. You know? Oh, I just need this function from this, you know, library. I'll just take this function, put it in my code.
Thorsten Ball:I would say that in the Rust community, they would add another crate or something. They would add another dependency. Mhmm. And that just is to me, you create a new like, I asked this for scientific purposes a while back. On Blue Sky, I asked, like, if you had to create a small web server with Rust, like, which library would you pick?
Thorsten Ball:And they would say, oh, Axum, you know, like, which is a web web framework with Tokyo. If you do this, I don't I don't have the number, but I would bet 50 dependencies will be pulled in. Right? And your target folder that contains, like, the comp the object files and compile dependencies, it'll be huge, like, hundreds of megabytes. And that to me is just, like, guys, like,
Shay Nehmad:I don't know.
Thorsten Ball:Like, I just it doesn't resonate with me. And there's other stuff like this where I think and I've wrote I've wrote this many times in news sites and on my blog that I am not a perfectionist. I don't believe in perfection. I think I'm an 80% inch, and I don't think perfection is achievable. There will always be trade offs.
Thorsten Ball:There will always be downsides to anything.
Jonathan Hall:Mhmm.
Thorsten Ball:And I think the Rust community would think that there is perfection achievable. Like, it's always a %. And it's this, oh, I I I I wanna get the current time stamp and print it. Right? And go time dot now and you print it.
Thorsten Ball:Mhmm. In Rust, wait. Wait. Wait. What time?
Thorsten Ball:Like, what time zone? You know, what time zone in relation to what other times how do you even wanna print it? Because that's not, you know, like, if you don't specify this, you just might get this out, and it's undefined. So blah blah blah blah. And this, you know, sense of, I don't know, perfection is achievable.
Thorsten Ball:Everything has to be a % perfect. That doesn't resonate with me. And the other thing, I think over the years as a programmer, I've kinda become wary of abstractions. And I think in Go, there's not a lot of abstractions. I kinda have to admit that I've never really tried Go with generics.
Thorsten Ball:I might be missing out, but I don't think I am.
Shay Nehmad:Don't worry. 90% of people do that.
Thorsten Ball:And so I become kinda skeptical of too many abstractions. And I think in the Rust community, there's still the sense of, oh, abstractions are good. Like, if we can make this indirect and this side doesn't have to know about this side, then that's better code. Even though there's only one caller and one callee, and this won't change in the foreseeable future. But how about we separate these concerns and put this layer in?
Thorsten Ball:And this is actually a trade, and this implements this trade, and this does this. And I think there's a lot of this and often for good reason. But, you know, everything I'm saying is just to say that there's no specific thing where I was, oh, this is super bad about this, and this is super bad about this. And I also don't agree with a lot of these blog posts that compare languages on, like, a feature by feature basis and whether this type system does this, whether this type system is this. For me, it's, you know, what the kids would call vibe space.
Thorsten Ball:It's like, do I do I think if I use this language, do I feel like I'm running in the same direction that the community or the author of that language was running towards? Or am I going with the flow of that community or that language? And with Rust, I often had this feeling that, ugh, no. You know? Like, no.
Thorsten Ball:No. No. Like, I I would trade a lot of the type system and the bar checker to get better compile times. And the Rust community, I don't think, would sign that contract. You know?
Thorsten Ball:Yeah. That's fine. And and the Rust the Go community is different. And JavaScript community is completely different, you know, again, and and Python and Ruby. And yeah.
Thorsten Ball:I don't know. It's not a technical answer, but that's kinda my my take on things that I like simplicity. I like ergonomics. I like trade offs.
Shay Nehmad:That means you're going back to Go, this year?
Thorsten Ball:Yeah. I mean, at Salesforce, there's still a lot of Go. I think, I don't think they ripped out all of it in a year. Yeah. I think there'll also be a lot of TypeScript.
Thorsten Ball:There'll also be Rust, I think.
Shay Nehmad:There's this wave of people who've been working a lot with Go. You mentioned, HashiCorp. That's a good example. Right? He wrote Go for, like, ten years and then moved to Zig.
Shay Nehmad:Yeah. And I saw a few of these blog posts. And, Jonathan, I think we even discussed some of them of people, like, sort of burning out on the simplicity and looking for something more challenging.
Thorsten Ball:Yeah.
Shay Nehmad:But I guess it's just what you're looking for. Are you looking for a challenge in the thing you're building? Do you wanna get it built and and and see what it does? Or are you looking for a challenge in your editor? Are you looking for architectural challenges?
Shay Nehmad:Are you looking for business challenges? Are you trying to, like, build a ton of businesses and the code is just a means to an end? It it doesn't matter at all?
Thorsten Ball:Yeah. I I everything I said so far in this recording is always comes with the asterisk. And it
Shay Nehmad:will be used again.
Thorsten Ball:It always comes with the asterisk of it depends on the context. Like, do I like somebody code golfing and being, you know, building the most minimal thing in VIM language? I do. Yes. On a Saturday morning, but not at work.
Thorsten Ball:You know? Not when I'm shipping something that's to be used in production. And I think I I I watch a interview with, you know, person who's using Rust, and they also switch from Go to Rust. And I don't know how experienced they are, but they said, oh, they like Rust because it's fun, because in Rust, it's like all of these little puzzles that you have to solve. And I think Rust in that sense, it's like catnip for programmers.
Thorsten Ball:It's this you get the feeling of accomplishing something by writing code in that language, and you get that that sense of, oh, hey. I solved another puzzle by making this match statement. Like, do exactly this in that amount of code. But I would say that's cool, but have you actually shipped something of value to your users or to your customers? And, you know, yes, on a Sunday morning, give me all of the code golf you have.
Thorsten Ball:I love it. I yesterday, I saw, like, a a Game Boy emulator in JavaScript that looked written by an alien life form. I'm gonna link to it too in my newsletter. Is this code that I wanna debug in production? 100%.
Thorsten Ball:No. Yeah. You know? And I think sometimes if you say, oh, I I'm tired of Go simplicity, that doesn't mean it's a bad language to build products in. Right?
Thorsten Ball:Mhmm. It just means that you got tired of something. But that's a different thing, man. Like, that's a different thing than
Shay Nehmad:So the formula is rust on the weekends.
Thorsten Ball:Yeah. Yeah. I mean, to be fair, like, it's an absolutely the how much do you enjoy working with the language you're working? That's it's completely fair thing. Right?
Thorsten Ball:Mitchell Hashimoto, he said, on the Primogen, like, podcast two weeks ago, he said, all he can say is that he doesn't enjoy working in Rust. Like, that's all he can say. Like
Jonathan Hall:Mhmm.
Thorsten Ball:There's technical stuff you could talk about, but it doesn't change the fact that at the end of the day, he doesn't feel like he's enjoying this. And for me, it often was the same. Like, I would ride Rust and think, it's not fun. Like, it's not I don't I okay. I can see why, but this is not fun.
Thorsten Ball:And
Shay Nehmad:I'm getting back into TypeScript right now and and Yeah. It's it's been difficult for me to find this spark of, like, oh, this is awesome. This is fun. Yeah. I'm programming instead of, like
Thorsten Ball:Yeah.
Shay Nehmad:Fighting with a a type type system bolted onto a browser language for some reason.
Thorsten Ball:The other just to get back on this point, like, people saying Go is born when it and and Mitchell switching from Go to Zig. I think for the stuff that he's building, Go is just not a like, the terminal emulator Go see, Go is not a good fit. Like, it just wouldn't work. Like, there's also this, and Zed is written in Rust. And I think it's Rust is the perfect choice for this.
Thorsten Ball:Like, they built this high performance text editor that interfaces with macOS, you know, system level frameworks like AppKit and, like, the the, GDC, the global what was it called? The global dispatch queue and whatnot. Grand Central Dispatch, GCD. Sorry. And you can't do this in Rust.
Thorsten Ball:Sorry, in Go. You you have to use Rust or some other language should drop down to the same. And for that, it is perfect. Right? But that doesn't mean it's perfect for all of the use cases all of the time.
Thorsten Ball:Right?
Shay Nehmad:For sure. So Cool. Cool.
Thorsten Ball:Mile take, I guess.
Shay Nehmad:So it's actually refreshing. Yeah.
Thorsten Ball:Is it?
Shay Nehmad:It's it's a shield they give.
Jonathan Hall:I think it's also a great a great segue into our stumper question. I guess this isn't a stumper question anymore. The the first one we did a couple of years ago, we tried to kinda stump people. They can think hard, but this shouldn't be too hard. The question is when you were learning Go, who influenced you the most?
Jonathan Hall:And and, like, you've already talked about how you identify with sort of the Go ethos. So I'm guessing you thought about this before. Who who who has influenced you? You already mentioned some names, but just just
Shay Nehmad:you know,
Jonathan Hall:who who who maybe the most?
Thorsten Ball:Two come to mind. One is my friend, Felix Geissendurfer. He now works at Datadog. I I don't know if you guys had him on, but, back then, 02/1213, he was starting to get into go. He would not shut up about it.
Thorsten Ball:Like, he would be he would we made a joke. Like, we were at j s conf in Florida in 02/2013, and he would be sitting at tables with different people. And and we would say, where's fields again? And and we would make jokes like he's trying to convert people to go again. And we would walk over and he would be sitting and talking to them about go, and he got me really interested in it.
Thorsten Ball:So then I started to look into it. And then I think, I don't know, like Dave Cheney was a influential guy. Like his blog posts and his, you know, writing on Go really made me see, like, the beauty and the simplicity of it. Then, obviously, you know, Ross Cox's writing or Rob Pike, like, his talk and, like, all of those early, you know, Go programmers that had this back then, I was coming from Ruby and JavaScript, and I felt like I was now in this other world where there was this sense of systems programming, which, you know, now looking back is kinda didn't turn out to be, say, Go's forte. And Mhmm.
Thorsten Ball:But there was this sense of, oh, you know, like, this is this is proper engineering. Like, the people writing the Go Garbage Collector, they know what they're doing. They they they really hardcore on this and the simplicity and the trade offs and the thinking not just about a friend of mine put it as he said, the big problem that Ruby made was that Matz thought that, you know, you know, Matz, the inventor of Ruby, he said, developer happiness. That's what's important. You know, like the the language should serve developer happiness.
Thorsten Ball:And my friend said he made the mistake of thinking that developer happiness is only about syntax. He forgot the tooling. Mhmm. And I think I don't know how true it is, but I think it's kinda an interesting angle to look at things from because ten, fifteen years ago, yes, the language itself, like, the syntax, you would write stuff in. That was so important.
Thorsten Ball:Like, people would discuss operators and whatnot and blah blah blah. Because they had
Shay Nehmad:time while their c plus plus was combined, so they had time to work it. Yeah.
Thorsten Ball:But then Go came along and you had, like, this fast, you know, Go test and Go build and and, you know, how you ship stuff single binary. And people would say, oh, it's such a fat binary. This is eight megabytes for hello world. And guess, you know, who cares? Like, nobody did and still doesn't.
Thorsten Ball:And this focus on ergonomics and, like, this conscious choice of trading, say, in some elegance for ergonomics, that's what, I don't know, really appeal to me.
Jonathan Hall:Mhmm. Cool. Cool. Cool.
Shay Nehmad:We are definitely writing down the people you said, which is difficult because some of their last names are hard to spell. To try to invite them on, we we did, I'm 85 sure we had Felix's, tweet about performance optimization in Go that he did. We we featured that on the show. So we might reach out to him as well. Cool, like, seeing these threads link because I didn't know you know the guy, but I I, follow him on Twitter.
Shay Nehmad:Cool. Where can people find all this joy and curiosity and high performance editors and all these beautiful links and yourself and the books, if you haven't read
Jonathan Hall:them?
Thorsten Ball:Yeah. So you can find me at torstenball.com if you know how to spell it, or you just the name of the newsletter is register spill, registerspill.TorstenBall.com. I'm on on Twitter slash x blue sky. Mostly these two, mostly these days. You can always send me an email.
Thorsten Ball:I'm happy to receive any sort of email. I don't know if I'll regret saying
Shay Nehmad:this. Yeah. People signing you up for all these sorts of
Thorsten Ball:My, books you can find interpretabook.com and compilerbook.com. And I think, yeah, that's basically it.
Shay Nehmad:And all these links, listeners, you can find in this week's show notes, which is in your app somewhere. I don't know what app you're listening with. So if you're out to your own custom Rust slash go slash TypeScript RSS feed, podcast thing with Claude last night, I don't know where you put the show notes, but there you can find all these links. Thanks a lot, Darston, for coming on.
Jonathan Hall:Thanks so much. It was a pleasure.
Thorsten Ball:Thanks, guys. Thanks Thanks for having me. This was fun. Great.
Jonathan Hall:We'll have to do it again and talk about other things.
Shay Nehmad:Waiting for the next, joint curiosity. I'm I'm I'm I can't wait to see what's on it. Awesome. Thanks a lot.