Oxide and Friends

Brogrammer Garry Tan has been boasting about "writing" tens of thousands of lines of code each day as the paragon of productivity. Is this really the right way to think about building systems? Bryan and Adam were joined by Polish software engineer, Gregorein, who took a closer look into what Tan was generating to discuss what was found.

In addition to Bryan Cantrill and Adam Leventhal, our guest (who we introduced a mere 4 minutes in) was gregorein.

Some of the topics we hit on, in the order that we hit them:
If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Creators and Guests

Host
Adam Leventhal
Host
Bryan Cantrill

What is Oxide and Friends?

Oxide hosts a weekly Discord show where we discuss a wide range of topics: computer history, startups, Oxide hardware bringup, and other topics du jour. These are the recordings in podcast form.
Join us live (usually Mondays at 5pm PT) https://discord.gg/gcQxNHAKCB
Subscribe to our calendar: https://calendar.google.com/calendar/ical/c_318925f4185aa71c4524d0d6127f31058c9e21f29f017d48a0fca6f564969cd0%40group.calendar.google.com/public/basic.ics

Bryan Cantrill:

Hello, Adam.

Adam Leventhal:

Hello, Bryan.

Bryan Cantrill:

How are you?

Adam Leventhal:

I'm good.

Bryan Cantrill:

And we gotta see if everyone else can hear us,

Adam Leventhal:

which they may. Anyone else can hear us.

Bryan Cantrill:

Can we get how do people vote with their emojis? Terrible sound. Terrible sound.

Adam Leventhal:

You can barely hear us. Oh, okay. Okay. Okay. Nope.

Bryan Cantrill:

Nope. Nope. Nope. Nope. Nope.

Bryan Cantrill:

We

Bryan Cantrill:

can solve that. Oh, I I Sorry. I I have I know what the problem is. Nevermind.

Adam Leventhal:

Look, I'm not blaming you.

Bryan Cantrill:

I know in this case, actually blaming me is exactly the right call. In this case.

Adam Leventhal:

All right. Are we off of Linux audio now?

Bryan Cantrill:

No, this was really embarrassing. Okay.

Adam Leventhal:

Okay. I can't hear anything on my headphones. Okay.

Bryan Cantrill:

Now you can.

Bryan Cantrill:

Okay. Now you can.

Adam Leventhal:

Ryan is now pushing buttons at random on a console.

Bryan Cantrill:

Hold on. Also making some sort of chimpanzee sounds. Okay. He's He's mashing banana into the keyboard.

Adam Leventhal:

Working out some script to Hamlet. Yeah. It is like, wait a minute.

Bryan Cantrill:

This is Okay. Now Now I'm thinking about the

Bryan Cantrill:

New Yorker cartoon again.

Adam Leventhal:

Now I have to Okay. No, the New Yorker cartoon

Bryan Cantrill:

that I My life's grand grievance. I'm just back there so quickly.

Adam Leventhal:

You said New Yorker cartoon for some reason, because you often talk about Far Side. I was thinking about Far Side cartoon and there was one recently captioned. This is a Far Side cartoon from like 1980.

Bryan Cantrill:

Okay.

Adam Leventhal:

That's So

Bryan Cantrill:

when you say recently, you meant like No.

Adam Leventhal:

But it was posted recently because the caption is the love boat wanders into Strait Of Hormuz. Oh, I remember that. I remember that far side. Was like

Bryan Cantrill:

I remember that far side. Why am not making that far side difference? Apparently,

Adam Leventhal:

is happening, like, right now. We're like cruise

Bryan Cantrill:

ships Attention love boat.

Adam Leventhal:

Are running the blockade.

Bryan Cantrill:

Okay. Does okay. Does love does the love boat cross the generational chasm?

Adam Leventhal:

No. No. I mean, it barely makes it

Bryan Cantrill:

to me. Are you right? Yeah. I the okay. So, yes.

Bryan Cantrill:

We know you can hear us. Now we wanna yes. Now now chat, now we want to we're we're beyond audio chat. We need you to weigh in on something much more important. Have you ever heard of the love boat?

Bryan Cantrill:

Yeah. I mean, it's

Adam Leventhal:

I know. And people say we don't have intro music.

Bryan Cantrill:

They they don't say we've got me having a few bars of the

Adam Leventhal:

love boat. That's right.

Bryan Cantrill:

I okay. I only know the love boat from South Park, says one. I South Park, there there is something that that about this that, like, we get these, like, echoes of of pop culture history Well, this the references that are made. Because, I mean, South Park is a is a rough peer of ours.

Adam Leventhal:

But this is true of of, of The Simpsons for us. I I don't know how often you're like watching some classic movie.

Bryan Cantrill:

And you're like Yeah.

Adam Leventhal:

Oh, right. That's what they were talking about. Yeah. Or like or like the Simpsons with the Twilight Zone episodes. Mhmm.

Adam Leventhal:

Like the black and white Twilight Zone episodes. I used to watch those. I I did too, like on Nick at night. Yeah. You know, but

Bryan Cantrill:

I know. I used to watch those on, like, channel two. So I mean, because I am like the I mean, the writers of this of the Simpsons are only a couple years older than I am. Yeah. Right.

Bryan Cantrill:

I mean, I like Conan O'Brien is probably 55 maybe.

Bryan Cantrill:

Yeah. 56.

Adam Leventhal:

I bet he's older than that.

Bryan Cantrill:

Is he? The yeah. Now we're on to like it's like, you guys are old. You guys are like next, we're gonna talk about our our does Quentin O'Brien have the same joint issue that I have, do think? What what does he use to alleviate back pain?

Bryan Cantrill:

The

Adam Leventhal:

Do do we have a guest?

Bryan Cantrill:

Conan O'Brien is 63. We got a guest. Right.

Adam Leventhal:

You know, I don't wanna

Bryan Cantrill:

we do have a guest. Where's our guest? Oh, dear guest. There's our guest. So, yeah, we've got we have guest.

Bryan Cantrill:

How long would we have gone? We you know what? History will never know. History will never we could have gone for a while. I just like you know, we don't even need to have this podcast.

Bryan Cantrill:

This is we do bladder.

Adam Leventhal:

It was like that time. We went like fifteen minutes muted. We'll do it.

Bryan Cantrill:

We'll we'll go fifteen minutes muted. Yeah. Alright. Okay. Greg Orion, am I pronouncing your name correctly?

Gregorien:

Yeah. That's correct. You can call me Greg. It'll be easier.

Bryan Cantrill:

Greg. Okay. That will be thank you. That that is a very polite no. You are not pronouncing that correctly, which I really appreciate.

Bryan Cantrill:

Thank you, Greg. It's really great to have you here. And we okay. So the intro to this, admittedly, like a couple weeks old at this point. But you know what?

Bryan Cantrill:

It's actually kinda helpful for us to to not be so hot on what the day on the internet that we kinda totally lose context.

Adam Leventhal:

Well, and you got to write a whole blog post

Bryan Cantrill:

about I it did get a whole blog post about it. Yeah, I did. How much were you Okay. Like, do you go to X still Twitter?

Adam Leventhal:

Like I'm maybe Greg.

Bryan Cantrill:

I'm talking to you right now. Greg, I actually know the answer to

Adam Leventhal:

Greg because No. I basically am not on Twitter.

Bryan Cantrill:

Yeah. I'm not posting to Twitter, but And I would love to not be there at all. But I I there I I am the the the cocaine addled mouse on I guess I feel like there's like just this quantity of news right now that is breaking on Twitter,

Adam Leventhal:

but maybe not. Oh, no. I I however you wanna justify it.

Bryan Cantrill:

I think it's perfectly justified. Thank you. This guy gets it. This guy oh, you could all stand to learn a thing or two from this guy. Yeah.

Bryan Cantrill:

And Greg, I know you are on Twitter because you and I or so you're not engaged. Like, Twitter DMs have become so much more user hostile or there is something basic that I am misconfiguring. Greg, maybe you can Twitter has

Adam Leventhal:

Yeah. Really gone

Gregorien:

Yeah. Yeah. And, basically, I think they are breaking the app every update.

Bryan Cantrill:

It does feel like they're breaking the app every update. And so in particular so you're missing out on, like so the VC think pieces, and I think, like, missing out.

Adam Leventhal:

I mean, I I have I mean, the the recent Palantir stuff has, like, permeated even to blue skies.

Bryan Cantrill:

So I'm I'm That one, felt like that broke the containment boundary. But there's so many, like, obvious so that one went went everywhere, but there's so much other, like, smaller stuff Yeah. That is and, like, I hate it, but I I I can't quit it. Yeah. You know what I mean?

Bryan Cantrill:

You do. It is actually I think we have a name for this called a bad habit, and it is a literal bad habit. I don't post there. You said that already. No.

Bryan Cantrill:

Know. I know. I I keep saying that to I've got it read it down. It's part of my day little about it. Part of my daily affirmation is that I don't post there.

Bryan Cantrill:

No. I don't post there. I bet I but I'm there. And so you're missing, like missing again in air quotes. But these all these, like, little stupid dramas.

Bryan Cantrill:

And this Gary Tan thing has been this like brewing drama. Have you have you met Gary Tan? No. Never. He's apparently he he's like he's he's quick on the block button as all the I mean, I feel like that's as befitting the investment class.

Bryan Cantrill:

They are the good ones aren't, but the bad ones are very quick on the block button. So I'm kind of like hoping to get blocked by him and all this. Do you think that's a reasonable aspiration?

Adam Leventhal:

That's the stretch goal for

Bryan Cantrill:

this podcast. That's stretch goal for this podcast is to get out, which is hard because we're not actually on Twitter. It's like a little hard to know if we're gonna get blocked there or not.

Adam Leventhal:

This does remind me of the podcast that had a public market CEO blowing up my DMs on Twitter after the podcast.

Bryan Cantrill:

That was impressive. I feel that was When are we gonna be able to What's the tell all?

Adam Leventhal:

That's well, we'll we'll we'll we'll tease that one. That's a that's just a teaser.

Bryan Cantrill:

Yeah. We're gonna go into go into Claude and ask about statute of limitations in various jurisdictions before we yeah. Okay. Fair.

Adam Leventhal:

Fair

Bryan Cantrill:

enough. But yeah. The the so you've been not following but Greg, you were obviously following this. And this was this kind of, like, slowly building thing where he had discovered LLMs and then would not shut up about it. So Gary Tan, who is currently the CEO of Y Combinator, previously known as, like, literally the tenth employee of Palantir.

Bryan Cantrill:

It's like his claim to fame now. I was like, okay. Alright. Not someone that I, anyway, Greg, what what was your you were obviously seeing Tan post about this stuff. And at some point, you yourself lost a bit of patience.

Bryan Cantrill:

Could you describe what you were seeing in terms of what what was was Tan kinda prattling on about, first of all?

Gregorien:

So to be honest, I don't follow him. So being the whole battle of VCs and pumping AI stocks all over the place is not my main thread. So I I usually get some drips of NVIDIA and other companies just announcing more absurd corporations. So when I saw Gary, it was like, okay. The the containment has breached into into lower lower decks of of of economy.

Gregorien:

And I thought it was it was like every other manager slash small company CEO that discovers AI that can write everything he wants and basically, he's a a personalized yes, man. But in case of Gary It's actually let's

Bryan Cantrill:

take that apart for a second, Greg, because I think you are I mean, this really is something that, Adam, you and are kinda sheltered from. But I mean, this has gotta be I mean, this is like Dun and Kruger. In terms of like every CEO is kind of I mean, not every CEO. There are many CEOs that are convinced like, I don't think these software engineers are working very hard. Yeah.

Bryan Cantrill:

I think I I think I I just I don't have the domain expertise. I bet I could do this in and out. And now I mean, Greg, we're now seeing that I mean, you you gotta think that like a lot because you're saying that, like, the CEO is now getting this kind of Fantasia software engineer. Yes, man. That is willing to like, really willing to grind out the hours.

Bryan Cantrill:

It's kind of a dangerous combination for like, really requires a very advanced immune system for a CEO to be able to resist the temptation.

Gregorien:

I was actually preparing for the podcast, and I was trying to find different effects or things that that could be related. And I found this Betty Crocker effect. Maybe maybe you know it better. But, basically, in the fifties, there was this cake mix that failed spectacularly until they reintroduced it to the market, adding a single step for users, for bakers, because basically, you had to crack an egg into the mix. And this totally changed the ownership or or or the idea of sorry.

Gregorien:

The idea

Adam Leventhal:

of or of, like, now you are cooking something.

Bryan Cantrill:

You are cooking. You did it. You cooked.

Adam Leventhal:

As opposed to, like, oh, it's just trash that somebody else made. But now it's like, oh, it's like a homemade meal. And you get to get the same thing with LLMs. Can really see that. That's a great analog.

Bryan Cantrill:

And Greg, you've also given Adam the gift of an image for this episode. So I mean, you've really given us, where did you come across that? That's really interesting, Greg.

Adam Leventhal:

The

Bryan Cantrill:

effect? Yeah. The the Betty Crocker effect. I'd not I'd not heard of that.

Gregorien:

Well, I was prepping with Claude for basketball. So yeah.

Adam Leventhal:

There will be a

Bryan Cantrill:

few such moments. Oh, delightful. I I mean, I hope to god it's a hallucination. I hope so much it's a hallucination.

Gregorien:

No. It's a real thing. It's a new we have something like IKEA effect. So with the flat box, you build your own furniture, and you feel more attached to it instead of buying a proper furniture, but deliver delivered in one piece. So yeah.

Gregorien:

It's it I I I'm 101% sure it's a real thing.

Adam Leventhal:

Yeah. That's awesome.

Bryan Cantrill:

When part because you put in your prompt to be a 101% sure of anything that you propose, Claude.

Gregorien:

Do not make mistakes.

Bryan Cantrill:

Do not make mistakes? Okay. So Interesting. So alright. So go on.

Bryan Cantrill:

So yeah. That you had this Betty Crocker effect that now CEOs are now yeah. They're convinced. No, I am doing the work. I'm sorry.

Bryan Cantrill:

Like Yeah. Who do you think clicked on this thing?

Adam Leventhal:

So it would be like these folks on social media who are like cross because people are making generated images using similar props that like Have you not seen this?

Bryan Cantrill:

I have not seen this. How do you wait. Okay. Oh. Mister, I'm not on Twitter.

Bryan Cantrill:

I'm too good for Twitter. I'm like you, mister dry drunk. Where are finding these? This is gonna be like a massive thing. I

Adam Leventhal:

I'm gonna answer that, but I did. I left my notes in in my car.

Bryan Cantrill:

I'll be

Adam Leventhal:

I'll be right back. Right. Right. She goes to a different school.

Bryan Cantrill:

Yeah. You you have no one. You would know her. Yeah. So, Greg, sorry.

Bryan Cantrill:

I cut you off there.

Gregorien:

Sorry. I I I'm not sure where to continue from.

Bryan Cantrill:

Yeah. Sorry. Sorry. We that we we we have this effect.

Bryan Cantrill:

Yeah. Sorry. So you found the Betty Crocker effect and your I mean, there there are other CEOs that are doing what Gary is what Tan is doing, but he seems to be, like, particularly annoying about it.

Gregorien:

Yeah. Basically, my whole action started when I found out what was it? He was some kind of teenager who commended under his main post about about bragging about lines of code is is a bit little silly, and, somehow, Gary took it very personally.

Bryan Cantrill:

So just to explain this to Adam, who is who who is unlike the rest of us, is is is clean. Is is Right. It is not among the unwashed. So this was a 17 year old who is literally like, look, I'm only 17. I don't know very much, but it seems isn't like lines of code a poor metric for software engineering?

Bryan Cantrill:

And he goes like open loop on the 17 year old.

Adam Leventhal:

That's amazing. What I wanna I wanna hire that 17 year old.

Bryan Cantrill:

Yeah. Exactly. I know. So and it was it was Greg, it was bad. That whole thing was a bad exchange.

Gregorien:

It was very tasteless. Yeah. He found the girl's project, but she I think she she she wrote a web browser engine from scratch in Rust and learned about c s seven, wrote a big write up about it. A very cool thing to read about how person how how a junior can learn about code and new technologies that they have never touched or interacted with. So that little single write up in her project was, I think, way more valuable than the whole GSAC.

Gregorien:

And that interaction was very, very, very quick and very unnecessary. So I I wrote that's kind of inappropriate and then decided, okay. So if he's doing the audits, let's do the same to

Bryan Cantrill:

Oh, that's interesting. Okay. So that that's what inspired you to go, like, let me go in and learn what he so in and so describe, first of all, the thing that he because he's built a couple of different things. One is he's got this Gstack thing, and then he's got Gary's list, I think. So describe kinda what you found as you went into it.

Gregorien:

So not everything he writes as public, Fortunately or unfortunately, I'm not sure. It will be interesting to read stuff. However, the Gstack and Gary's list are public, and he was basically, Gary was bragging about how many thousands of lines of code he's writing. Yeah. 37,000 per day across five projects.

Gregorien:

And he was he was keeping track of it in, like, a dual a Duolingo style in a seventy two day shipping streak with this kind of agent that listed him in terminal, how many projects he's shipping, how many comments per day, how many lines and thousands of code added. And, basically, the math showed me that every comment he's rewriting 40% of code.

Bryan Cantrill:

Oh my god. Well, is this you should mention Duolingo style too? Because I I mean, everyone that I know that's used Duolingo kinda complain. I mean, they have these long streaks and then realize like, I can't speak the language. I'm like, I've got a, you know, eight sixty five day streak and I was just in Mexico City and could not make myself understood on anything.

Bryan Cantrill:

I tried to buy an apple when I was at it or whatever. Yeah. The so it kind of an interesting analog in terms of, like, the emphasis very much on the streak. And the number of lines is crazy. I mean, in terms of tens of thousands of lines per day.

Bryan Cantrill:

Yeah. And I feel like we we we know so deeply that that is the wrong metric. And we've said this before that, like, any metric that you have, it's it's perilous to have these kinds of metrics.

Adam Leventhal:

Oh, so I imagine if instead of lines of code, we thought of ourselves as, like, a number of assembly instructions admitted or whatever. Totally. You know I mean? I feel like it's a similar kind of analog where you're like, you know, would you brag about, like, how much program text, like you have just checked in effectively? Like how many megabytes of

Bryan Cantrill:

Yeah, another 800,000 instructions in the bag. Yeah. Yeah, it's like, what was

Adam Leventhal:

that? Anyway, it's so sort of obviously perverse.

Bryan Cantrill:

That's right. Yeah. The actually points out in the chat, like the number of L three cash is trashed per day. It's like, it's a very strange one, which is actually very close to this like tokens burned per day metric. I mean, also that we, I mean, because we at Oxide are like Silicon Valley's couch because of it from the psychotherapy sense of people apply to Oxide when they're upset at their employer, right?

Bryan Cantrill:

And they describe why they're upset. And those of you in Silicon Valley, you should know that like the latency from you announcing some bogus new initiative to someone like washing ashore here at Oxide and explaining that there have been why they've been unhappiest and why. But there are a, I actually like there's one company that did a big layoff. A bunch of those folks that were laid off applied to Oxide. And a bunch of other folks that were not laid off applied to Oxide because the layoff was done so poorly and done so haphazardly.

Bryan Cantrill:

But the one thing we're hearing a lot of are people that are incentivized by how many tokens they use per day. It's like, is that I mean, at what point is a that's not a perverse incentive. Like, that's just like the incentive.

Adam Leventhal:

The wrong is it is so crazy. You're like, why don't I just wire these two LLMs together and have them talk to each other until I run out of tokens? Yes. I struggle

Bryan Cantrill:

with that. So yeah. I mean, Greg, this is definitely it feels like the the wrong metric in terms of of tens of thousands of lines per day. And and then the, like, as you say, like, the the gamification streak and see how many days in a row you can do it. And it's like, oh my god.

Bryan Cantrill:

Alright. So you went into this thing. You're like, okay. Let let me go. Let's see what is actually on Gary's list.

Gregorien:

Yeah. But I did one more step for that. I was was quite in a sussing mode, so I decided, you know what would be funny? If if Claude would do the audit, let's see how an AI works against Oh, yeah. AI generated code.

Gregorien:

How it how it does basically, code review on prod production output artifacts. So and mostly because I was tired, and I I really wasn't planning on spending a few hours reading someone's bad code. But, yeah, within one session yeah. But because Gary really liked to show how many sessions he had active, how many times he committed code, and so on. So I was interested in what would be the difference if you have an engineering's mindset and you say to an agent, like, in two or three sentences, this is what we are doing.

Gregorien:

It will be funny. Here's the saved output from the website. Here is the h a r file for networking. And let's figure out what can be stalled from the, basically, deliverables.

Bryan Cantrill:

Could you describe the the HAR file? The the because I had not have you used a HAR file before? Know. I know. I've

Adam Leventhal:

not used it. What is an HAR file?

Bryan Cantrill:

Yeah. I did Greg, do you wanna describe that? Because I actually learned about it from your post. I had not used one of these before.

Gregorien:

So I I've been working for years in web tech, and this is the first time I even used it. So it's a list of you could say a JSON file of all requests and responses, including all timings and headers and such. So it's a very, very nice format for LLM to just go through data and parse it and little stuff. It will make mistakes. Like, in my case, I had to correct it.

Gregorien:

Say, no. The total load the total load time for a website is not six minutes because it started at summing up values. But so many requests and so many different interesting things happening in that it was a nice read, I'd say. So the whole audit file has, like, 500 lines of things that are not working properly or misaligned or broken, and it went from pretty silly things like maybe I'm skipping too far ahead for your podcast, if if so, but from silly things like too many logos file being shipped or test files being attached to every single thing to pretty legally liable things like standards not being fulfilled for a registered organization in California. So yeah.

Gregorien:

Yeah. 500 lines of audit.

Bryan Cantrill:

Yeah. That's that's pretty interesting. Okay. So it did obviously found some real issues, and then so you you expand I mean and Adam, if you looked at the issues that it found, but the there are what Greg, there were like eight different versions of the logo, including a zero byte one. That would I mean, was just and then the one that's actually serving you is not the the the WebP.

Bryan Cantrill:

It's serving you this, like, bloated ping. Four megabyte ping. Got it. Do people say people say ping.

Adam Leventhal:

I think ping is right.

Bryan Cantrill:

I mean, get I get like my I know Sphig. People bust my balls for Sphig, but like but ping is fine.

Adam Leventhal:

I think ping is fine.

Bryan Cantrill:

Okay. Well, the I'm glad I can say ping. The and so I actually went Greg, when I first saw you just going open loop on him, I'm like, well, I'm gonna go on my phone to this thing. And the site was down. It just didn't work at all.

Bryan Cantrill:

And then finally, got to where I I don't know. Greg, do you think that was just because everybody was going to it at once because it was yeah. There was a a bit of a Streisand effect going on.

Gregorien:

Yeah. I think I think the site went down after, like, six hours. My pawn went live. So I went to when I wrote the thread and dropped it in the, I'd say, late night Europe time, I said, okay. I have a 100 views.

Gregorien:

What's what's going to happen? It's my usual views for post. So I went to sleep pretty calmly and happily. And when I woke up to the raging fires, I've called up on this website. The website app was working for, I think, three hours, and then it was just DDoS to that.

Gregorien:

And it lasted, I think, five hours, six hours. I remember Prime Prime Agent streaming, and he also was counting down if whether site loads at all. So, yeah, for

Bryan Cantrill:

a

Gregorien:

civic engagement website, it was a big outage.

Bryan Cantrill:

Yeah. Yeah. Absolutely. And so were you surprised by the by the reaction? I because as you say, like, a lot of people it it got a lot of turned a lot of heads.

Bryan Cantrill:

I think people were also like just were probably in the same category that I was in, which is like, okay, this guy's shooting his mouth off, kind of ignoring him. And look, I'm not taking this apart. Like, is this is just I'm sure this is garbage. So but it was very vindicating for someone else to take it apart and reveal that it is garbage. And because, Greg, there was there was definitely a lot of reaction to to your work.

Gregorien:

Yeah. I've never seen something so gross of us in in my case, in my years of contribution to to the Internet. But I think it just came out at the right time. So first, this whole anti AI movement Yeah. Against data centers and and and and such.

Gregorien:

I've been seeing in your country beyond the pond, but also so many so many jobs lost because of such mismanagement, I say. And when the screw is being tightened just enough, I think that was the moment my post by by chance showed up in in your feed, things went from there because it was I'd say factual. It it worked well enough to show the I'm just the right word for it, the emptiness of the promises they are trying to sell us.

Bryan Cantrill:

Yes. Yeah. I think that that's it. And I think that and I think that this is all on a trajectory of overshooting. I mean, I think we're gonna look back on this at this era.

Bryan Cantrill:

Well, I don't know. How do you think we're gonna look back on Saturday? Do think we're gonna look back at this as like the calm before the storm or when it just feels like

Adam Leventhal:

Yeah. It's it could go I mean, one could imagine that that these things get better. I think I was on a on a call with a bunch of alumni from my university and a bunch of professors talking about how AI should be incorporated into the curriculum. Oh, interesting. And some of these folks

Bryan Cantrill:

Are we ringing the chime for that? I feel like it's a Michael Littman.

Adam Leventhal:

Yeah, yeah. Michael Littman came. Yeah, for sure. And Michael Littman was on the call, but a bunch of folks were from like, the AI companies, Google and Anthropic and whatever. Oh my

Bryan Cantrill:

god. Okay.

Adam Leventhal:

Basically saying, like, humans writing code. Like, sure, that happens today, but, like, it will be swiftly a thing of the past. Really? And I was I was one of the lone voices of

Bryan Cantrill:

like Smoking helps your cue zone? What's the the Simpsons reference? The only thing I think Yeah. Right. Or whatever it

Adam Leventhal:

is. So I I think there is, I mean, certainly, folks betting hard on that, but other folks who I think are going to see this kind of, output and or or or the the that world looks like LMs get better, they're not just pooping out 37,000 lines of code a day. Yeah. But and with 7,000 test frameworks and pings of different types, but they're, but they're more, you know, optimization function or their, their reward function is a little more highly tuned. One can imagine that, right?

Adam Leventhal:

Can imagine even going to this code base and be like, can you do the same? But like, as shitty.

Bryan Cantrill:

Right. I mean, was kinda like Greg, you kinda did that. Mean, Greg was like, I had Claude review Claude's

Adam Leventhal:

other work. Right. Right. So it's it's it's sort of self aware, but who knows if Gary's Claude is gonna no. It's like, nope.

Adam Leventhal:

Looking good, boss. Like, everything is great. Like, spending those tokens.

Bryan Cantrill:

Yeah.

Adam Leventhal:

Or, know, does it get better? Or do do we realize, okay, we've we've Overshot. Overshot way too far and it's good for some things, great for some things. And for others, we we still need better guidance.

Bryan Cantrill:

Yeah. I and that's interesting that the folks in the AI companies are like, no, no, no, you're only smoking two packs a day.

Adam Leventhal:

No, no, no. Is it interesting? I mean,

Bryan Cantrill:

it's sort of It feels a little interesting.

Adam Leventhal:

There was a new grad, like a class of 2025 or 2024 on that call working for one of the AI companies. Really? And she said she writes no code by hand. Wow. And I guess that's fine.

Adam Leventhal:

I guess free to be you and me. And, I just worry. It makes me worried. It makes me worried because I've seen what kind of code it emits. Yeah.

Adam Leventhal:

And sometimes it's fine and sometimes it's not. And it's very contextual.

Bryan Cantrill:

I also just feel that we see, I mean, this is the history of technology is a history of overshoot. Yeah. You know, my mother was in town and she has named her walker Maybell. Okay. After the cable car, Maybell the cable car.

Bryan Cantrill:

Okay. No. Have you Virginia Burton? Nope. Virginia Lee Burton wrote these amazing children's books about Mike Mulligan and the steam shovel.

Adam Leventhal:

Mike Mulligan, I'm all in.

Bryan Cantrill:

Yes. Mike Mulligan. I know you could see, I could see it was in the Little House. Yeah. Yeah.

Bryan Cantrill:

Yeah. So The Little House, Mike Mulligan and Steam Shovel. And I mean, he was one like Virginia Lee Burton to go write a 100 of these and then about Made by the Cable Car. And it is all about people adjusting to technological change that and technological change and having like really mixed feelings about it. Like, we need a Virginia Lee Burton and maybe like we can go get caught on this.

Bryan Cantrill:

Like, what is a Virginia Lee Burton? What is Mike Mulligan and his steam shovel in the LLM era? This is like Mike Mulligan and his handwritten SQL query. I'm not sure what the you know, I'm not sure what what what the actual, but all of these are about the the the like the very, and Katie, snowplow. Thank you.

Bryan Cantrill:

Yeah. That's the and all of these things are about how people are I mean, and the the I mean, the books all have a narrative where the nostalgia has real value. Mhmm. But as you remember, like Mike I mean, the steam shovel, ends up being a furnace. Yes.

Bryan Cantrill:

I mean, it's a little dark. I mean and, I mean, maybe all the cable car ends up being saved by citizen action and the the the cable car lives on today, albeit only as a tourist attraction. So maybe that's me. Maybe we have the tourist attraction where people write code by hand.

Adam Leventhal:

In the same way that like when we go to the computer history museum and we've got like the punch card emitting oxide on paper, we would audit. And somewhere in the back, there's probably someone saying like, that was my livelihood. You jerks. That's not a tourist attraction.

Bryan Cantrill:

Exactly. My culture is not your costume. That's

Gregorien:

the

Bryan Cantrill:

it's That's I think what we're kind of in the throes of, but in also in all of those, the technology did overshoot. Like I think everyone I mean, I think many people decry ripping up of light rail, certainly in the East Bay where you and I live. The key line was ripped up. And it was do you know I mean, I feel like there's like a time in your life when you think like, okay, that's a bit of a conspiracy theory. Really, the bus companies, the rubber companies and the gasoline companies were conspiring to kill the light rail, but that's exactly what happened.

Bryan Cantrill:

Well, I just

Adam Leventhal:

believed it. Yeah. Just thought. That's right. Know it's even easier.

Adam Leventhal:

I never questioned it.

Bryan Cantrill:

Even, it's like it's the bell curve meme. It's like I like the you know, it turns out like yeah. You were that was the right thing to go do. It's just to believe it because they that is what happened. And like to our to our detriment, and then, I mean, things kinda swing back.

Bryan Cantrill:

And I you know, I kind of feel that like the this overshoot is endemic in technology. And we overshoot all the time

Adam Leventhal:

on everything. Like a Java. Java was a big time overshot. I mean, not just at Sun, right? But like including, you know, changing the name to Java and

Bryan Cantrill:

I mean, we just have to bring up the fact that like, again, we the cafe was Java Java. That was the name of the cafe. Yes. It's like the Java Java is the name. Like, you can't come up with an even like better Java so dominates the discourse that it has to be both words And in the cafe

Adam Leventhal:

changing the stock ticker to Java.

Bryan Cantrill:

Oh, God. Oh, still painful.

Adam Leventhal:

Just smelled so desperate.

Bryan Cantrill:

It's so desperate.

Adam Leventhal:

There were things that had no Java code in them that were renamed. Like the desktop system was like the Java desktop. We're like, isn't that

Bryan Cantrill:

That's right. Oh, yeah. Right. But But nothing of, yeah, Java. But yes, it's some, but

Adam Leventhal:

then even industry wide, there was like, there's a Blu ray player that was like all embedded Java. Anyway, another example of overshoot.

Bryan Cantrill:

Another example of overshoot.

Adam Leventhal:

And it's like, yes, Java is fine today, and people still use Java today. Yes. It's just not in all the ways that people envisioned it, as in its maximalist

Bryan Cantrill:

In fact, I almost wonder like, when has there not been overshoot? Like when has something not overshot the mark? I mean, I think it's like, it's so endemic that like you And it was also like, if you didn't overshot the mark, were you trying hard Like, maybe you should have overshot the mark. I almost feel like it it it's that kind of like baked in Yeah. On on that overshoot.

Bryan Cantrill:

And it just feels like, okay. But you when are you in to an overshoot kind of phase? And it feels like gut says, we are getting into an overshoot phase. Not that like, this is like, again, indisputably useful. Yeah.

Bryan Cantrill:

But the, you know, and Tan's entry or Tan's work is kind of a concrete example of this. So I alright. So then I I was inspired to like, alright. I gotta like, what what is kind of my problem with this? Yeah.

Bryan Cantrill:

And to a certain degree, was just like not actually ruminating on how to do things simpler.

Adam Leventhal:

You mean, the problem with Gary is like, I'm writing 30,000 lines of code a day. It's missing. Because clearly there's something.

Bryan Cantrill:

Clearly, this is like, there's a mouth feel that isn't right here. Yeah, exactly. And that's what sent me into the peril of laziness lost

Adam Leventhal:

Yeah, it's a great blog blog post. And you gave me a little teaser and I did not remember Larry Walls' The Yeah. Three Even though you said it's in the Camel book. Right?

Bryan Cantrill:

It's in the Camel book.

Adam Leventhal:

But, like, the Camel book I don't know if I've shared this. The Camel book was, my bible when I was, like, in middle school or high school. Like, I fucking loved that book. I had multiple editions of that book for like Pearl three and Pearl four.

Bryan Cantrill:

Really? Yeah. Pearl three, I feel like that's OG, isn't it? I mean, Pearl three is I don't know. Anyway, it was it was like So you you had you had the camel book on your lap.

Bryan Cantrill:

You've got The Love Boat is coming on later. You just watched at night.

Adam Leventhal:

I was on IRC, geeking out because one of the other authors of the book, not Larry, responded to me on IRC. I was like, oh my god. Or it might have been Usenet or IRC. I don't remember which one. Either way.

Adam Leventhal:

But, yeah, but I love that book and I don't remember I didn't remember at all the the three virtues.

Bryan Cantrill:

Yeah. That the and the the three virtues, I can't remember when I first learned about them, but they kinda they they they're they're a reference that, you know, people make reference to them. So the the the three, virtues are laziness, impatience, and and hubris. The Cliff made you know, hubris our hubris Yes. Is I should ask Cliff, but it is certainly an implicit, if not explicit reference to the three virtues.

Bryan Cantrill:

So the, hubris I need to go actually and I should have done this before the podcast episode. Hubris also used to mention, impatience, and laziness. So it was like starting laziness and like they there were the the, the initial, like, boot messages for hubris. It's all been sourced. We should be able to go determine when we had when we decided that we were that hubris was too grown up for laziness and impatience.

Bryan Cantrill:

At some point, we removed the virtues. Well, the virtues are just like not quite, you know, like, people like starting laziness. Like, does that what what does that mean? I mean, Littering Wall really forms a real narrative arc for you, Adam, because you have these early days of Pearl. That's right.

Bryan Cantrill:

How about You

Adam Leventhal:

you you yes. Have I talked about meeting him? I on this I don't know that I have.

Bryan Cantrill:

I feel like this you know, this is so great to have this role reversal. It's so helpful for me where I am convinced I am saying something that you don't know and I'm saying it for the fourth time. I mean, yes, think we've surely talked about it here, but if not, I think

Adam Leventhal:

you should. There we go. Well, anyway, so Larry was at a conference I was at in Amsterdam. I think it was one of the first Oscars in Europe, maybe in 2004, that kind of neighborhood. Larry was there.

Adam Leventhal:

I was giving a D Trace talk. He was giving like a a Pearl themed poetry slam, like concurrently.

Bryan Cantrill:

Oh, good. Now was this the Pearl sex era? Was he was he doing an out reading from an apocalypse?

Adam Leventhal:

No. This was actual poetry. Like, I'm not This is not like pretend poetry. This was actually Pearl, the language themed poetry.

Bryan Cantrill:

Okay. But you you can you hear yourself how this sounds like the recollection of a dream.

Adam Leventhal:

Sounds like sounds like something I remember for a dream. Yes. I'll go to go find Pearl.

Bryan Cantrill:

I mean, you're in Europe.

Adam Leventhal:

Yes.

Bryan Cantrill:

Larry Wall was there. Reading Pearl poetry. Amsterdam.

Adam Leventhal:

Exactly. Yeah. Exactly. No. No.

Adam Leventhal:

So and so they were going we I was head to head with them. I did have one attendee at my talk. Everyone Head to head? Yes. Did you

Bryan Cantrill:

have one attendee? One. Yeah. Did you have only one attendee?

Adam Leventhal:

Yes. I had only one attendee at my D Trace talk at OsteCon You 2,000

Bryan Cantrill:

and had hand on heart, only one? I had

Adam Leventhal:

one real person. Okay. And then I had two Sun Marketing people.

Bryan Cantrill:

Okay. Okay. So you

Adam Leventhal:

had three. I mean, I'm only counting one because the others No, no, no,

Bryan Cantrill:

should only count one. No, I'm not. The reason I Have we talked about my presentation with only one person here?

Adam Leventhal:

I don't think so. Yeah. I don't think I might not even know this one.

Bryan Cantrill:

Okay. Now you're I I appreciate appreciate you, encouraging and patronizing me. But the okay. So no. This is STS two thousand.

Bryan Cantrill:

This is the very worst conference idea of all time where they were organizing the Sun Support Symposium in 2000. Okay. And one of the things that the organizers had heard is that people didn't like to choose between talks. So they asked like, okay, we've got a solution to that. We're gonna ask every presenter to give their talk back to back to back.

Bryan Cantrill:

Oh, yeah. Okay, you've heard that. Yes. Yeah, all right. And so the And this was in Denver my mother comes back.

Adam Leventhal:

I do remember your mom.

Bryan Cantrill:

Your mom making a second cameo.

Adam Leventhal:

Twice. Twice in one box.

Bryan Cantrill:

Twice in one but pre Maybell. Pre Maybell. So the my my my mom was in and was gonna wanted to come watch me speak. And there's only one other person that shows up for my talk. Yeah.

Bryan Cantrill:

And I tried to tell him, come well, good news. I'm gonna give this talk in like forty five minutes. Go to the other talk that you wanna see now and come back to this one later. He's like, yeah, don't wanna do that. I'm like, okay, I guess I'm gonna give this talk to just you.

Bryan Cantrill:

Yeah. And then he started like spacing out. And he started like looking at the ceiling or whatever. I mean, because but you had that same must have that same experience where you're like, there's only one non son person in this room.

Adam Leventhal:

You're like, so if there are any questions If there

Bryan Cantrill:

are any questions, like, obviously stop me. And also, your response okay. Look. I get this is an important message. If you yourself are an audience member in a one person talk, your responsibilities are different, I feel.

Bryan Cantrill:

I think that's right. You can't space out. You're in like you've asked someone a question that you've insisted that they answer that has a forty five minute answer. And you can't space out. Like you just can't.

Adam Leventhal:

You gotta be ask

Bryan Cantrill:

the question. You ask the question. You gotta be locked in. So how So you're giving a Well,

Adam Leventhal:

that was actually just beside the point actually that the Pearl poetry slam out. But recall, Larry Wall's my hero from like middle school and high school. Yes. He's my girlfriend at the time was actually teaching his son in high school. Right.

Adam Leventhal:

So I go, I sit down next to Larry and I'm like, Oh, hey, great to meet you, been a big fan. My girlfriend teaches your son. And he was like, Oh, where does she teach? Which I thought was a strange question.

Bryan Cantrill:

Yeah. It's a super strange question. You feel like you've given There's been enough information given to be able to Well, mean, it's like, well, I mean, I I Look, I've got 17 sons.

Adam Leventhal:

That's right.

Bryan Cantrill:

You gotta need more. You

Adam Leventhal:

never know.

Bryan Cantrill:

Right. Yeah.

Adam Leventhal:

So I say where where she's teaching and he says, oh, my son goes there. I'm like, okay. Uh-huh.

Bryan Cantrill:

Oh, yeah. Okay.

Adam Leventhal:

Lost his connection. Asked what she teaches, which I thought was another strange one because there are only three teachers at this particular, like, kind of magnet school or whatever. And then we started back at the beginning and we were around in that loop a couple of times. And then I excused myself and I talked to Alan Burleson from Sun who was actually responsible at Sun for integrating like Pearl stuff. I knew Larry pretty well.

Adam Leventhal:

I asked if Larry was just fucking with me and Alan insisted that he wasn't. And then I sat at lunch that day by myself and thought about how I had the wrong heroes when I was in high school.

Bryan Cantrill:

Yeah. Never meet your heroes.

Adam Leventhal:

Yeah, exactly.

Bryan Cantrill:

I feel that that is often true. Yeah. I feel that heroes are often. Yeah. Are are often

Adam Leventhal:

I mean, it's like that look of disappointment that you must get when when you meet young people.

Bryan Cantrill:

I get that disappointment a I really do. And I mean, I tell them like, if you listen to the podcast a little more closely, you would know that I am actually advising you not to

Adam Leventhal:

meet me. That's right.

Bryan Cantrill:

But this is usually when they ask where you are. There was like, Adam around. Is Adam around? I just like, where's where's Adam? What's

Adam Leventhal:

his Where's Robin? Is it just Batman?

Bryan Cantrill:

It's Batman. It's kinda disappointing. It's Robin. It's like, I didn't realize how really they should change the ordering. This is it should be a more of a Robin and Batman kind of a show.

Bryan Cantrill:

Yeah. No. The okay. When did what tech heroes I think we will not retell for presumably the third, fourth, or fifth time us chasing Dennis Rachino to a bathroom. I know we I I I know That's right.

Adam Leventhal:

We've done it at least twice.

Bryan Cantrill:

I know I know there's someone on their who's got that on their bigger card right now. And, like, I feel you should be able to cross off on the bigger cards for making passing reference to it, but we are not gonna, we won't talk about chasing Dennis Ritchie to the bathroom. What of your heroes have you met that have not been disappointing? If any.

Adam Leventhal:

I'm gonna give sort of partial credit on this one. I went to Austin for a sales trip. Yeah. That was only 50% because, I was gonna sit down with, Kevin Millar who played for the Red Sox. Like, I was gonna have lunch with him.

Adam Leventhal:

Oh, okay. Oh.

Bryan Cantrill:

Yeah. Okay. Yeah.

Adam Leventhal:

And he was like, it was not the worst. Like I still thought he was pretty neat, but he was definitely like more racist and spat more like inside than I thought was appropriate for a grown man at like a nice lunch club.

Bryan Cantrill:

Excellent. Forgetting like old conventional wisdom, get blocked by Gary Tan. New conventional wisdom, get a cease and desist from Kevin Millard.

Adam Leventhal:

Yeah. Actually that that's super likely. He signed a couple of baseballs, he was very generous and told some nice stories. So kind a mix. Yeah.

Adam Leventhal:

A little

Bryan Cantrill:

bit of a push, I guess. Guess this was the positives that I getting out of there.

Adam Leventhal:

Yeah.

Bryan Cantrill:

The okay. So I would say got or I got two for you. Yeah. Sticking to tech heroes for the moment. Yeah.

Bryan Cantrill:

Although, have we talked on this podcast about me meeting Shyam and Aya? I mean, I know you and talked

Adam Leventhal:

Oh, about no. Don't think so.

Bryan Cantrill:

Okay. So Shyam and Aya, we pitcher for the A's. Yes. And I made a a though and sorry, Greg. We're going into very strange territory here.

Bryan Cantrill:

But in terms of on meeting your heroes, I was I was standing in a security line behind Sean Manaya, in airport security line. And I'm like, this guy is now plays for the Mets, I think. And I'm like, oh my god, is Sean Manaya. And I'm like, I really I'm like, I'm right behind him. We're in a security line.

Bryan Cantrill:

There's nothing else to do. I'm very mindful of other people's time, but I also feel like I gotta shoot my shot here. And I just should have thought about it a little bit longer. Just

Adam Leventhal:

rehearse it quietly to

Bryan Cantrill:

yourself? I should rehearse it quietly to myself. And I what I should have done and I didn't. And the and in particular, a colleague of ours was a huge Steve Young fan. Mhmm.

Bryan Cantrill:

And do you remember him telling us the story? We put ultimate with him. No. Where he was he went to the airport and Steve Young was in baggage claim in San Francisco waiting for a bag. At like peak Steve Young.

Bryan Cantrill:

Yeah. And nobody is talking to him because they all It's like, Oh shit, it's Steve Young. And so there's like a Bubble. There's a bubble. There's like a 20 foot bubble around him.

Bryan Cantrill:

And he's like, Tony's wife, Oh my God, it's Steve Young. And she's like, You gotta go talk to him. He's like, I'm not gonna go talk to him. I'm like, What? It's like, No, come on, dude.

Bryan Cantrill:

Like, you're a huge Steve Young fan. Like you should go talk to him. And he pulled a really good move. And he's like, hey, sorry. I'm sorry to bother you.

Bryan Cantrill:

Do I recognize you from Superboy? So apparently, Steve Young had like a cameo in Superboy that only a super fan would know. And he started laughing and that's a veteran move.

Adam Leventhal:

There you go.

Bryan Cantrill:

So that's a veteran.

Adam Leventhal:

So this was not you. This was not me.

Bryan Cantrill:

This was not me. And the first thought that came to my head was the last time I'd seen him pitch, which was the 2019 wild card. And in that 2019 wild card, I stuck with VCs. I missed the first pitch, but only the first pitch of that game. In the first pitch, Shalmania gave up a jack.

Bryan Cantrill:

Gave up a solo home run. The A's never had the lead. That was the end of that season. And it's basically like the it was like 53,000 people or whatever in the coliseum.

Adam Leventhal:

It was

Bryan Cantrill:

like the last the last positive.

Adam Leventhal:

So like this traumatic moment.

Bryan Cantrill:

This totally traumatic moment for me. It's a traumatic moment for him.

Adam Leventhal:

Yeah, for sure.

Bryan Cantrill:

And he's kinda like, oh, like, oh, yeah. Right. I'm like, woah, yeah. Remember that game. And he's like, oh, yeah.

Bryan Cantrill:

Like, I remember that game. We're kinda like I

Adam Leventhal:

was trying to forget it. I've been going through

Bryan Cantrill:

a lot of therapy on that one. Exactly. I I try every day to to to forget it, and it's still not working. And so the and and as we are as we're talking and as and I'm like asking him dumbass questions about like, oh, you know, what was it like to pitch in the post season? It's like he got bumped out of the post season and he had hit up one post season start and that was it.

Bryan Cantrill:

He's like And he's kind of giving me like, oh, like it was a real honor or whatever. Could see he's just like And then like I finally like my brain was kind of like finally catching up and I'm like, asshole, you were at the Sean Manaya no hitter with Adam. Yeah. And I'm like, oh, also, I was at your no hitter, which is like a much more obscure thing at some level. Right.

Bryan Cantrill:

Because they were not 53,000 fans or whatever. They they were like, whatever it was, 18,000 fewer.

Adam Leventhal:

Most of them were Red Sox fans.

Bryan Cantrill:

Most of them were Red Sox fans. Exactly. And he was I mean, like, immediate disposition change. He's like, oh, yeah. You were at that game?

Bryan Cantrill:

Like, I was at that game. He was like, oh, hell yeah. And we're like, oh, okay. Oh, God. I'm like, so like diving save and that he is a I mean, he is a great guy.

Bryan Cantrill:

So I I really and at the end of the conversation, he's like, puts his hand out like, I'm Sean. I'm like, yeah, we know you're Sean. Like, I know that. I'm Yeah. We know you're Sean.

Bryan Cantrill:

So the but I I so I wanna give I wanna give Sean Mania his proper. There you the Video hero. Yeah. Exactly. Really great guy.

Bryan Cantrill:

Tech heroes. Yeah. I would say Andy Bechelsheim. Yeah. Delivered.

Bryan Cantrill:

Nolan Bushnell.

Adam Leventhal:

I don't know who is I don't know Nolan.

Bryan Cantrill:

Nolan Bushnell, founder of Atari. Oh, cool. He's a he's he was amazing person. I really enjoyed both those guys. I have met many more people that were disappointing.

Bryan Cantrill:

So and I don't I feel like we don't need to rattle those off.

Adam Leventhal:

But Okay. Good.

Bryan Cantrill:

I would say I would say that that that many people are just actually as it turns out, meet in person. And next thing you know, you're chasing Dennis Ritchie into the bathroom. Where are we? Back to LLMs.

Adam Leventhal:

Yes.

Bryan Cantrill:

Getting back to I get Larry Wall. That's how we got here.

Adam Leventhal:

Larry Wall. Yes. Meeting here is And your lovely blog post, which I thought I thought was great. And in in which you focus on laziness in particular.

Bryan Cantrill:

Laziness and when in particular, like because the other thing that, like alright. What do you what to you is the highest praise for code? What is the word? I think elegant, but

Adam Leventhal:

simple Yes. Jackpot. Okay. There we go.

Bryan Cantrill:

No. No. No. No. No.

Bryan Cantrill:

Got it in one. Like, that was what I had written over in my,

Adam Leventhal:

like Yeah.

Bryan Cantrill:

My card as well. Elegant. Yeah. Yeah.

Adam Leventhal:

And actually, that that it's it's funny.

Bryan Cantrill:

It is do you think that because we're kinda coming from the same folkway or is that because I don't think so.

Adam Leventhal:

I think a little bit of the kinds of things that we've built too. Because there's, you know, there's certain kinds of things. Like if you're writing a web app, and it's just like gonna have a bunch of business logic

Bryan Cantrill:

Yeah.

Adam Leventhal:

Some of that shit's just never gonna be elegant.

Bryan Cantrill:

There isn't there isn't not gonna be the word.

Adam Leventhal:

There isn't some like unifying, like, insight where you're like suddenly, I can do all

Bryan Cantrill:

No, no, seriously, like, not yet.

Adam Leventhal:

Maybe, but sometimes these things are just like, it's just turning the crank. And we've all done these kinds of activities where you're like, there might be better ways or worse ways. There's not actually a good way. All the ways are gross. I think that is the beauty that I think we appreciated certainly in that Solaris Kernel group where we both kind of grew up, where everyone appreciated when you're like, No, no, no, that is the right way.

Adam Leventhal:

Like you have found the elegant way of doing it.

Bryan Cantrill:

Found the elegant way of doing it. You've deleted a bunch of code. You've deleted a bunch of redundant. And I feel like, I mean, we got a chance to see that a couple of times. I feel we got a chance to participate in that couple of times.

Bryan Cantrill:

But I I still feel that, like, elegant is the highest praise. And I think, like, elegant is not something that I mean, the the in the systems, I would actually say moreover, the systems that are most important that we build on are actually elegant.

Adam Leventhal:

Yeah. And often, it's where simplicity and performance matters. I was talking to someone once about they were writing some, you know, very high performance, storage system, and they only had, like, so many, you know, nanoseconds to compute, like, to to do some transaction or whatever. Yeah. Yeah.

Adam Leventhal:

So I was like, oh, the well, you must not have written much code then. Right? Because like tautologically, you can't execute that by the instructions. So Right. And and Right.

Bryan Cantrill:

If you measure your output by instructions per day, as we do.

Adam Leventhal:

That's right. Yeah. Or or even lines of code because there, like, it's not how 30,000 lines of code, but it's it's the thirty thousand hours that go into, how can I do this with eight instructions or 12 instructions or whatever? Totally. And it it kind of prioritize that elegance when you are building foundation, when you're building things that are performance critical and observable and all those things.

Bryan Cantrill:

Totally. No, I think that's actually a good point. So you think about the things that you do handwrite and assembly or your instructions per day output is low.

Adam Leventhal:

Yeah, exactly.

Bryan Cantrill:

But it's because it's extremely high consequence. And you're doing it that way for a reason. And you wonder maybe like that will be like the future will be those things that are because I mean, I think you and I both believe that this idea that, like, the software the software generated by humans is simply going away Yeah. And that we're only gonna write prompts is ridiculous. Agreed.

Adam Leventhal:

Or at least certainly not. Yeah. Mean, that the that that that that of absolute is ridiculous. I mean, I think there are there are certainly some kinds of things, and, you know, both experiences, I think, where these kinds of grindy, there isn't a great way. There's just going to be a bunch of code.

Adam Leventhal:

It's like, yeah, go write the code for me. Fine.

Bryan Cantrill:

But

Adam Leventhal:

not have the elegant insight for me.

Bryan Cantrill:

Okay. So here's also what I wonder is, do you I mean, in an ideal world, I am right now reminding myself of how I was thinking of the Oracle acquisition of Sun.

Adam Leventhal:

Okay.

Bryan Cantrill:

Where we get Sun's technical excellence and Oracle's business acumen, right? Which is just like totally wrong.

Adam Leventhal:

So I I not to not to have a have a diversion or diversion, but do you remember when when Sun was gonna buy Apple? I do remember when

Bryan Cantrill:

Sun was yeah. Yes.

Adam Leventhal:

And everyone's like, it's gonna be like Mac OS nine with the With the CD. Yeah. Absolutely. No. Absolutely.

Bryan Cantrill:

No. We would like we we what you want is actually like the the their user interface on our engine. And what we're gonna get is our user interface on their shitty engine. No. Totally.

Bryan Cantrill:

That's right. Yeah. Absolutely. No. I I think that the and people actually forget that that this is guile amelio days.

Adam Leventhal:

Yeah. This is like before before jobs came back, before the iPod. Yes. When, yeah, Apple was struggling.

Bryan Cantrill:

Apple Apple someone asked McNealy about Apple in an all hands, and he said, I think they've got great office space. Wow. Isn't that amazing? Wow. Nice.

Bryan Cantrill:

I know you think of like Nice.

Adam Leventhal:

Tell that to Zuckerberg.

Bryan Cantrill:

Yeah, exactly. Like, eyewitness to history. But okay. So in like and this is an overly optimistic world. But like take I'm I'm actually gonna kinda connect all the dots here.

Bryan Cantrill:

Let's take Torvald's talking about not eliminating P trace. Mhmm. An inelegant abstraction. Right. Replaced with an elegant abstraction slash proc by the late Roger Falcon.

Bryan Cantrill:

Absolutely. The part of the reason that that part of of the the reason you weren't gonna do that is because of the actual just like manual labor involved. Do you remember the KMDB? No, not KMDB. The KRTLD bet I had with Mike back in

Adam Leventhal:

the day. Do you remember this? Was it like cough executables or something? No. No.

Adam Leventhal:

No.

Bryan Cantrill:

No. Very reasonable, but no. Like, I'm sure that was like a I'm sure that was like Parlay. The next day's lunch conversation. No.

Bryan Cantrill:

This was on the staying power of KR TLD. Okay. KR TLD is the linker in the kernel. Yes. For like kernel modules.

Bryan Cantrill:

For kernel modules. It is exactly as you described in terms of like, this is not

Adam Leventhal:

elegant. There's a little shit to do.

Bryan Cantrill:

There's bunch of shit to do. You gotta link this thing. Yeah. Someone's gotta link it and yeah, you gotta handle like Unix and Gen Unix differently than you had. And then the Unix, Gen Unix divide is itself historical.

Bryan Cantrill:

There's a bunch of garbage you need to go do. Yeah. And Mike at the time was like, this will be rewritten. And I'm like, it won't be rewritten. Yeah.

Bryan Cantrill:

Because it's really important. It's important that it function correctly. It does function correctly. Its problem right now is in elegance. And it is too much work.

Bryan Cantrill:

It just will never pay for itself economically. Yeah. And in like the the most optimistic scenario, you could and remember do you remember that he and I have like a ten year bet? That in like 2006, that could have been a twenty year bet. Yeah.

Bryan Cantrill:

Because we still I mean, like right now in Helios, we have like the linker is basically Probably unchanged. Unchanged. Right. Unchanged in the last twenty years. And but I'm actually less confident about the I mean, I think with the CRTLV, feel pretty confident.

Bryan Cantrill:

But you could say like, well, no, actually now a lot of the heavy lifting, okay, the human has a better Elgin abstraction and the LLM can backfill on some of the heavy lifting. Is the most optimistic. Are we gonna

Adam Leventhal:

look at stuff? It's interesting because I see that it's it's kind of a double edged sword because I think what you're talk or the or a more general expression of this is like my backlog, like the shit that I was never gonna do. Right? The stuff that's like so far on the backlog that it's like, yeah, we keep that bug, but like maybe if Stalebot closed that out, wouldn't even be mad. That's right.

Adam Leventhal:

But like suddenly that, you're like, well, why not? Let's go do that thing. Yeah. And like sometimes that's a good idea because sometimes it's like, well, involves risk to some degree and everything requires some domain expertise to go evaluate. So it's like, I think that it reduces the upfront cost while maybe not changing some of the review and testing and longevity costs.

Adam Leventhal:

And I think we have not fully embraced how that kind of shifting in costs from generation to review and testing and so forth should be regarded in terms of how we prioritize things. So this like, for this care TLT example, it's like, okay. Now it's economic it's free to go change arbitrarily to make the changes, but then how we assess that? And somebody's gonna review it. You know who it's gonna be?

Adam Leventhal:

Robert Bastocki. And it's like the most overburdened person at Oxide.

Bryan Cantrill:

Right. Right. Right. And you would wanna make sure right. No.

Bryan Cantrill:

That that that change still needs to economically pay for itself. It still has cost. Yes. But the cost is potentially quite a bit lower.

Adam Leventhal:

Yeah. For sure. For sure, it changes the cost dramatically. Yeah. But it also moves like it also moves the cost.

Adam Leventhal:

So if you just say, Okay, LLMs go burn down my whole backlog, then it creates so much burden on the reviewers and so forth. But sometimes it's hard to recall the total the fully burdened cost of these things and the the people who they fall to.

Bryan Cantrill:

Yeah. Well, and it's certainly and I think what we are, I think, all awaiting to a degree is what are the consequences when we actually get these systems kinda in production? And Greg, you kinda help, like, hit on one of

Adam Leventhal:

these

Bryan Cantrill:

consequences with Gary Tan's work of like, yeah, it's kind of garbage. And But I think we are all kind of waiting for what I mean, AWS, I don't think this is betraying your confidence. AWS has already had outages that were because of That are being attributed internally to use of LM, inappropriate use of LMs. Yeah. And if that, like, the kind of the accelerationist approach of the people on your call would say that, no, no, this is like, that's because like the models are insufficient.

Bryan Cantrill:

Right? But I there's also a reason to say, like, no, no. This is, a consequence of overshoot. You're actually into overshoot right now.

Adam Leventhal:

Yeah. And and certainly, like, when I'm going to do some, like, big change. Right? Like, oh, I'll give you an example, Bryan. You may need your postmortem lawyer for this one.

Bryan Cantrill:

Oh, hold on. So there was a there was a Okay. But you you you're postmortem lawyer. Are you are you saying

Adam Leventhal:

I'm gonna recuse myself on this. You're

Bryan Cantrill:

so clicked it out.

Adam Leventhal:

Shit. Yeah. So there was a change you made impact. There was a a issue with ZFS years and years ago Uh-oh. Where, I think you basically saw a lock ordering problem where, readers could start start writers.

Bryan Cantrill:

I don't

Adam Leventhal:

even remember the details of the original problem that you saw, but the way you fixed it was in a macro.

Bryan Cantrill:

Said this,

Adam Leventhal:

I'm gonna change this flag that says, I'm taking a write lock. Yes. No, pardon me, that I'm taking a read lock to I'm taking a read lock that also starves out writers.

Bryan Cantrill:

That's

Adam Leventhal:

right. Because normally if a writer comes along, writer goes to the front of the line figuring, oh, you wanna modify something that's probably more important than all these directs just trying to read it.

Bryan Cantrill:

That's right.

Adam Leventhal:

That's the way of inverting it.

Bryan Cantrill:

Yeah. We found a Well, and and so to to be clear, like the just to play my own I'm I'm gonna be representing

Adam Leventhal:

Pro set. I'm I'm representing myself today.

Bryan Cantrill:

Yeah. So I'd like to put myself on the stand. The That was a consequence of a production issue. Yes. Where readers were getting completely starved.

Bryan Cantrill:

Right. Right. Right. The solution though is the solution I will be the first to admit. I Worked.

Bryan Cantrill:

That's right. There you go. My lawyer is here. The the it it kind of solved the immediate problem, but it obviously flips the disposition now. Like you you you haven't actually fundamentally solved the fairness problem.

Adam Leventhal:

That's right. That means and worse, maybe this is

Bryan Cantrill:

your point. It like definitely is misleading in

Adam Leventhal:

terms of That's right. That that was actually my point on this one was actually the way that you did it was like this the like eight line fix.

Bryan Cantrill:

It fixed it.

Adam Leventhal:

Yeah. I think Fewest possible lines. When

Bryan Cantrill:

Wait. Was I too lazy in this one.

Adam Leventhal:

ICSI is here in chat and and we both got confused because we're like, how is this starving out the writers? Because it's just a reader lock. Right. And I think the way we would do that in an LLM world is turn it loose. On all the use cases.

Bryan Cantrill:

All these

Adam Leventhal:

cases. Go find it everywhere. Go fix it in a way that is more visible. And presumably the reason you can do that is like

Bryan Cantrill:

Not trying to create. Yeah, absolutely.

Adam Leventhal:

And you're like, how do I write an Aux script that is context aware enough Yeah. Yeah. Yeah. To only do this in the right places and then But but

Bryan Cantrill:

I I probably could have reviewed a delta of it.

Adam Leventhal:

Yeah. Yeah. Yeah. You I mean

Bryan Cantrill:

If if I had said no. That's a this is a really interesting case where had I put a like a well constructed enough quad on this. Yes. I could have reviewed Delta and done that in a safe way. Yeah.

Bryan Cantrill:

But it would have changed every call site. But the but the camera's like, you're changing every call site anyway. Right. The actual like, yeah, is that you're generating more diffs. But like, you're actually not changing your testing burden at all.

Bryan Cantrill:

Right? Because if we actually measure you the way software engineers should be measured on the instructions changed.

Adam Leventhal:

That's right. As God intended.

Bryan Cantrill:

As God intended.

Adam Leventhal:

But an example of a place where LLMs would have made for a Judiciously,

Bryan Cantrill:

Yeah, yeah.

Adam Leventhal:

Arguably more robust solution or more visible or that's probably what you wanted to do, but we're just like, actually, if we got a production outage, I have time to like review 10,000,000 lines of code. I know these eight lines of code.

Bryan Cantrill:

And I'm sure I was also optimizing for like, community optimization. There's that one guy in the community that's gonna be like, can I get under the radar screen here? I'm just trying to get this thing. Right. Just trying to get this thing through here.

Adam Leventhal:

That's right. Yeah. Whereas like a thousand line diff or whatever, or you know, would have raised more Yeah.

Bryan Cantrill:

Same thing. No, absolutely. That's a good case. Then because then the change I made, I think we can safely say inelegant.

Adam Leventhal:

Well, there's different kinds of elegance. It was certainly elegance in its simplicity. Yes. Right. It's lines have changed.

Bryan Cantrill:

Yeah. I think I'm in the parent teacher conference.

Adam Leventhal:

It was Many kinds of intelligence.

Bryan Cantrill:

There's many kinds of intelligence. But yours is a dumber kind.

Bryan Cantrill:

Yeah. This is the problem of us being in

Bryan Cantrill:

the same room. But that's a So that's really Okay. So, and how did the call work out on the In terms of like, you're talking to these folks about how I've Oh. Heard about

Adam Leventhal:

this question about, about that call. So we, we, how it worked out. You know, I think that the, the, university is trying to figure out, like, how does this change? Not just how we teach, but what we teach. And I thought one of the interesting ideas, and I don't know how plausible it is, is in the same way that I found it very instructive to go build an operating system rather than just using one or build a compiler instead of just using

Bryan Cantrill:

Even though other people had built operating systems that you could just Compilers. Go

Adam Leventhal:

And even though my operating system and compiler, you know, suck compared to like real ones, but it's very instructive.

Bryan Cantrill:

Yeah.

Adam Leventhal:

We were talking about, you know, what, what could one do to go build an LLM, you know, you know, in in the context of university course. Yeah. Not one that would be as capable or accurate or sophisticated as obviously what you train for hundreds of millions of dollars or whatever. But what could you go do that would also inform you of the limitations in the same way that like

Bryan Cantrill:

So this is the Simon Wilson kind of Yeah, yeah, yeah. Channeling Simon Wilson, yeah.

Adam Leventhal:

In the same way that like when I am handling a signal, I think back to 1999 when I was writing my first signal hand, like the code that delivered signals. Yeah. And that, don't under, you know, I don't always remember how the Helios signal handling software works, but, like, I I can I can a few bars? Right. In the same way that if you've built a chatbot, you know, used these ways, trained up a model, even if it's it's terrible and limited, it gives you a feeling of what's actually going on under the covers and how can I use this thing more appropriately and responsibly?

Bryan Cantrill:

So I really like that. Where does that leave the rest of the curriculum? Where does it leave the operating system and the compiler and everything else? I don't

Adam Leventhal:

know. I don't know. I don't know what it moves aside. I mean, you do have to choose kind of like, what do you not teach if you're gonna teach this new thing?

Bryan Cantrill:

Well, no, I'm happy to teach Let's assume you can teach the new thing and just make everyone take an additional course. Like, I'm happy to do that. I'm like genuinely Like, I hope people still develop. I mean, there's so much pedagogical value in developing your own operating system and developing your own compiler and so on.

Adam Leventhal:

Yes. But I think it still comes down to like what, I mean, I think something's got to go away, both in terms of like how professors prioritize the time. The students only have a limited number of courses they're gonna take.

Bryan Cantrill:

You and I both have college kids. We talked about this before, but in terms of how they're using LLMs. You've got a college kid who's studying computer science. He's right in the heart of the storm.

Adam Leventhal:

Yeah. So he uses it as a pedagogical tool. I'm not really sure. I think your son may talk to you more about what he's doing than I think I mean, like, you know, like, what I, I think I've probably told this story about, like, I walked into his room and I saw a book on his, on his bed. It was like systems programming.

Adam Leventhal:

And it was like that ad from the eighties. That's like, who taught you this? Right. And he learned it from watching me. And he didn't want to talk to me about systems programming.

Bryan Cantrill:

Is that, I mean, I, oh, I gotta assume like who taught you how to do this? You all right. From watching you. That's not a reference that's gonna go.

Adam Leventhal:

No. No. That's not like SpongeBob is not citing that one.

Bryan Cantrill:

SpongeBob is it, but it could.

Adam Leventhal:

It could.

Bryan Cantrill:

Could. SpongeBob could cite that.

Adam Leventhal:

We should talk to them.

Bryan Cantrill:

We could. I mean, it's Or South Park. That's the kind of thing that South Park or that might make reference to, but it didn't. Mean Moving right along. Partnership for a drug free America, effective ad, I felt.

Bryan Cantrill:

Yeah. I mean

Adam Leventhal:

I still remember it.

Bryan Cantrill:

I still remember it.

Adam Leventhal:

Exactly. There you go.

Bryan Cantrill:

Yeah. Anyone got the reference to that. It also needs to cite their birth year. I'm sorry to sorry to out you. It's like it's that that that you got the reference that you were born in 1975 is not helpful.

Bryan Cantrill:

Yeah. Sorry. So yes. Walking the yes. I've I've got a more more forthcoming.

Bryan Cantrill:

Although actually, he God, I'm gonna I'm really gonna out him. He's got this headshot that he had to do for this thing for the department that he's in. And his grandmother was up visiting him. And she was like, that is a very nice photo of him. And I'm like And he and I both kinda like make eye contact because he and I both though.

Bryan Cantrill:

Yeah. That is an LLM assisted. Yeah. Yeah. A headshot.

Bryan Cantrill:

It looked good. It looked And I'm like And I told him at the time, I'm like, this does not fool me, but it will fool anybody who knows you. Yeah. And it turns out like it will fool your own grandmother.

Adam Leventhal:

Yeah. But yeah, LLMs in school, I think it's Especially in computer science,

Bryan Cantrill:

I think it is so true. A major quandary. Yeah. And then, okay, so then how do you get to this, getting to kind of this, I mean, we can agree that Tan is just like down the absolute wrong path. Yes.

Bryan Cantrill:

Of just like slop maximalization.

Adam Leventhal:

Well, even before LLMs, you talk about the bro coders and the rise and grind of just The programmers. Yeah. Did I say? Oh, anyway, programmers.

Bryan Cantrill:

You call bro coders, but you know what? I kinda, know, it's

Adam Leventhal:

Thank you.

Bryan Cantrill:

Thanks for cleaning it up for me. Know? A hub of git.

Adam Leventhal:

So, but just like that, that was already the mindset. And and and as you say, this this think you had a great You had line, a which was like, this is a lightning bolt into kindling or something like that. This

Bryan Cantrill:

Thank you. Thank you. I had to argue with the LLMs on that one. They thought it was a little over the top. I'm like, no, I'm keeping it.

Adam Leventhal:

I thought it was great.

Bryan Cantrill:

Yeah. Thank you. Thank you. I mean, just Oh, you And so actually, because Eliza had had to split, but made an earlier point that I thought was really important that I I I think is actually the way you intellectually wield these things. I think it is so important to use them as reviewers for helping one augment a creation rather than the whole whole cloth creation.

Adam Leventhal:

I think I agree. I agree. Especially when when it matters. How about this? I I've been thinking about code review.

Adam Leventhal:

Yeah. I'll bring it back. I think that my hot take is some code does not need to be reviewed. Like some code, you actually need like five code reviewers.

Bryan Cantrill:

Need I mean, you were dialing for yes here. You know that. You well, you're just like some I think SINSAC highlighting should not

Adam Leventhal:

always be used. Like, interesting. Okay. Go on.

Bryan Cantrill:

But my black and white.

Adam Leventhal:

But I think for the kinds of code that doesn't need review, fine. Like, just just pump it into an LM. Like, I don't care what comes out. I just care that it does the thing.

Bryan Cantrill:

I I mean, it's something like for me, the eye the eye opening moment in my personal history with respect to LLMs was on Oxide and Friends when someone had access to GPT four that I'd not have access to. And they reviewed my PR from earlier that day that I'd done in Rust and looking at its code review comments. Yeah. And I'm like, these are not bad. Yeah.

Adam Leventhal:

So so if you don't I mean, if you don't need code review, fine. Like, let it go. And if you need a lot of review, then probably means you need to do the thinking for it. Yeah. You know, when I'm asking it to, like, reconfigure my my blog to, like, render something a little differently, I don't give a shit what the code looks like.

Adam Leventhal:

I care what like the statically generated site looks like. And that's enough. So I mean, Which

Bryan Cantrill:

is Gary Tan by the way. Like that you're describing Gary Tan's, that's his use case. Is it? You're just that's Adam's list. You then you're like, I don't give a shit.

Bryan Cantrill:

I'm serving up eight different logos, and one of them is, like, eight megabytes, and I don't care. But the the difference is that Well he thinks that is that all applications.

Adam Leventhal:

Well, I would Sorry, Greg.

Bryan Cantrill:

You you trying to get a sorry. Sorry. Oh oh oh, sorry. Does our guest have something to say?

Gregorien:

Hello. I'm still alive. And so, yeah, I would disagree with Gary Sliss at at this one point because

Bryan Cantrill:

Okay.

Gregorien:

It's not his personal block, and that's the issue that came up in the Fast Company article. I think it was

Bryan Cantrill:

Oh, interesting. Yeah. Yeah.

Gregorien:

Trying to defend him, but it's it just say doesn't block what we care. And it it's not a personal website. It's a civic engagement site. It has to follow federal law from what what I researched. So Yeah.

Gregorien:

There are very hard set guidelines on what standards have to be fulfilled. I believe it's WCAG a level even. So he had five a a level failures. So how would you say it? So basically, he was he was legally liable until I

Bryan Cantrill:

find out.

Adam Leventhal:

So this is not, like, my my personal blog where if it's if it's, like, you know.

Bryan Cantrill:

I mean, I think Greg, first of I I think you're doing him an enormous service by assigning such a weight to his personal blog. But it's I mean, because this thing also is like a I mean, and again, I know, you're you're giving him maximum credit as well. You should. But like, this is a part of, like, why this guy ends up in hot water is because he shoots his mouth off. And this is like a website that allows him to structure the shooting of his mouth off politically.

Bryan Cantrill:

And the because I this guy's got a solution to every problem and has been the been one of these that's gonna, you know, gonna revive San Francisco and so on. And so Gary's list is because this is all like I mean, that image that you're looking at, Greg, is an image of the California flag. So this is like for for Californians, this is like near and dear, and we've got this kind of like alt right VC moron programmer demographic that has the solutions to our problems that's kinda kicking around. So I admire you giving him maximal credit, but I think Adam's blog is much closer to what it actually is. But I I and I actually think his problem is not that that that Gary's list is some weighty thing.

Bryan Cantrill:

I think the problem is that he doesn't Well, he does think it's a weighty thing, first of all. And he also thinks that it's a proxy for all software engineering. He doesn't see any other software engineering other than Gary's list.

Adam Leventhal:

That's a great point. And I think that's something that is really missing in a lot of the LLM rhetoric, which is like not all programming is programming. Like not Yes. Yes. It's like, not well, you just tell the thing and you make the website.

Adam Leventhal:

It's like, yeah, there's some kinds of things that look like that, but not everything looks like that.

Bryan Cantrill:

And this is also a theme that has happened over and over and over again. And that's what people should kind of take solace in the history of this. In terms of people felt like you you to to go back to your earlier Java reference. Yes. And like everything was gonna be in Java.

Bryan Cantrill:

And you had a bunch of like c kernel programmers who were like, what? It's like, no. No. You don't exist. Yeah.

Bryan Cantrill:

And I think that that happened over and over and over again. I remember like with the, getting very trolled by by some of the lock free synchronization work. Saying that like systems won't have locks. And you're like, what? And you think about like the kernel, you're like, you can do that without locks.

Bryan Cantrill:

That's gonna be an absolute you can imagine like literally eliminating a blocking synchronization primitive as as something you would use in a parallel system is like ridiculous. Yeah. But like, no, no, your system, it doesn't exist. Because it invalidates my thesis about the way the entire world is gonna operate.

Adam Leventhal:

Right.

Bryan Cantrill:

And history's dustbin is loaded with these turkeys. Yeah. And people who were overshot and who were wrong. And it's like the other Yes, you've hit on something that is valuable for this substrate of software. But what's good for this substrate of software does not apply to every human endeavor.

Bryan Cantrill:

And now on this case in the LLMs, it's like, it does apply to like, it's probably broader than some of that stuff. I don't know, maybe Java is the best analog.

Adam Leventhal:

Yeah. I mean, I think someone was on the show, maybe even Simon was on the show, was talking about like having LLMs writing drivers. Yes. And I think that made both of us pretty itchy.

Bryan Cantrill:

Yeah. In my view is that like it can it plays a role again. Yes. It it plays a role and I think it can play a really good it can play a very positive role. And maybe that's like the difference is that like you in the Java world, you could have C programmers, which is like, this thing has never I mean, the thing that I did that was closest to Java was working with Tom Rodriguez, wherever that I love T Rod.

Adam Leventhal:

He was great.

Bryan Cantrill:

Do you just You know we call him T Rod? Surely not. Avid listener to the pod. That's right. I I really loved working with that guy.

Adam Leventhal:

That was great. He was like he was he just knew everything about the JVM. He was great.

Bryan Cantrill:

He was great. Was really

Adam Leventhal:

very clear.

Bryan Cantrill:

And so just just to give everyone else a little bit of context, Tom Rodriguez was an engineer at Sun that we collaborated with on the JSTAC on the Java stack backtrace that we could get out of DTRAS.

Adam Leventhal:

That's right. Which is really, really cool. This is in the kernel or after the So the the way that stack traces work for like C programs is like, we just write down the address because like nothing's moving around.

Bryan Cantrill:

Nothing's moving around. So the address being like, we just record all the frame pointers.

Adam Leventhal:

That's right. So we just, we just walk the stack in the kernel, write down the address of, of the program counters and we and we can figure it out later. Yeah. Java's not like that because everything is moving. The JIT is reorganizing things all the time.

Adam Leventhal:

So if you take a stack trace, the things you figure the things you resolve later may no longer be true. So we had to figure out in the kernel, how do we take a bunch of state just like register state and and pointer chase our way to a name.

Bryan Cantrill:

So importantly, we are in a context in the kernel when we do this. It's like, yeah, comfort LLMs. Stay for stay for the the internals. Use Stack Out for d internals session. The, importantly, you are in a context where, for example, you cannot block.

Bryan Cantrill:

And you wanna be in a context where you cannot block because that is that is your sched off CPU probe. Yeah. Like, it's really, really like, that is actually a context you care a lot about. Like, I'm coming off CPU. You can't block Yeah.

Bryan Cantrill:

Trying to figure out what's in the stack. So you've gotta walk this you gotta as you say, you've got a pointer chase. So do you wanna describe the mechanism that we use? Because this actually this is very on point because I grappled with this. We invented a crazy complicated mechanism for this.

Adam Leventhal:

Yeah. I think we piggyback on a different crazy complicated mechanism, but basically, we write a d program that then, in in this case, a very long d program that can like use the We use the D language, safe implementation.

Bryan Cantrill:

No. Just this this is No. I just like love the fact that you have somehow You have to be the press secretary right now for helper actions having to explain the having to put the best possible face on because I mean, this is a this is weird.

Adam Leventhal:

This is weird. So we we write a d program that has all this awareness of the g v JVM built into it. We compile it into this, like, intermediate format that the kernel is able to execute in a safe fashion using this little interpreter that we built for that was already we were using for For probes. For probes, for, like, the the actions for probes. And then we stuff that into an ELF section in the Java binary.

Adam Leventhal:

Right. So when the Java program starts up, it sends down to the kernel, hey, by the way, if you are interested in walking a stack, here's a pile of code effectively Yeah. Yeah. That you can go execute. A little bit like Dwarf, I mean, in a way, in terms of it being machinery that you can use to interpret a stack.

Adam Leventhal:

But I octal's that down to the kernel and the kernel just dangles a pointer off of that proc structure and says, okay, if this comes up, then I know how to walk a stack.

Bryan Cantrill:

And we call that a use stack helper. And we've done T rod. We worked with T rod to do the initial use stack helper, which was for Java. Have Dave Pacheco did one for JavaScript. John Levin did one for Python.

Adam Leventhal:

And for JavaScript, it's like V eight, right?

Bryan Cantrill:

V eight for V eight. Yeah, exactly. Yeah. Yeah. Yeah.

Bryan Cantrill:

That's that's important. Yeah. This is like not you are not doing it for JavaScript. Right. You it is it may have some knowledge of V eight internals.

Bryan Cantrill:

Yes. And absolute bonkers that it works at all ever. Yeah. But I remember just on point thinking, because it's so complicated. I'm like, how can we make this simpler?

Bryan Cantrill:

And I remember, like, turning it over and over and over my head. I'm like, this this is this is it. This is a simple this this very, very complicated thing. It needs every part of this. It's really hard.

Bryan Cantrill:

Like, you can change the mechanism a little bit and you could but, like, what we've landed on is actually the right thing. Yeah. But but going back, like, it I don't think we would call it elegant. No. No.

Bryan Cantrill:

We would call it, like, a miracle.

Adam Leventhal:

Yeah. I think we like kind of squeezed that stone as much as we could. And like, that's the best we could do.

Bryan Cantrill:

That's the best we could do.

Adam Leventhal:

Yeah. And it's like, there's some I think there is a certain elegance to it. Really I I

Bryan Cantrill:

actually do. No. I I agree with you. There's a that's what it is. That's what it is.

Bryan Cantrill:

It's like, no. There's a certain elegance to it. It's like, okay. Yeah. But is it elegant?

Bryan Cantrill:

Right? Well, there's a certain elegance to it, which I do think is true. Mean, because for the user's perspective, it's amazing. Yeah. And it's from a user's perspective, it's like, oh, I've got my my job as Stack Backtrace.

Bryan Cantrill:

Like, yeah, you've got no idea how many kittens were slaughtered to bring that thing to you.

Adam Leventhal:

Yeah. Setting the kittens aside, we we think we Photographic. I think we were I think we had already built the USDT mechanism. So we already were gluing this kind of, like, special ELF section into binaries. That was something we had we were always and that that was another that was a case where USDT, came about because we we tried to write PLOX that.

Adam Leventhal:

Yeah. I tried to write PLOX that, but I did it in the grossest way possible. Yeah. And we decided that was too gross to ship. So the

Bryan Cantrill:

so You playing the role of like, of a quad one shot in this.

Adam Leventhal:

Kind of. Right. That was kind of like, what's what's the what's the dumbest, like, functional way of doing this. Yeah. But then we cared about, you know, how how the sausage was made.

Bryan Cantrill:

Yeah. And I just don't think like I don't think an LLM gets you to that mechanism. I don't know where it gets you, but Jesus Christ, this is gonna take you to the wrong some wrong spots.

Adam Leventhal:

Yeah. As someone was saying in chat, it's like, you know, I learned it from the corpus. It's like, yeah. I mean, that that's part of it. It's like, I I mean

Bryan Cantrill:

Yeah.

Adam Leventhal:

I think for some of these things, the the median is not gonna kind of give you the elegant solution.

Bryan Cantrill:

The norm

Adam Leventhal:

core. Yeah. Yeah. It's like, whereas for some things absolutely, when you're like putting together a web app or like with a bunch of business logic

Bryan Cantrill:

Or you need parenting advice.

Adam Leventhal:

Exactly. Yeah. Yeah. I mean, another podcast we'll talk about are, dead or alive, which SaaS company survives.

Bryan Cantrill:

Oh, the yeah. Dead or alive SaaS bingo. Yeah. Yeah. Do we I I I think we do we audit that list for, like, Oxide customers first?

Bryan Cantrill:

Exactly. Cherish cherish Oxide customers.

Adam Leventhal:

Listen. Anyone who's on prem, you are alive. You're on

Bryan Cantrill:

the alive list, but you have time now to buy a rack. And actually If you're

Adam Leventhal:

if you're looking to repatriate on prem,

Bryan Cantrill:

also alive. Also alive. Exactly. I call sales. Yeah.

Bryan Cantrill:

The but yeah. That I mean, it it's really because I I I think that in that kind of grappling with is you must be you know what Paul Erdrich said. We talked Paul Erdush talking about the supreme fascist and the it took a pen to mark this down. I I I can see you marking down the things you wish to cut later.

Adam Leventhal:

I was reaching for my Apple. You

Bryan Cantrill:

tried to write with your Apple. I don't know if

Adam Leventhal:

you noticed that. Exactly. You tried to find

Bryan Cantrill:

a time stamp with your Apple. This is helpful to have you, like, in front of my eyes. I can actually see all the time stamps you're writing down, like, cut this, cut that, cut the other thing. The but Paul, should talk about the proofs that are in the book. Yes.

Bryan Cantrill:

The the ones that are in the book being the elegant ones.

Adam Leventhal:

Yes. Yes.

Bryan Cantrill:

And the and, you know, and this is like the Fermat's last theorem. I was just thinking Andrew Weil's hand.

Adam Leventhal:

Exactly. Where you're like, when that came out, you're like, that's not what Fermat had in mind.

Bryan Cantrill:

That's not what Fermat this was not the one that was in Fermat's margin. Yeah. I think the Andrew Weil's proof, I think we can say

Adam Leventhal:

with And,

Bryan Cantrill:

you know, it's gonna be wild because I think that that the and we can use LLMs to kind of on we can use LLMs to help us as a tool to get to that elegance, I think. Yeah. But I I also think like I want to do I I just the the thing I I I know this is an extraordinary example, but I also feel that this kind of, like, fear and overshoot that we have is so common for so many different technologies.

Adam Leventhal:

Yeah. Well, and part of I mean, maybe this is what you're saying, but, the the overshoot is bred from fear. Right? Like, Yes. If if I don't, then someone else will.

Adam Leventhal:

If if if Java's the big thing and I'm not Java, then I'm gonna be that. If if everyone is, you know, is just vibe coding their way to success. And in the meantime, I'm telling my staff to use it prudently and to moderate the number of tokens that they use, then I'm gonna be dead.

Bryan Cantrill:

Well, if I can take a photo, why would I ever paint anything? Yeah. Right? And it's like, I mean, which surely existed with the dawn of photography. Yeah.

Bryan Cantrill:

Where it's like, there will be no more Like, why would you ever paint anything? Yeah. And I just feel like we have seen that analog over and over and over again, but we are as always kind of divorced from a sense of history. And and this is a big one. I mean, also think it's like, this is big.

Bryan Cantrill:

I I also feel that like the rhetoric is not helping from people like Gary Tan. Probably the reason like I did not mind taking I mean, well, I never mind speaking my mind. Yeah. Write that down. Put that put that in

Adam Leventhal:

your little book. Reluctant. You were Right. Right. Shrinking flower.

Adam Leventhal:

Exactly.

Bryan Cantrill:

But I also feel like it kind of like the the counterbalance needs to be offered. It's just like you're saying in the call, like you're the only one in the call saying like, wait a minute, can we get to a more moderate?

Adam Leventhal:

Yeah. No, you and I are the same. Exactly. That's right. You're wizzy.

Adam Leventhal:

That's right.

Bryan Cantrill:

But it just, it feels like the counterbalance is really required of like, no, these things are still going to be a tool. Yeah.

Adam Leventhal:

Speaking of, did you see Sri Sri Lama had a post? I did. Where he was talking about, he gave a he gave a lecture. This is another Brown professor. Gave a give a lecture where he was showing prompts that he had given to an LLM, which had produced plausible solutions.

Bryan Cantrill:

Oh, I know I didn't miss this.

Adam Leventhal:

Oh, interesting. Then the students code reviewed and evaluated it and found critical flaws in

Bryan Cantrill:

And that's the assignment.

Adam Leventhal:

No, that was like the lecture. That was, it was like a seminar. It was like a, you know, graduate level class, but, you know, we're we're like a small class. Yeah. But I was like, oh, Shriam, can you please, please share that?

Adam Leventhal:

Because it sounds so incredibly valuable. Right? Again, is not to say LLMs bad. Like, that is not his point at all, but rather how do you apply the appropriate evaluation of these things and know what they're capable of and remember to do your own thinking? And I think that's where we fall apart so frequently is it kind of turning over the thinking to these machines.

Bryan Cantrill:

Turning over the thinking. And I think the thinking is going to still play The role is gonna shift in a lot of ways, but the thinking is And that's definitely the thrust of my piece is that like, the thinking is still important. That laziness is a proxy for that thinking. Yes. And like that kind of churning to find the minimal, the elegant solution.

Adam Leventhal:

The other thing in your blog post that I wasn't familiar, I haven't watched this video. Think you talked about like hammock driven driven development. Yeah.

Bryan Cantrill:

Am I getting that right? Yeah. Hammock developer. Have you watched the Rich Heckey series? No, I didn't.

Adam Leventhal:

Haven't watched that. But what I, what I inferred was that it's like the thinking part, right? The like, I'm laying in the hammock thinking about how this thing is. Yes. And that is so familiar to me.

Adam Leventhal:

Like there is a there's a there's a a project I've been working on for a couple of years now where it's just like every time I'm walking the dogs in the shower, driving, whatever, it's like, how do I squeeze this down? How do I how do I find this more refined approach? And so much of that happens not It

Bryan Cantrill:

front of does. When I do true hammock driven development, which is to say like I'm actually

Adam Leventhal:

Actual hammock.

Bryan Cantrill:

Like when this case, like lying in a bed thinking, I tend to fall asleep.

Adam Leventhal:

Yes. That means

Bryan Cantrill:

And then I have like intense software related dreams. The only time I do have like software related dreams is when I'm like, I'm noodling out a problem. I fall asleep and then I descend into my own kind of madness. And is this No.

Adam Leventhal:

That one we're different.

Bryan Cantrill:

Could you, noticed you're not writing this time down. Would you mind writing this time down? I'm just playing.

Adam Leventhal:

Right. The show says, exactly. That's right.

Bryan Cantrill:

Well, this is this feels like we it feels like now that I'm I'm doing descriptions of dreams. You and I both hate when we describe like dream descriptions. True.

Adam Leventhal:

And I and I and I feel terrible because I told someone recently that about their appearance in my dream, former colleague of ours. And their response was, yeah, dreams be weird. And I'm like, yeah, I am the asshole. I should have just deleted that tweet.

Bryan Cantrill:

I should have deleted the tweet. That's right. I I mean, this has been Greg, thank you very much for joining us

Adam Leventhal:

for the This wild ride. This wild ride.

Bryan Cantrill:

And I mean, think that we is going to be an evergreen. It's going to be just wild to watch how all this unfolds. I still want the Virginia Lee Burton for LLMs.

Adam Leventhal:

I love this idea of the like Mike Mulligan in the

Bryan Cantrill:

steam Where it's like Mike Mulligan in the compiler.

Adam Leventhal:

Exactly. I mean, I guess you and I end up in the boiler room. The compiler is a furnace now. Like maintaining the firmware that drives the building or whatever. Yeah,

Bryan Cantrill:

that's right.

Adam Leventhal:

I hope we can do better than that, but I feel like that's kind of where we're heading.

Bryan Cantrill:

Yeah, think we're going to kick it around in LLM. We'll we'll get there. Yeah. Well, Greg, thank you very much for joining us. Thank you very much for also, taking it upon yourself to prompt Claude to take apart the the, the archive and figure out where the the this thing audit this thing.

Gregorien:

It it was a burden, but had to be done.

Bryan Cantrill:

Yeah. Exactly. I I had that had to be done. And we I I think I I think I said this last time, but we and actually, you know, Adam, I kinda did the math. Like, we you and I did have five separate spring breaks and none of them overlapped.

Bryan Cantrill:

No, no, actually was the That wasn't just speaking euphemistically. We actually did have Between us, none of our kids had overlapping spring breaks because they're in five different schools. And we were just doing things with our kids for many of those spring breaks. That's right. Part of the reason that we've been kind of spotty here in the spring, but I think we got a bunch of episodes in the pipe and looking forward to what is a fruitful summer.

Bryan Cantrill:

Where am I going with this? Write it down, write down, write down. Exactly. No, that's right. Do you

Adam Leventhal:

wanna do it again?

Bryan Cantrill:

Or you happy with that? You know what? I'm actually happy with that. I'm actually, no. Got a bunch of great episodes coming up.

Bryan Cantrill:

Bravo.

Adam Leventhal:

That was really

Bryan Cantrill:

good. Thank you. I thought it was, I thought that time was really, really good. Nailed it. All right, Greg.

Bryan Cantrill:

Thank you. Thank you everyone. Take care and see you next time.

Adam Leventhal:

See you. Thanks everyone. And sorry.