Laughing Through The Pain with David Crawshaw

Some of you might know David as a co-founder of Tailscale. In this episode we're talking about a new kind of cloud he's building at exe.dev. The SSH-only interface is a breath of fresh air compared to the monstrocity that the AWS console has become. We dive deep into why the cloud needs a reset and how integrating AI can be more than just a chat bot.

• David Crawshaw on Bluesky
• exe.dev
• exe.dev Discond community

You can find FAFOFM on your favorite, non-nazi infested social media platforms

• Bluesky
  • Follow episode guests with this Starter Pack
• Instagram
• Mastodon
• LinkedIn
• YouTube

Creators and Guests

Host

Autumn Nash

Host

Justin Garrison

What is Fork Around And Find Out?

Fork Around and Find Out is your downtime from uptime. Your break from the pager, and a chance to learn from expert’s successes and failures. We cover state-of-the-art, legacy practices for building, running, and maintaining software and systems.

Welcome to Fork Around and Find Out, the podcast about building, running, and

maintaining software and systems.

Hello, and welcome to Fork Around and Find Out.

I am your host, Justin Garrison, and with me as always is Autumn Nash.

How's it going, Autumn?

This is delightful.

That's how it's going.

We've been in this like this room for 10 minutes now.

And the guest on the show is David Croshaw.

Thank you so much, David, for coming on the show.

Thanks for having me.

So this has been amazing.

All of us have had a rodeo trip.

And so I had to hit record just to make sure that we don't miss out on anything

else, because this is too good.

Like my eyeliner is smearing.

Yeah, go, go subscribe on, on YouTube, or we actually have an

Instagram and we put the clips on.

There is a dedicated Fork Around and Find Out on LinkedIn, which I just love

that, like, it's like a business LinkedIn account, which we post clips on there.

So if you're not subscribed to like the social channels, we post clips of all

these shows and they're super fun because it's all the little fun things about,

you know, seeing Autumn's mascara smearing or something like that.

Or the faces Justin makes.

Right before he says something really shady and then he like smiles really big.

And it's great.

I'm glad that the world can see it now.

I actually get a light bulb.

It's like, it actually shows up.

I can see it.

It's great.

But we, I met a grown adult the other day who was like half my age and said like,

oh, I watched a podcast the other day.

And I was like, no, that's what, what's happening.

Yeah.

Like, do you know the etymology of the word podcast?

I just want to tell you.

I am of an age that I do.

Yeah.

And so they did not, I'm not that old.

And so it's a little bit of gray.

But if you remember when the video iPod came out, then you're old enough to know.

You know what's funny though?

Like this, this grown adult I was talking to was born in 2006.

And like, dude, see the, see the problem.

Every time my little brother reminds me like what, like how, like when he was born,

like I was graduating high school in 2006.

I'm like, why are you like, you didn't have to do me like that.

I know it hurts.

Yeah.

So, uh, on the topic of being old, um, David, uh, we wanted to talk about, uh,

clouds being old, not you, but the industry and technology, uh, just being too old and

needing, needing a revamp on how things work.

And so you are one of the co-founders of tail scale, which in my opinion is like

rebuilding the network for the internet or what networks should be on, on a, on a

giant amorphous thing that we call the internet.

That's how I feel about it.

Network should be better.

That scale improves the situation.

Yeah.

I'm a huge fan of tail scale.

Can you tell us what you, what you, what tail scale is from your point of view?

Cause I think sometimes our like listeners don't always know all the things Justin

does, so that way.

I spent six years trying to come up with an elevator pitch for tail scale.

I still don't really have it right.

So, you know, it's, can I, can I do it?

Well, no, but I can try.

Like tail scale is a network for my computers.

Like they connected when I installed tail scale on them.

Like they can ping each other.

And without tail scale on them, they mostly can't ping each other.

Like my phone is like on some Verizon thing.

I can't ping that without tail scale.

But once I'm, once I did, once I can, then, you know, I can open up, you know,

port 8080 on my, on my Mac or whatever.

I can just look at stuff.

And like, when I started using computers, that's how they worked

because we had a bunch of PCs.

Uh, we didn't have the internet yet, but we set up a LAN between them.

This was my father's medical practice back in the nineties.

And, you know, we, it was IPX originally, but eventually we switched to TCP IP and

they all had IP addresses and all the machines could talk to each other.

And that to me is what networking is.

Uh, and now we live in 2026 and everything's on the internet, but

nothing can talk to each other.

And that's kind of weird.

And so tail scale is like, what if everyone could talk to each other?

Oh, that was such a good, your elevator pitch is better than you think it is.

Because I feel like a lot of times people can't explain what

problem they're solving, you know?

I really appreciate that.

Also.

I just totally winged that and I've never said that before, so

you know what, save this podcast for later.

So you can then use it again.

Cause it was good.

I'll enjoy watching this on TikTok.

Perfect.

Actually, we did close our, do we close the TikTok account?

I think I did.

Yeah.

The fact that I tried to redownload it again the other day and I went to go turn

off the location services after asked me a bunch of very scary questions about like

we get your data, but like not passively, like actively, like every five minutes.

I know what room of your house you're in.

Like this is like not my neighborhood, but like I know exactly where you are.

And then I went to in the settings and there's no turn off

location services settings.

It does not exist.

Yeah.

Well, I was like, no, I shut it down.

I was like, you know what?

Nevermind.

That's I did too.

But I was like, well, everybody's still on it.

Maybe I should go like, look.

And I was like, I feel like I'm losing, missing so much news.

Like looked at this, like the agreement.

And I was like, not today.

Say nevermind.

Which, which also by like a complete aside on the tail scale stuff.

And we're going to move into the next thing.

Tail scale up, I think is in August.

This podcast is coming out in June.

I am, I will be speaking at tail scale up.

So if anyone's going, it's in San Francisco.

I think the tickets are open now.

You can sign up at least when this podcast is out for sure.

Tickets are out and available.

Maybe it's sold out.

I have no idea, but if you're in the area, come stop by because I will be there.

So yeah, that's fun times.

We know that's not fun because you'll be there.

Yeah, I will.

I will build something ridiculous.

Absolutely.

On top of it, I think last year I spoke at tail scale up

and I built my own game streaming service

using moonlights from an AWS instance.

I'm always like, what is he going to put in the like

kitty backpack today?

I actually did the entire

entire presentation from a steam deck, which was which was also fun

just just because it was a new toy and it was fun to do.

But let's talk about the next thing you've been working on,

because you're building a brand new cloud.

And that's the thing that is too old now.

Existing clouds suck.

And I am a huge fan for anyone that builds a service

that has no app or unique interface.

And it's just like just when I saw SSH into exe.dev, I was like, I'm in.

I want to I want to see how this works.

Can you explain exe.dev to us?

Yeah, exe.dev is a cloud.

Yeah, it's computer elevator pitch right there.

Yeah, I know it took us took us a while, but we finally got around to like,

like it's very tempting to try and not say you're building a cloud

because like that's too big.

You can't possibly do that right.

I'm doing some smaller thing.

But like, let's just go right at it and say what we're doing.

So, yeah, we put computers in data centers

and then we let you use those computers for money.

And that's the service.

And I think that's what a cloud is.

And yeah, clouds are not in great shape right now.

And that's why we're building one.

Like every time I try to use one, I don't enjoy the process.

I gave them all a try before building this one and said,

yeah, I don't want any of these.

I've always thought that.

Actually, I thought that for at least 10 years now.

It's gotten slightly worse over that time as clouds continue to,

you know, change exactly who they're for.

I would claim clouds have never really been designed for developers

because fundamentally the people buying resources and clouds are not developers.

They're companies that are looking for compute.

And, you know, if they can save a certain amount of money on a cloud product

by making developers do a bit more work to use an even worse API,

they will, if like the dollars add up.

And so, you know, the purchaser is not the developer.

And like it really shows with clouds.

You know, I have a command to like LS my AWS VMs so I can see them in EC2.

It's like four lines long and involves like parsing JSON several times.

I'm like, why?

Like, I just want a list.

Could I have a list, please?

I don't understand why this stuff is hard.

And, you know, I kind of do.

There's a lot of path dependence.

The stuff is old.

And so I wanted to build a cloud for a long time, and it's never made any sense

because for exactly the reason I said, the developers don't buy clouds.

You know, they're not the big spenders.

And so why would you build one?

And the answer is things are changing and they're changing because of agents.

We're going to write a lot more software.

It's going to be agent driven and agent written and agent deployed.

And what is best for agents is not the clouds that we have right now.

What is best for agents is the big question.

And my co-founder Josh and I spent a while working on this problem

and we finally figured it out.

What's best for agents is actually just what's best for developers.

And there's a really simple reason for that, which is

models that agents are powered by are trained on transcripts of developers working.

That's that reinforcement learning cycle.

That's that's where it comes from.

And so, you know, it's whatever is best for a developer is best for an agent.

And so none of the clouds are best for agents because they're not made for developers.

So the idea is if we build something we actually want,

we're also building something that agents actually want.

And so we have this, you know, great reason to go and build all this infrastructure.

And so the real focus initially is developer environments.

You know, the idea is use XE.dev for writing software.

So that's the that's what we're doing and why we're doing it.

Now, I feel like there's a.

I don't know, a shift in what people were thinking

maybe last year or the year before, what agents wanted, right?

Like agent wanted APIs and clouds made sense there because you get APIs

and like MCP is basically like, what if my my shell commands were an API?

Right. Like that's basically what MCP kind of is doing for a lot of folks.

They're just like, let me just show it.

And like agents talking to MCP servers was all about

how do I get this agent to talk to an API that is better

than the output of FFmpeg slash help?

Right. Which is which is sometimes hard to parse.

But agents are generally or at least LLMs in general kind of figure it out.

But in a lot of cases, like there's an FFmpeg MCP server

that gives you more verbose information around that.

Why isn't that the right solution of exposing

LLMs or agents to APIs?

Because it's not what developers are trained on.

You know, it just comes back to those those those reinforcement learning loops.

Developers type stuff into the command line and the command line is a great API.

And you can see this with agents today, right?

Like if you take the playwright MCP and plug it into your agent

so that it can, you know, for cloud code or something

so that it can use Chrome while you're doing some web work,

which is really good, by the way.

Like if you're if you're ever trying to like get cloud code to create some CSS

for you, it's just like a person.

It'll like make some CSS and then you try looking at it and it doesn't work

because, you know, the alignment's wrong.

And with a playwright plugged into it, with Chromium plugged into it,

cloud code can open it in Chrome, take a screenshot,

look at the screenshot and say, oh, yeah, nothing's aligned correctly

and then go and fix it.

And so you get this wonderful feedback loop from plugging playwright

into cloud code or, you know, some other equivalent Chromium system.

And yeah, you can plug the playwright MCP in,

but you can also plug the playwright CLI in and it works just as well.

And, you know, if you try measuring like how many tokens

does it take to solve a problem, either way,

it's about the same once you both get them working.

Only I can get the playwright CLI working faster.

And so, you know, I don't see the fundamental advantage

for agents in MCP from a

easier use.

The advantages of MCP is that for us, not for agents,

it can be easy to put auth on that stuff.

And so it can be easier to expose services

that are otherwise just really difficult to expose.

And so, you know, MCP potentially has a lot of value.

I'm not saying don't use it,

but it's not fundamentally better than a command line interface.

I actually think that using AI and command line interface is easier

for me just in general.

Me too. It's easy for me to know what's going on, right?

Exactly.

Well, none of those command line interfaces were ever designed for

something like an agent or like there's no auth built into

pretty much any Linux CLI tool or at least standard CLI tool.

So the way you're getting around that is basically more sandboxes, right?

You just say, actually, I'm going to give you full access in a VM

and go have fun because there's no files there for you to destroy or.

Which is like really hard, too,

when you're trying to interact with multiple things like what I'm.

I feel like that's like one of the biggest barriers to using agents

because I'll be committing to open source, but then wanting to commit

to like a proprietary repo or like my own personal repo.

And then the off between all those three of those things is so difficult.

And then you're trying to like be secure and not just give it access to everything.

And then sometimes you can't give it access to the one thing you want to give it to.

Yeah, absolutely. I agree.

That's actually the hardest problem with agents is setting them up so they can work.

Yes, it's it's hard because agents have to be able to do it themselves.

They have to have their own feedback loop without you in the middle saying,

I ran the tests and here's the error.

Because otherwise you just sit there doing that 30 times a day,

but on four minute intervals, which is the worst.

That's and yeah, so you've got to figure out how to set that up.

And right now it's a ton of work to do all that stuff.

And it's it's, you know, we've spent actually

we've spent 18 months thinking about this problem and trying various things.

And I don't think there's any easy solution here.

I think we have to rebuild a bunch of stuff and it's going to be pretty painful.

That's where do you see people using agents?

You know what I mean? Like, where do you see?

Because everybody wants to be like AI is going to replace people.

And then people are like, we hate AI.

Then they're like, we love AI.

And like, you know what I mean?

Yeah, like you have a more measured approach from what I've seen on your blogs.

Because I think my opinion of AI even has changed over the last year.

Where with like currently, like where do you see people

like using this technology that you're building the cloud for?

I mean, it's good your opinion has changed over the last year

because AI has changed a lot over the last year.

I feel like it's changed a lot in the last couple of months.

Yeah, it was it was somewhere near the end of last year.

So like October, November, depending who you ask, where models got really,

really good at the tool calling loops for agents,

where agents went from like writing 20% of my code to almost all of my code.

You know, it was a really huge shift.

And, you know, it changed the way I think about them.

Are you ever scared that you're going to like lose your muscle to write code?

Because like, I think like not to plug the company I work for,

because it's not about that.

But I've been using GitHub CLI more because of the CLI interface

and being able to like see what it's actually doing,

because I feel like I learn the commands that I don't know,

but also just kind of knowing and tracking where it is and what it's doing.

And I feel like it's slightly less scary.

Like, do you feel like like do you ever worry about that, I guess?

I mean, I think it's a real risk.

Like I've been using agents to do all of my Git work for me

because driving the Git API, it's the CLI itself has always been too hard for me.

I've never been very good at it.

And I'm never going to rebase again my hand ever.

And I've already forgotten half of how to do it.

It's so much better than I am.

Like me resolving merge conflicts is just a disaster.

Like if you if you ever wanted to make like a

like a ridiculous video for the Internet of embarrassing programming,

it would be me handling a merge conflict.

Like I get it wrong like four times.

But it's rad that you'll actually admit that, because I love how people are like,

you can't be a developer unless you know all the Git,

but we're never going to teach Git in college or do it.

You know what I mean?

Like, and there's never any room for people to be like, I hate it.

And it was hard.

I feel like if I became a Git expert and really knew how to use it,

there'd be no space left in my head for programming.

And so like I'd be really good at merge conflicts,

but I wouldn't know how to write a computer program.

So don't you think that's something that's like not talked about a lot?

Like it's like I think sometimes we want to like posture to be like,

I know everything and it is impossible to know everything.

Like you're just not going to be very good at something if you can't,

like if there's no room to grow.

But then I also kind of wonder, do you think that like

we'll get to the point where all the knowledge will be like behind a paywall

because we're going to have to use AI?

Like, you know what I mean?

Yeah, I mean, I don't know.

That was the cool thing about development.

You had all these websites, right,

that you could go to and you could learn code for free.

Like, is this now?

I've never thought about that.

You know, I haven't played through the idea of like knowledge

being locked behind one of the models,

because right now the models are desperate for any knowledge they can find.

Right.

So they're only good at things that are very public and very on the Internet.

But you're right.

There is I think there is a long term risk there of some sort.

But it's it's years out, I would say.

Well, one of the things you mentioned that like end of 2025,

agents got better at writing code.

And I feel like in in May, when we're recording this of 2026,

the open source models are getting better at like they're catching up.

And so all the price hikes and changes in every AI company

and people are just like, oh, you know, like we we can't go back to the old way.

But I need to be able to have some control over this

or at least be able to just like buy a model, like buy box software.

That's like an element, right?

Like, because like,

you know, like just being like somebody who came from the graph design world,

like be usable to buy Adobe like suite and you pay for it once

and you use it for years.

And then it got to the point where it's like having to pay for it every month.

Like some like some artists don't make the same amount of money every month,

you know, so it's like like and I wonder like a lot of what we're training on

and what, you know, developers used with Stack Overflow in these forums

that were public and that everybody talked about the problems they were having.

And it gets to a point like if those like if those forums disappear,

like what are we even going to train models and developers on, you know?

Well, there's lots to train models on.

That's what that's what the subscriptions are.

Like when you buy a subscription to one of the major platforms,

you're agreeing in the terms of service that you can be trained on.

And that's true. Yeah.

And so there's a there's a lot of training data out there.

It's harder to come by.

It's not all nice and open like Stack Overflow anymore.

Fortunately, the open models seem to be able to train on the closed models.

You know, I'm not sure they don't want to.

Yeah, that's right.

This this whole distillation process, it seems it seems quite easy.

And, you know, it's, you know, they're very upset about this.

But but they also stole the world.

So screw you.

You know what I mean? Like, it's so funny.

It's like they're just like and this is like egregious.

And you're like, but you like did it to like artists and writers.

You know how many lawsuits are you like?

And you you just, you know, rolling through every YouTube video

and not just that, but our answers on Stack Overflow and questions.

You know what I mean?

Like I spent like a lot of my early career at AWS

answering people Stack Overflow for the service I worked for.

You know, nobody's paying me for all of that work.

Yeah, I don't know what the correct moral and legal outcome of all of this is.

But my violin is very small.

So so that's.

David, you're really good at software,

but you're really funny, like you've had me like laughing the entire time.

I want to. I wish my kids agreed with you.

No, my kids think I'm so not cool.

Kids never know.

Like our kids have to talk to each other and then they'll think each other's funny

because a lot of kids thinks I'm hilarious.

But my kids are deadpan.

So annoying. They don't think I'm funny or cool at all.

They get on like the like they get on FaceTime with each other

and Justin will be in the background

and they think he's the coolest thing since Slice Bread.

And I'm like, well, how do you think you know him?

My kids looking at me like, what do you mean?

Him? No, this is my dad.

Yeah, I don't think my kids even know what I do.

That's fine.

So I want to take a step back here on on building a cloud.

Everyone has known for a long time.

The wrong way about going, building anything is going and buying

and racking and stacking a bunch of servers, right?

Yeah, not a capital and time and stuff you have to do there is just

you're going to you're going to die and someone's going to surpass you.

Why did you go that routes?

How did you go that routes?

And like, what does that look like today?

Yeah, it's I mean, you're right.

Like you can't compete with a hyperscaler on racking and stacking computers.

Like that's actually true.

You get economy of scale by buying billions at a time.

And so the machines are necessarily more expensive than they they'll sell.

And that is a certain deal size.

That's the killer.

That's that's what makes it all not work.

But the deal size is very large is the key thing, right?

Like we actually started XeDev on AWS.

And that's that's when we launched in late December.

We launched, we silently launched around Christmas

and then like got overwhelmed.

And, you know, we were off AWS, we were mostly off AWS by the end of January.

But like our AWS bill got to 100 grand a month before that.

That is not enough for an AWS sales rep to call you.

They don't care.

100 grand a month, that's nothing to them.

Like there's no discounts at that price.

Which is like wild, because I think that the bar is going up.

I remember when we used to give out discounts at like a certain scale.

And if you think about the workloads with AI, that scale has got to get bigger

and bigger because people are consuming more compute power and their bills are bigger.

And the gap between the top end and the bottom end is wide.

That's what I'm saying.

It is just getting ginormous.

They don't care about anyone in the middle there.

Yeah. Yeah.

And not caring about anyone in the middle is why there's a chance to try

and build something and why the economics of this can work out in the middle size.

Because right now, if I try to run on AWS, I pay their vast margins on top of their machines.

And then how can I charge users on top of that?

Like I can't pay for everything.

But if you rack your own machines, they're dramatically cheaper than AWS's list prices.

And so that's at a medium sized scale to build one of these things.

You have to rack your own machines because they're not going to give you the discounts you need.

It's either that or go and raise all the money in the world and then give it all to AWS.

Which is, I mean, if you're not wrong, look at what people are doing.

People are either doing colos because that's the only way that can control their bills.

Because, you know, if AWS decides to like charge a certain amount for data coming in and out

and they change something like you're now stuck because that's what you built it on.

You know, so it's like, that's what I'm saying, like, oh, guess what?

I get free bandwidth. What the hell?

That's something that like, and think about the fact that for a long time, that was like a secret, right?

Like people did not know and they didn't take that into consideration when doing these budgets.

Right. So, but like, you're not wrong.

Like how many times have we heard from like Blue Sky and all these new places,

these new like companies that are scaling at this extent.

And they're like, we are running in colos because that's their options.

Yeah. Blue Sky is too small for AWS to care to give them proper discounts, which, you know, is a little astonishing, right?

And they only have like, well, they have like 10 machines and like, like three of those are Raspberry Pis, right?

Even if they had a hundred, they'd be too small.

Yeah.

You know, it's a...

But that's what I'm saying, like, that is the threshold.

Like, think about how much Blue Sky has scaled over the last like few years, you know,

and they're still too small for that discount.

I mean, what you're saying is you can do a lot with one computer, which is very true.

But isn't it like cool?

Like if you mentioned the two things that you've built, right?

You're talking about Tailscale and XEDev.

Those are, it's almost like solving problems more in a way that we used to solve them a while ago.

You know, like actually like having, and not just that, but it's kind of like more control over your network,

more control over your data and your cloud.

Like that's all things that we kind of gave up some control of things for like ease.

And now it's not easier or cheaper.

And it just seems like a lot of this is like going back to almost the basics, you know?

Yes, that's a hundred percent.

My whole philosophy with building everything.

You're saying your third startup is going to be a storage provider.

Blob storage.

I mean, that's not a terrible idea.

You know, yeah.

Yeah.

I mean, I would say one other thing, which is one of the reasons why I was willing to jump so quickly into Colo is

building on top of another cloud means inherently accepting some of their design decisions.

Identity.

Yeah.

Yeah.

And it would be different if you knew exactly what you were signing up for when you were accepting those design decisions,

but a lot like just different regions and different clouds and just different ways that those services were built.

Like sometimes you have no idea what you're truly building on top of.

Absolutely.

It's always, and that's particularly true at the PaaS level, when like you're using those services, you read the docs, you say,

I can build my software like four weeks in, you discover some new constraint of the PaaS that ruins everything, your design is invalid.

Oh, that's happened to me several times.

That's why I won't touch PaaS anymore.

I don't think people like realize like what you're being sold.

Like this is somebody else building a technology on top of other constraints that you are now bound to that you don't even know exist.

That's right.

I mean, and AWS Lambda, one of those PaaSs, it's built on EC2.

Yeah, it's all VMs under the hood, right?

It's computers at the end of the day.

All of it is a Linux server in the cloud.

Exactly.

There's one physical constraint of AWS machines and GCP and all the rest.

They're all very similar.

That really interests me about doing our own machines that I'm very interested in trying to do something new in, which is they're all very big

into remote block storage.

And this made a lot of sense when they built it 15, 20 years ago.

And in fact, I was working on big storage systems at Google or adjacent to the storage team because we had a lot of storage back in 2010.

Like remote storage was exactly the correct solution for everything back then.

And it was because we were using hard drives.

And with hard drives, a seek to random read or random write takes 10 milliseconds to move the head and the hard drive physically.

And then you add a millisecond of Ethernet roundtrip time.

It's not that big a deal.

And so we built all these remote systems.

And then at some point we swapped everything out for SSDs and SSDs have 10 microsecond seek times and then one millisecond of Ethernet roundtrip time on top of it.

And so now we've taken these really fast SSDs and put them behind very slow network.

And, you know, we've taken away a lot of the advantage of NVMe.

And I think that there's an opportunity to try and revisit that.

And like clouds don't want to revisit that.

Like you can get machines with NVMe in them on AWS, but they're odd.

They're odd in the sense of the NVMe is relatively small compared to the capabilities of the machine.

Or you've gone to one of their more exotic instances that are clearly designed for running really big database servers.

And they're very large.

And you're probably much more expensive also.

You're looking at five or six figures per month to rent something.

It's that's right.

It's a very expensive machines.

And even on top of that, they're unusual and really interesting ways, right?

Like if you rack a traditional machine with a bunch of NVMe in it, like maybe a drive will fail and you've set up some sort of RAID 5 like thing.

And, you know, the drive fails.

You have an ops person you're paying.

They go in, they swap the drive.

The machine is running integrated mode the rest of the time.

Swap is good.

In the cloud provider, when one of those drives fails, they have you reset the machine and you immediately come up on a new machine with all the drives there.

But you lost all the data.

You know, your drives get wiped between those events.

And then you have to restore from your remote block storage, which is a multi-hour procedure, which is totally fine if you're doing postgres replicas because your postgres primary fails.

You fail over to a replica.

You bring up a new replica.

You spent eight hours filling, you know, 50 terabytes of data into the replica.

No big deal.

But it also means there's a whole stack of software you can't write that is easier and more reliable to build on physical machines than it is to buy it from a cloud provider.

And that's very counterintuitive.

I think of the cloud provider as the most reliable source out there.

And they are, except in this one instance.

And so I'm really interested in this problem because I have multiple times written a program on my laptop and then shipped it up to the cloud.

And it's really slow.

I'm like, why is it really slow?

And it's like, oh, well, my server is using the disk.

Yeah, exactly.

The server in the cloud's got 3,000 IOPS and my local laptop's got 500,000 IOPS.

I'm like, OK, yeah, there's the problem right there.

So, you know, we're experimenting with that and seeing if we can do fun things with it.

It also depends on if you're on like something that's hosting multiple workloads or just your workload, too.

The noisy neighbor problem is always fun.

Going back to January, like we're out of the AWS.

What did that practically look like?

I want to because like this audience is like really interested in the details.

And at that point, you say, OK, I need you're like I need you need four locations.

You're like, let me find some colos.

Let me just go like rent some servers from them.

What does that actually look like?

I've like core like how do you calculate?

We have a hundred thousand dollars in AWS and we're going to move to that's whatever.

Maybe a hundred machines, let's say, or whatever it was.

And you're moving to and you're going to move to what, like 20 physical servers?

Is that like what the calculation ended up being?

Like, how did that look?

Yeah, it we when you're doing startup things, you always think about growth.

And so you immediately say to yourself, look, we're spending this on AWS for these users.

We're growing at this rate.

Let's build something that the users can grow into.

And then we'll think about the previous users and moving them off.

And that's very much the model.

You know, we still have a bit of an AWS built from users.

We haven't finished moving off.

That's because you can move me if I'm there.

I don't remember when I signed up.

But yeah, we we really should move a lot of users because we have more regions now.

And so we can move the machines physically closer to you.

You get a better experience.

The machines are bigger.

They're they're they've got less load on them.

You know, there's a lot of nice things about them.

But we have a stack of users to move.

And so it's you know, we've got lists and we're going to start sending people emails

and like trying to semi automated because there's a few.

But the we've been we're using it as an excuse to build good live migration machinery

for moving VMs between regions without restarting them, which, you know, is going pretty well.

But it's a there's a long tail of things we run into, including just networks going bad in the middle.

And like suddenly there's a lot of packet loss and all of our live migrations go slow.

It's astonishing.

But yeah, we started by reaching out to providers who sell bare metal in in data centers.

But do quite a lot of management around it.

So they've pre racked them and you're purchasing machines that are already there.

Yeah, we could service.

Yeah. And there's there's several of those.

And that's you can do that very quickly.

You can get machines up in days doing that.

And so we did that.

Those services are very busy right now because a lot of people interested machines.

Also, they're all going through price pain right now because.

Computers have got a lot more expensive, and so they're all busy revamping all their prices,

and so everything's very messy right now.

But there's several very good services out there.

And then the step after that is you start talking to a separate set of companies who basically run ops and data centers.

And for this, you can actually go to some data centers, do this directly.

You can talk to someone like Equinix and they have op services they offer.

But basically, you want to have some company on retainer who is employing someone who is there at 2 a.m.

who can swap out the disk when it breaks, because I'm not, you know, if I have to fly to London to swap out a disk at 2 a.m.,

you have to wait for the plane to get there.

And that's going to suck all night and I'm going to be tired.

And so it's very much and, you know, it doesn't make sense for us to employ a full time person in London to maintain just our machines

because there's not very much maintenance on a small set of machines.

And so this is a great thing to share between companies.

And this this is done at several levels.

And there's a whole ecosystem of companies that offer these services in different forms.

And so we just spoke to a whole set of them and picked one.

And then we are racking machines with them.

And then it's a matter of ordering machines from a machine provider and you have to price them out with several people.

And then there's a question of how much extra do you order and where do you store it?

And again, a lot of these intermediate services will do the dropshipping of the machines for you, too,

and the racking and like help manage all of this.

And so we're in the middle of processes there right now, you know, proving out machines.

What what kind of scale were you at in January?

I mean, if you just launched it like soft launch in December.

Yeah, you're not you're not doing hundreds of physical machines in all these locations.

No, no, we're doing dozens.

Yeah, this is more than more than number.

And we're building out some extra capacity as part of it all.

One of the interesting challenges with us is we're trying to, you know, again, we're trying to make VMs easier to use.

And the big thing for me is and this is if you want to get technical, I don't sell people VMs.

I sell people a cgroup in which they can run VMs.

And, you know, that's probably too technical.

But the basic idea is great for the audience.

You know, it's right here.

Yeah. I mean, I want to give you CPU and RAM and disk and you run as many VMs in it as you like,

because from my perspective, a VM is a Linux process.

And if you want two of them, that's OK.

You have two processes. They're in a cgroup.

The resources are assigned to the cgroup.

You want a hundred of them. That's fine.

You might run out of resources.

Like you need to buy more resources.

That's that depends on your machines.

And that's the other thing I want to talk about was your pricing model is difference where it's not a easy two.

I need I need five micros and I get charged for each one.

You're selling a pool of resources, right?

Like your pricing model is how much CPU, how much memory, how much disk space.

I don't know if if network is counted in there, but I know that AI tokens are at something you can buy more of each of those things.

Right. These are all knobs I can turn and say I need more of this thing because whatever my workload is or my constraints are, I just need the constraint to go away so I can pay for more and divvy it up however I want.

Right. Yeah. Network is in there.

We just dropped the price because the original price was AWS's bill plus five percent for my accounting errors.

And now which was enormous, of course.

And I've dropped it a bunch, but I haven't dropped it as far as I can because we have some interesting things we want to launch and we're trying to decide how complicated to make network pricing.

Like, you know, it'll it'll I think it'll make sense when it comes out.

But we can do the nice, cheap networking that you would expect in a colo.

Because, like you said, it's like a 10x bill for just immediately going into a cloud.

So why? Why go that route?

Why pool? Yeah.

So this is entirely motivated by the fact that agents have just changed what programming is to me because I'm one of these programmers who always has like 10 things they want to do and can do maybe half of one of them and like not as well as I wish I could.

But, you know, it's something.

And so I've often ended up with like lists in an Apple note on my phone of like one liners of a program I wish I could write.

And like I write it down just so that I can then forget about it, never think about it again.

And the nature of an agent now is I can take that one line, drop it into an agent and like a good quarter of the time the program I want it pops out or something close to it, especially with a few more minutes of work.

And, you know, quarter time is not great, but it's a lot better than zero, which is what I had before.

And it's getting better all the time.

So, you know, agents are, you know, creating this is what I mean by every program is going to have so many more programs now because it's just so easy to like build these small things.

And so instead of an Apple note, what I do now is exe dev slash new, a little prompt box appears.

I type the one liner in there and hit create VM.

And then I forget about it.

I look at it again a few days later and like sometimes it's useful or else sometimes I just delete it.

And for that flow to make sense where it just makes me the program and keeps it running every time I create it, the marginal cost of the VM to me has to be zero.

Because my idea that I just put an Apple note is worth less than a cup of coffee to me.

I'm not going to go and buy a micro VM.

It's like, you know, it's like, oh, three dollars a month.

No, I'm just not going to do it is the actual answer.

So that, you know, because most almost all of my ideas are terrible, as you would expect.

Right.

You know, I have a lot of them and I obviously the quality has to be low.

I make it up in quantity, though.

And it's the bulk buy.

It's the bulk buy.

But when I was reading your blog, that was one of the most relatable parts when you're like, I always have all these ideas, but like rarely do I get to them all.

And I was like, me too.

Do you know how many domains I have?

Yes, exactly.

Right.

So I feel like it was like, and honestly, the only time I've ever used those domains was when I was using Copilot to get that, like to get those websites up faster.

So that way, you know what I mean?

Because I don't have time to do all these things.

And I feel like you're solving a really interesting problem that we all have and experienced.

I'm glad you feel that way.

The yeah, it's very, it's very hard to explain that.

Like, yes, the innovation in my software is the pricing model has reduced the marginal cost of VM to zero.

It's like one of those technically correct statements that everyone's like scratching their heads around.

Why would I want that?

But I think people, I actually want that.

That's not crazy, though.

I mean, if you look at it, a lot of AWS's success was being able to lower the prices when they were offering things to people because they had excess.

Like, you're not wrong.

I think AWS's benefit early days was lowering the time to get to something.

Right. Because it was like the process of I have an idea is like, OK, we'll go price out.

Wait for the next procurement cycle.

Go price out what your budget is going to be for that idea for the next year.

Then we're going to rack and stack.

It's like three months later.

You're like, oh, yeah, I don't think I want that idea.

That's part of it.

But think about it.

When you went in as an essay to most conversations, you were talking about how you were going to like offer more and continue to make it less or to

be more simple. You know what I mean?

I mean, that was just lies.

That was just marketing.

The price will go down.

Like, yeah, he said the quiet part out loud.

I will say that with the right architecture, I've saved people a lot of money, but that's also because they were building it wrong.

When we shipped the carpenter, like we saw like people's bills go down by like double digits.

And we're like, that's exactly what we wanted.

That's great. And we helped you do that.

We didn't drive the the price of the AWS instance or the network bandwidth down.

We drove their consumption down.

Exactly.

Yeah, no, that makes perfect sense.

And like, you know, how would I do this if I wasn't, you know, I hadn't built XCDEV and I wanted this, what would I do?

I'd go and get a VM, VPS like thing somewhere.

And I'd set up something like Docker or some other, you know, nested VM thing on it.

And then I'd set up a reverse proxy that routes to all the things.

That sounds horrible.

I would basically build XCDEV is the idea.

And I was like, well, what if someone built that for you?

Oh, yeah.

All right.

So that's what it is.

And that's what it is for individuals.

And then for businesses, it's that at a larger scale of like, instead of you bring up three EC2 VMs and you've got to size them appropriately, you just purchase a pile of CPU and RAM from us.

And then you run the VMs you need to run and call it done.

And, you know, size your pool appropriately so that all your VMs can run.

That's actually rad because it's the same premise as the cloud of not me not having to go and rack and stack my own stuff and not having to like think ahead and buy and like, you know, procure all these things, but at the same and but have the ability to use like compute power.

So that's actually like the best of both worlds, but much cheaper.

I think it could come out cheaper.

So like, in a sense, AWS has two kinds of margins on their product.

One margin is like they charge you more for the machine than it costs them, which is fine.

Like, that's correct.

That's good for them.

The second is like on, you know, I buy machines that are too big for the thing I'm doing.

And that's traditionally one of the arguments for the PaaS systems is like, you know, don't go and buy a bunch of, you know, vCPUs and leave them lying around on a VM for spikes.

But, you know, another way to approach that is like you have a pool of vCPUs that you buy for all of your machines.

And, you know, you handle it there.

There's interesting open questions about like, how regularly do we let you dynamically resize your pool?

Like, you know, I price it per hour for the business case.

Like, it seems reasonable that we'd have APIs that let you resize the pool hourly.

You know, we have to figure that out.

That's, you know, it gets more complicated there.

But, you know, there's a stack of spike demand things I'd love to support somehow down the road.

Do you think you'll ever get to the point where you do like spot pricing when like compute is like really not being used and then it gets cheaper and then more expensive when it gets really expensive?

I would love to do that.

I think you need large numbers to make that work.

But it's a fundamentally possible thing in the sense that if you're racking machines in London for a UK audience, then at 2 a.m. in London, those machines are going to be less used.

There's going to be extra resources.

And so you should be able to spot price them at that time of day.

Like, that just makes sense.

And it's just complicated.

And so it's not, you know, we're a team of eight right now.

I said nine yesterday.

I was wrong.

We're eight because I can't count.

You counted the agent.

It's not meant to do that.

They're not people.

Actually, that brings up another question I had.

It was like you integrated Shelley.

You have this agent built into your like VM interface with a proxy available.

Can you explain technically how that works and why you decided to go do that?

Yeah.

It's very easy to build an agent harness.

Honestly, it's not super easy, but it's much easier than writing a language compiler.

You know, in terms of like from a software engineering perspective, it's like everyone

should do an agent harness 101 course in computer science.

And honestly, if you spend a whole semester building an agent harness, it's going to come

out about as well as Cloud Code.

Honestly, there's nothing in there.

Like, I don't know why they built a bunch of stuff in there for like sandboxing and

stuff.

But if you skip all that and just let it be root on a VM, it's really easy to write.

Your sandbox is outside the harness.

Exactly.

This is easier.

Yeah, that's right.

So like almost all the machine, you know, it exposes a couple of tool calls and it's a

big while loop that calls an LLM.

That's just easy to write.

And it's not just easy to write.

It's really fun to customize.

Like, I actually think, you know, I suspect those things will end up open source because

engineers are going to want to customize those more than they ever wanted to customize a

text editor.

And like we have no end to customization of those.

So it's quite easy to build one.

So that's that's nice.

That's your first thing is like, what's the cost?

Then there's a question of what's the benefit?

And like one of our problems was like, oh, what if someone shows up to XeDev on their

phone?

Because people do that.

I do that.

I learn about things on my phone.

Be cool if they could make a VM and do something useful there.

And like with all these CLI agents, what's your story for like how they would log in and do

something?

It's like, oh, well, we need a web based agent.

What web based agent can we use?

Like, oh, crickets.

You know, there's nothing.

For some reason, it's a thing.

So, you know, we built Shelly, which is a web based agent.

And we said we'll include some LLM tokens as part of the base plan because that's super

useful to individuals.

Let them buy more.

I'd love to just plug the the subscriptions in and maybe we can now with the open AI one.

I need to be really cool because I'm just thinking like I could be like sitting there

with my son going to bed and like like just orchestrating a bunch of programs to be written

while I'm like doing that.

That's rad.

And then by the time you're done putting your kid to bed, you can go and like code review

and see what it actually made.

That's pretty rad.

Yeah, I, you know, I use Shelly for actually all the XCDEV programming I do these days

precisely because it's easy to use for my phone and it works just as well as codex or

code, code, code, code, code.

Always have trouble saying that.

I love a good efficiency tool like that.

I can do my life, but also be like being productive at the same time.

Yeah, that's it's also, you know, you know, we're trying to it's just it's super useful

to have there because like, oh, I want to set up a VM.

I want to put something on it.

I say to myself, it's like, well, I've got an app, get update and app, get install a

bunch of stuff or you can write a one line prompt to an agent running on the machine

and let it do it.

And how do you, how do you do that today?

Like you mentioned, you have an idea you, you know, XCDEV new and then give it a prompt

and then you just go away.

Like I, I, my first experience was basically that with interacting with Shelly and saying

like, okay, I want to try this thing, but I spent probably three, four hours like prompting

it again to tweaking things.

And it wasn't like a fully autonomous, you know, dangerously skip off permissions.

I mean, the, if I'm doing an idea just out of the blue, yes, I just type something into

XCDEV new and let it go.

Uh, and the first thing I see is like, is there anything potentially useful here?

You have to iterate to get any, you know, to actually get it to a good place, but that's

the nature of agents today.

They can't do it all themselves.

Honestly, most of my ideas these days are in our code base and they're like, how do

I, uh, uh, you know, I want to make a change to how XCDEV works.

Uh, and for those, I've gone through several stages of like what I do.

Uh, the thing, the, the thing that's most reliable right now is I have a whole set of

VMs and I treat them as like reusable Git branches.

And like, I just drop into one, reset the thing.

I have a little reset button, uh, and then, uh, uh, type something into Shelley and let

it, let it rip and see what happens.

Then it brings up all of XCDEV inside one of our VMs.

And I just take, play with it and like start creating VMs inside this little, you know,

embedded universe.

But, uh, the thing I've been working on is the same thing my colleagues have all built.

We've each, each of us is building one of these things, which is an orchestrator.

And, you know, one of the, uh, the basic idea is I have one VM and I say, I want to do a

thing and then it starts a new VM, gets the Git repo checked out.

It has like a standard play.

And then, uh, it takes the results and drops them into some sort of task list like thing.

It's every time I open the orchestrator, there's like all the ideas I've tried are in

this thing.

And we've thought about building a generic one of these, and there's several companies

out there that have built one of these, but what we've seen so far is like each one of

us that's built one has built such a radically different orchestrator.

And like, they're also, it takes like an hour to build one, uh, that, uh, we're not ready

to commit to building one of those generically for everyone.

So what we're trying to do is build all the machinery.

And so things are about to launch, uh, that, uh, a couple of our customers using, we're

going to make these public in a moment, uh, is, uh, a scoped, uh, SSH key for Exedev,

uh, that every VM gets so that every VM in Exedev can make new VMs.

It gets its own little like sub account universe, uh, where a tag gets applied.

And so like it LSs and it can't see all your other VMs, like the, you know, it's its

universe starts empty, but then it can make as many as it wants.

And they're part of your account.

Uh, and that lets, uh, that lets you write programs that use VMs and, uh, and that is

like the basis of an orchestrator.

And then if you check out our integrations page, this is our, this is our version of

secrets management.

And so one of the things we've realized is almost all of the secrets we have, and I'm

not all of them, but almost all of them are basically HTTP headers that you send to a

service.

There are token that get embedded in a HTTP request.

Uh, so we set up HTTP proxies, uh, outside the VM with the token in them.

And so from inside the VM, you can send a request to like to GitHub and you can pull

private repos and push them and everything.

But what's actually going on is you did a GitHub OAuth flow outside and that's what

the token lives outside the VM.

So the agent can't X fill it or anything like this.

And the neat thing is you can set up all those integrations on tags, uh, so that, you

know, your orchestrator can go and make a new VM.

It's immediately got GitHub access to whatever repos it wanted to give it to.

It's got access to your Clickhouse logs because it's a Clickhouse integration, you

know, whatever external services you want.

And so our whole goal is to build all of the infrastructure, to let teams write their

own orchestrators, because my guess is for the foreseeable future, engineers have to

build that stuff themselves.

And if you try buying one off the shelf, you're going to find a really awkward piece

of software that almost solves your problem that you'd have to customize a bunch.

And like the work of customizing it, it's easier just to open a, open a, uh, uh, an

agent and start building it.

Uh, if someone gives you all the pieces, you know, I mean, this is the idea behind like

Gastown, right?

Like Gastown idea is an orchestrator for all these agents.

And so like layering these things in pieces of like how they get contained or like the

harness at the very bottom of like the tools calling LLM, we have an orchestrator on top

of that.

That's calling all the, all the harnesses and you're boxing all that into container or

into, into VMs, a VM sandbox that also can do more VMs so that each agent is, is contained

inside of that.

Each, I guess at that point it's a harness, right?

So you have the orchestrator on top of a bunch of VMs.

Yeah.

That's, and there's several, there's several ways you can arrange this.

Someone yesterday told me about type two agents.

I was like, what's a type two agent?

You know, I thought I knew about this stuff.

Is that like type two fun?

No, no, because I'm really into type two fun.

This was something else.

This is, I mean, isn't programming just type two fun?

Like you just stay yelling at a computer for hours.

Because like the whole time you're just like, this is horrible and it's the worst.

And then all you're like, but I love it.

When you like something good happens.

Oh my God.

I just had a revolution.

I like, I usually judge people that like type two fun, but you just made me look at

myself.

It's you Autumn.

It's you.

Wait, are you, are you going to, are you going to make me Google type two fun?

I don't know this.

You have to like, you need to know this.

Okay.

Pause.

I need to know David, whether it's your type two fun, because just in the like

brief hour of knowing you, your type two fun tracks, like you're like, let me go

build all the hard things.

You mean my type two fun?

That's not programming.

What else do I do?

Uh, I mean, exercise is my type two fun.

It's just, it's awful.

You know, I, I hate it.

It's, it's the worst, but, uh, you know, I do it.

You know, running all those sorts of things.

It's great.

I just feel so called out and attacked right now.

Like I've always been like judging the people that do like type two fun.

And I was like, Oh my God, my life is built around type two fun.

And this whole time I didn't realize it.

No, it just, you just, uh, found a whole new group of people to love.

You know, now you get them all those, all those people, you know, when you, when

you see them like doing an iron man or something, we're going to be friends,

but like, I just need to like, somehow, like, I need you to find a way to put

like your facial expressions and the hilarious, like jokes in between.

Cause like they're fire.

That's very kind of you.

Uh, two days ago, someone told me I, uh, had a face made for radio.

Jeez.

Who said that?

They were a hater.

They said it themselves too.

And I was like, they're right.

I get it.

The ones that also watch the podcast.

No, this is someone who knew what radio is.

See, I, oh, wow.

Does that mean we're old?

Sorry.

Oh, I'll stop now.

Also, I feel like we've lost like that part of storytelling and I think that's

what people are obsessed with TikTok, you know, like the being able to see

people's facial expressions and react to like saying something funny.

Like that's like such a missed art because sometimes you like drop a one

liner and it's hilarious and you know, it's hilarious, but you wish you saw

the other person's face and then like every time you guys say something and

it's like too real and you both start laughing before it even gets out of

your mouth, you know what I mean?

I'm like, I miss seeing people's facial expressions.

I've always struggled with jokes for that reason because I find them really funny.

Getting them out is hard.

But like me too, but like, it makes it so much funnier.

Cause you're just like, oh, it's going to be so good.

Like I can see it.

I agree.

I agree.

It's fun.

So there's a British TV show I just watched recently called Last One Laughing.

But why is British comedy so much funnier?

It's because it's different and comedy is always funny when it's different.

And you're right.

It's all specialized in sarcasm.

Like nobody's business.

Like, yeah.

And misery.

That's a, that's a, I mean, if you lived in England, wouldn't

you have to be funny too?

It's like, it's just a, sorry.

I live in Seattle.

It's cold and rainy here too.

I really like Seattle.

It's nice.

I like it too.

It's a nice kind of rain.

Then you have a, you have a Harbor.

It's really good.

The water.

But yeah, Last One Laughing is great because it's a bunch of comedians in a room and

they lose and have to leave if they laugh.

And so they were just telling jokes to each other and having

to deadpan it the whole time.

Like it's, it's so hilarious.

And also sometimes I'm watching him like, am I meant to laugh now?

I don't even know.

Like what's happening.

Was that a joke deliberately or accidentally?

I swear.

Wait, like I liked your, actually like liked your joke about like laughing

through misery does not seem like sometimes that's the only way you

get through like a shitty time.

You're like, if I, like, if you can't like cry, you gotta laugh about it.

Cause if not, like a hundred percent, you know, that's a, I think that's why

like when you see like a developer meme, it hits so hard cause you're like,

Oh my God, somebody else struggles.

That's why I laugh so much.

It's still with the tears.

Yeah, exactly.

You got it.

So, uh, that's uh, but yeah.

Uh, let's see.

What are we talking about?

We're talking about computers.

I was going to ask what's next.

Next.

Yeah.

What's next.

I mean, that's one direction we're going in is like your VMs need VMs.

You know, it's like, you just need a computer.

It turns out your computer needs a computer.

Like we just need more and more computers.

That's absolutely it.

Uh, but those, uh, there's several other things we're working on simultaneously.

One is those integrations that secrets management, like we have to help as much

as we can there because it's hard.

And like I said, I don't think there's an easy answer to any of this, but we can

build a whole bunch of machinery to make it easy for you.

Like if you want, uh, every VM you start to just be on your tail net as part

of a tail scale integration, we should just make that work.

And so that's on our list.

I literally was trying that the other night where I was trying to run the, um,

the apertures, their new like AI proxy.

Cause like, again, I'm running local models and I was like, Hey, I like

Shelly, but I also want to be able to use my local models.

And so I, but I didn't want to go through the whole tail scale off flow of doing

this thing.

And it looks like there's a, like a tail scale aperture proxy that they're working

on, um, which will probably help there.

Um, so yeah, that was, those were like the things that I was kind of looking at.

I'm like, how would I get this XC dev VM to talk to my aperture so that I can, again,

all the auth is stored separately.

It's outside the VM, right?

That bootstrapping problem is always hard where I'm like, where do I put all these

secrets?

And then I have to like copy and paste them every time.

I'm like starting something new and go through new OAuth flows.

I was like, no, I don't want to do that.

I want to do it once and then scope it for the thing that I'm working on.

Yeah.

Well, tail scale has some good OAuth flows built into it that we can leverage for an

integration into XC dev, which lets us, you know, set it up on a tail net when you

create the thing without any extra steps.

And lots of other, uh, larger, more mature services have that too.

So it's actually possible for us to set it up so that if you go through a flow once,

that's not easy, but you do it, uh, then, uh, you can actually do an AWS integration

where each VM you bring up with a tag can automatically reach some S3 buckets you

have without any extra work.

You do the same with GCP, lots of other services, uh, Slack integration.

Where's that?

You know, there's so much to build.

Like our problem is really like our, you know, we have so much to do, uh, and you

know, only so many hours in the day.

That's kind of exciting though, because I feel like a lot of times people that have

AI startups, it's like, well, I wouldn't say that you have an AI startup, but let's

just like in this realm and this time, it's always like these really random,

obscure niche problem solutions.

And they're like, I'm just like, dude, can you do something to make my life easier

with AI and like yours is making lives easier so you can go build things.

But like, it's also so exciting because you have all these problems to solve.

And like, I feel like right now people are just making up problems to solve and

like, that's a legitimate, like you have like a very clear roadmap of all

these cool things you can add.

Like, is it exciting to be working on something that like you can actually,

you know, it's so much fun.

And, you know, we have, we have some really great users who like, you know,

there's a discord where people hang out and like tell us when we break things and

give us ideas and like ask for features.

And like the feedback we get from users is part of why we have an endless roadmap.

Like they keep coming up with good stuff we have to build.

That's so rad because a lot of people do not do feedback and they don't realize

like the, like what a community can offer like their company.

So that's rad that you are intentionally building that community and getting that

feedback because like, that's just like, it's so much, it adds so much to your

process of becoming like this really awesome product that people enjoy using.

Because developers love to build something that they like because it's cool and they

built it and nobody wants to use it.

And it's absolutely horrible.

You know, like you've almost created a way to have a better software

development loop, you know?

I feel like you're exactly right with, especially in the, the age of, of idea to

execution is cheaper, right?

Like just the, like I have a list of ideas and I can execute on them.

I am buying more domains because I'm actually executing some of them.

But be real people bought, people made crappy software because they thought they

were smarter than everybody for forever.

Now you can just do it faster and like do it at a bigger scale.

I think the problem I having is that sharing of the thing I built or the thing

I created from an idea that I didn't invest time into the thing that I want you to try

is really weird to me because it's, you know, some people call it slop.

Some people are like, if I didn't do the AI prompts, then I don't know how

yours works, so I don't want it.

And I'm like, that's weird too, because it's like, it's still software at the

end of the day, but I think the lack of taste in a lot of it of just like, Oh,

well, this model did it.

So I'm, I'm relying on Claude to have taste in what not to include.

And I see these people building these things that are more complicated than

they need to be because they didn't put any thought into, like,

come back to the software development life cycle.

Like, I think a lot of times the people saying that things are AI slop also

because like what, whether you're coding with an agent or not, you should be

thinking about how it's built, having like opinionated, like, you know, facts

about how you want to build this and your architecture and figuring out

these different things.

And that is what happens before coding ever starts, right?

Like the planning and the requirements and thinking about what you want to

build and how you're going to break it down.

And you can do that with an agent and you can do it with regular coding, but

that's the whole doing that first.

Right.

And I think that's a problem that we've always had.

It's just exasperated.

Yeah, that's, that's right.

I was writing slop long before we had AI.

That's what I'm saying.

Like, you know what I mean?

I can write terrible code all on my own, right?

This is not a.

This is just the point where you either continue to like, are you iterating on

the first thing that you got?

Where do you planning it properly?

Like, to me, it's like the same, but people could just can do it cheaper and.

Faster.

The weird thing is I'm just seeing if two people have the same idea and they, they

both, they could work together and like build a thing, right.

Or, but in most cases they both go off to their own agents and they build it

themselves in their own prompting style.

And they're like, actually, I just want this flag to be a little bit different.

And, and when someone else slot something and send it to me, I'm like, no,

I'm just going to build that myself.

To me, I was equated to like, when you fart and like, at the end of the day,

like, like you're like, this is why my kids love you.

Okay.

Like, are you, why are you like, like, like someone else's fart is so terrible

that you don't, you don't want anything to do with it.

Right.

But like your own, you're like, you can, it's fine.

Like, it's just like, I did this.

It's okay.

Uh, it's terrible.

It's a bad thing, but it's still mine.

And, and at the end of the day, like someone else's

killing your ugly children.

Like when you like, cause you spend so much time on a piece, right.

That you become so like romanticized and in love with it.

Like it becomes like your child.

So you can't take constructive feedback anymore because all you see is this

thing that you spent so much time on.

And you have, and you know the process, you know what, like why you made some

decisions, but at the end of the day, they're both, you have like two farts.

And they're both basically the same thing.

I think that's like the argument of if SaaS is dead or not, it can't be dead

because people like, if it's one thing to have a good idea and then contribute

it to like GitHub and then make it an open source project, but if you're like

two people are always going to be better at code reviewing and looking at something

and figuring out if it's good or not.

And just how David said with the whole having a discord and people giving you

feedback, like software is easy.

The fun part is writing the code.

It's not the fun, the not fun part is maintaining it and like how users are

going to use it and breaking it and making it reliable and updating it.

Right.

Like, and AI is not going to do all of that.

Yeah.

I think there's actually some pretty subtle stuff we can tease apart here

that I don't know if I can do it in a short enough time for a podcast, but let

me give it a try because I think not all programs are the same, you know, and like

there's a, there's different kinds of programs and like what's ideal has

shifted, so what's, what's gone on is agents are a force multiplier on our

ability to program and that means we can create more code and that means some

decisions we made about how we write software have changed some of the time.

And so to give you an example, there's always a tension when you're dealing

with a web server, for example, between, do I write my own web server using a

library that does almost all of the work, or do I take an Apache or

Nginx style thing and configure it?

And, and five, the generally accepted norm was, oh, you take the Apache server.

I don't even remember if you had Nginx at that point and you configure it.

And like, here's the 800 line config file and you get it just right.

CGI bin all day.

Yeah, exactly.

Here's a whole bunch of mods you add to it, but it's one program.

We all share it.

We've added a billion flags to it to handle the billion use cases.

And then the Go standard library came out and Brad built a, an HTTP library

into it that's just a full web server that you can start in like five lines of code.

And so you've got this other way of building, of doing a web server

configuration, which is, oh, write a binary that's like 50 lines long with a

web server, just the way you want it using this library.

And so there were these two ways to solve problems.

And in one of these ways, there's one web server that we all work on, that we all

configure, and we built lots of stuff in it to handle the cases.

There's another case where there's 10,000 different web server binaries out there.

They're all like slightly tuned.

And both of those are kind of workable, but which of those is the correct way to

do something has shifted because of agents, because it is now so easy to use

the library to build your own custom binary.

It's almost certainly the correct option now, whereas before you could have a

legitimate argument over, should we all pile into this one thing?

And so I think the question of whether we all pile into one piece of code is how

much do our needs overlap and how much configuration do we have to add to it

to handle everyone's cases?

If we can write a program that meets all of our needs and we don't have to

customize it dramatically for each person, then we should totally do that.

And we should keep doing that.

And I think the Linux kernel is a good example of this, even though it's very

customizable, it's everywhere because it's fundamentally, we all need device

drivers and we all need a pretty stable Unix API on top of that.

And so we shouldn't each write a new Linux kernel replacement in the world of

agents. We should keep sharing that stuff.

But there's probably a whole set of programs out there that we shared before and had

awful configuration on top of that should now be decomposed because of the force

multiplier of agents.

So that's my attempt to split hairs there on what kind of program.

Actually, I think that was a really good summary.

And if you think about it, like you said about the Linux kernel or just different

ways that we use open source, like I made a joke that Linux is like a religion and a

cult at the same time.

But it is because people want to do their own little differences.

But sometimes it's like when I use my Mac as my daily driver, but my dev environment is

Linux. Sometimes it's depending on what your risk or just how much work you want it

to be. So I think that the different levels of what you want that software to do and is

this just for fun? Is this like in production?

They're all different. So I think that was actually a really, really good summary of

that. I think one of my fears right now in 2026 is even

this week, in the last week, we've had multiple critical CVEs against Linux kernel

things that have come out and like, here you go, here's a root shell for nothing and

you can do whatever you want. And the complexity of that kernel and what it's been

doing at this Apache style shared interface, right, like it does everything for a lot of

people that there's going to be a lot more of these micro tuned kernels for specific use

cases that are like, actually, I don't need the whole Linux kernel.

I'm only doing agents.

And so like for me, like at work, like we only run Kubernetes, right?

Like our Linux distro only runs Kubernetes.

So we just disable a bunch of stuff that we're like, we know we don't need this stuff,

right? Like we're never going to have interactive users at this level.

And so but I think that's happening.

That's going to be happening more as as that decomposes.

The problem is it is infinitely harder to track when something is vulnerable, when it's

a tweaked your own implementation of like I can go to the library.

But that's like maintenance, right?

That's why everybody like when people get all excited about like writing a bunch of

code and I'm like, cool.

And so you have to figure that out.

No one's been in any of the stuff in like the one thing that I've always like come back

to that like enterprises need to centralize on is security for the reporting side of it

just to say like, are we vulnerable?

And if you're going at the go standard library, that's a lot of stuff.

And you say, oh, I need this patch level to avoid this CVE.

But also maybe you still implement it a different way.

And like those are too coarse, those like jumps from go versions when a CVE comes out

or just a hat comes out.

Can we just talk about how hard Go is when it comes to like how they do all of their

dependencies? Like it just makes me want to puke sometimes.

I have hated Go for a very, very long time.

Their dependencies are ridiculous.

But I think you should have existed before GoMod.

This was oh my gosh, it was trash.

I hated Go. Agents for me today, like solve so many of that problem.

I was like, actually, like I don't care about this anymore.

And GoMod does help. They do all the things that I absolutely like hate.

That's what I use agents for.

I think rebasing.

Yes. No, but honestly, like doing the little like tedious thing so I can do the like more

bigger thinking. I like I think that you're on to something, though, like because we are

going to have to give agents kind of more of room to build and to help us.

Right. Like I think that just like we were talking about privacy on our last episode

and making things more private by default, I almost think that we're going to have to

make things more secure by default.

Yes. So I think that a lot of we're going to have to really cut down on what we're

offering, whether it be a Linux distribution or, you know, whatever it is, like it should

be almost the least privileges or the least amount like what you need to actually get

the job done. And then kind of using whether it be like a smaller image and more secure

image when you're not really needing to do anything complex with it or you know, because

I think that's what we're that's where we're getting to.

That's why people are using change guard or people are wanting those smaller images,

because when you're letting an agent loose or just junior developers, right, like we've

been trying to cheat. We've been trying to teach people security for years.

It's hard. So now they're letting an agent go and these agents are breaking out of boxes

all the time, you know, so I think we're going to have to start building intentionally

like default security where we're releasing distros with less and if you need more, add

it to more, you know, and and even distro is too big of a like I saw a pull request

today for the Linux kernel to dynamically disable functions in the kernel where you

can just like do a kill switch to like this function name doesn't don't don't do it

because even if the CV can't be like I have no patch for this, I just need to disable

this function call completely Linux kernel.

Right. You need to build it dynamically at runtime across the board.

And even if your kernel says my you know, I have a CV against this thing like, no, we've

disabled those functions. Right.

Like you need it more fine grained at this sort of like hyper speed of how things are

coming out and we can't really software and we can't patch it fast enough.

We won't we never will be able to.

Just looking at the last big CV that was zero day, it was patched so so differently, like

a time frame over all of the distros and think about how bad that CV was and just trying

to roll that out, you know, like.

I think short term we're in a lot of pain around the fact that models have made it a

lot cheaper to find exploits and software.

And I don't know whether Mythos is really significantly better than Opus at this or

not. It's very hard for me to see through that fog, but let me use Mythos and maybe

I'll answer it. But Opus is really good at it.

And so there's clearly a problem right now.

And like we see this and all of the bugs we're seeing right now, long term, it's not

necessarily going to be a bad outcome because if the models stop getting better, if it

turned out what there was, was there was a step function into like, now we can find

exploits.

We found all the CVEs for a while.

It's kind of interesting, though, because is it like is it a bad thing or is it just

because now we have this ability to find them faster that we wouldn't have found them

previously?

Well, they were there. They were already there.

Right now we can find them. That's good.

And if the models don't keep getting better at finding exploits or they only get a

little bit better, then we're in a great place because right now at work, we have every

time I push a commit to the system, Opus sits there for 20 minutes in a fresh context

being asked security questions about it and being like, are there flaws, are there

exploits in this? And sometimes it finds them, which is very off-putting.

But we're finding those bugs in the software before we're pushing the binary to

production. And so if the Linux kernel starts going through that process of every

change that comes in is being analyzed by an Opus or a Mythos or whatever it is to find

these exploits so that we can catch them before they're deployed, we actually end up in

a much better place in terms of software security.

I think that's true. It's like finding all the pain up front and then getting through that

painful cycle of being able to do better.

There's no question we're in pain right now.

And I think it's going to get worse before it gets better, actually.

Yeah, I think there's some pain in our future.

Yeah. So everyone laugh through the pain.

That's what we're telling you to do.

Just if you're here, you should be laughing through the pain.

And if you're listening to this show, go SSH to exe.dev.

There's literally no account sign up ahead of time.

There's no request. You just get a shell and it's amazing.

So thank you, David, for coming on the show and explaining that and building the exe.dev.

Thank you. Thank you so much for having me.

It's been a ton of fun.

I'm so excited to see all the things that you build, just listening to you talk about

it. I'm so excited to kick the tires and try all the stuff out because it sounds like it's

really going to be extremely useful.

And it's exciting to hear a really useful startup idea that is going to make it easier

for us to build things.

Thank you. I'll try to remember to actually post the features when we release them,

because that's our biggest problem right now.

One of my colleagues last Monday, he was telling me, he's like, oh, yeah, I take the

Discord thing and I paste it into my orchestrator and it produces a PR.

And I look at it and I hit submit and the continuous deployment deploys and I fix it.

I was like, do you tell them that you fix this?

And he's like, oh, no.

Now you need to build an agent to go and tell people what you fix.

We need a Discord bot, yeah, it's on the to-do list.

Well, thank you very much. I really appreciate this.

This has been great.

Isn't that crazy?

Like, we're going to change the whole cycle in which all of this, you know what I mean?

Like, it's so rad to see it playing out in real life.

Thank you for listening to this episode of Fork Around and Find Out.

If you like this show, please consider sharing it with a friend, a co-worker, a family

member or even an enemy.

However we get the word out about this show helps it to become sustainable for the long

term. If you want to sponsor this show, please go to fafo.fm

slash sponsor and reach out to us there about what you're interested in sponsoring and

how we can help. We hope your systems stay available and your pagers stay

quiet. We'll see you again next time.