Subscribe
Share
Share
Embed
Ed has made Scalpels free, and Peter wants to know why his answer to everything is to just make it free. Plus evals on cheap models, keeping a mental model of a system that changes every day, and a Stack Overflow for agents.
A casual conversation over a drink between me and someone doing interesting work in or around the Laravel community.
Ed Grosvenor (00:02)
Alright, so what's new in your world?
Peter (00:08)
Right. F first all I have a lot of energy, although I was working until eleven PM last night. So whoo. what's new in my world? I wanna show you a few things. One of them is I'm actually starting to use Jeff. I just kicked that off this morning. but I think the the way in which I'm doing evals for it, and I think that that we can talk about that for a few minutes. and yet then
Peter (00:37)
There are I have a few other topics. One thing is how do I keep a mental model? So I'm telling I'm telling, you know, Claude, we're fixing all these problems and adding all these features and improving all this stuff, but how do I keep a mental model of what is actually the system? And I wanna sh show you that too. Yeah, those are top of mind. What do you got?
Ed Grosvenor (01:01)
Yeah. And maybe we
Ed Grosvenor (01:02)
can maybe we can talk about how I've I've solved that.
Peter (01:07)
yeah, if you've solved it, great.
Ed Grosvenor (01:08)
Yeah,
Ed Grosvenor (01:09)
well, I mean, I I say solved. It's it's the way that I'm pursuing
Peter (01:12)
Uh-huh.
Ed Grosvenor (01:13)
a solution to the same problem. and I feel really good about it for me. I don't know necessarily that it's something that I could make generic and and make work for everybody. It works well for the way that I work. you know, but we can we can talk through
Peter (01:26)
Yeah, yeah, yeah.
Ed Grosvenor (01:27)
that and and you know, decide whether or not that's something worth doing some sort of a, you know s publishing some content around because I do think it's it's working pretty well.
Ed Grosvenor (01:37)
So yeah, for me, it's it's been an odd week. It's it's a tough one to to make you know, big announcements and and big changes and and things, because the the kids are home. Well, at least the older kid is home from school. We have in Switzerland we have a two week school holiday. And for the first week, the school that he goes to provides a camp. So it's basically like he he just goes like he would on a normal school day. this
Peter (02:03)
Nice.
Ed Grosvenor (02:04)
week they did not, so
Ed Grosvenor (02:07)
We spent Tuesday all day at Technorama, which is the the big science museum outside of Zurich. And it is just amazing. it's the second time that I've been there. and you can very easily spend an entire day and just not even notice time passing because there's just so much to do. And it's unlike most kids' museums that are, you know, very kind of interactive and hands-on, this has
Ed Grosvenor (02:35)
stuff that will hold everybody's interest so everybody's having fun at the same time. So it's not like you're just sitting around watching kids get dirty or play with Lego or whatever. It's it's really fun. so that was good. But but in the midst of all this.
Peter (02:47)
Nice. Are you still solo parenting?
Ed Grosvenor (02:51)
No, no no no my my wife made it home from Italy. So she's yeah. So yeah, yeah. I mean yeah,
Peter (02:53)
it's both of you this week. Yeah yeah. Yeah yeah. Was the house was the house clean? Spotless? Mm-hmm.
Ed Grosvenor (03:00)
well, I mean, good enough.
Ed Grosvenor (03:04)
It worked out. but in the mid in the midst of all this, I you know, we we had talked briefly about the challenges I was having with the mental model around the two sided market with scalpels. and I just
Peter (03:17)
Yeah, yeah.
Ed Grosvenor (03:18)
decided to lop it off and make scalpels free to the end user. So
Peter (03:25)
wow.
Ed Grosvenor (03:26)
there is yeah. Well, my bet is
Peter (03:30)
Your solution
Peter (03:30)
for everything tends to be let's just make it free.
Ed Grosvenor (03:35)
Well, so yeah,
Peter (03:37)
But let's dig into it. Yeah, yeah, yeah.
Ed Grosvenor (03:38)
we'll we'll dig into this real quick because I think this is gonna kind of guide a lot of what we talk about today. one of the things that occurred to me, because I'm so I'm seeing just people tweeting, hey, I created this this free alternative to this thing, which is a kind of a clone of something that I've already done. Like nothing
Peter (03:54)
Uh-huh.
Ed Grosvenor (03:54)
that I've created here is particularly unique or difficult. So there's there's that
Peter (04:02)
Yeah.
Ed Grosvenor (04:03)
aspect of it.
Ed Grosvenor (04:04)
And
Peter (04:04)
Some of it's very useful.
Ed Grosvenor (04:06)
it it's it's extremely useful, but it's it's not super unique and even scalpels itself as a control plane that that facilitates the the deployment of things to Laravel Cloud. It's not that unique. I mean if somebody looked at what I had done here, they could one shot it in a day. It's not like it's not super complex.
Ed Grosvenor (04:30)
but what I'm finding
Peter (04:31)
Mm-hmm.
Ed Grosvenor (04:31)
already, even not doing any marketing on this or any of that, it's immediately it's lead gen. I'm getting people reaching out and saying, hey, I want to use hone. I I want to automate around it. you know, how do I set that up? And so the way I'm looking at it is, you know, I could just use it as lead gen. I could just put it out there for free and and just eliminate every other part of the
Ed Grosvenor (04:57)
the the whole thing and just have it be like here's all these free tools. I advertise the hell out of those. And then I offer our services if you want to create automations around all of this stuff. That's not really the sort of work I necessarily want to be doing, but there's plenty of that work out there. so I'm just gonna lean in on if you're a freelancer
Peter (05:15)
Got it. So mm-hmm.
Ed Grosvenor (05:18)
or an agency looking for lead gen or looking for a toolkit that you can take to clients and just say, look, you know, we can
Ed Grosvenor (05:25)
We can take these tools that you can self-host to replace all of this stuff that you're paying monthly for. We can automate around this stuff, and we can help you really own your stack. I think that's a good pitch for agencies and freelancers, and there are already interest coming in from just people who are stumbling upon these things and saying, Hey, I'd love to be able to take this telemetry and just make my application self improving. How can you help me do that?
Peter (05:54)
Think that's a killer use case by the way. and okay, so so so what you're saying is you're building set of tools under scalpels, which is but it's all free, it's basically enhancing the kind of Laravel ecosystem with useful free things that are well built and maintained by ad. for now it's leech and the future we'll see.
Ed Grosvenor (06:22)
Yeah, well it's
Peter (06:23)
Is that what it is? Yeah.
Ed Grosvenor (06:24)
and and so my bet is that I won't be the only person that finds value in the lead gen. So, you know, as you saw I had that
Peter (06:31)
Okay.
Ed Grosvenor (06:32)
that partner section for freelancers and agencies. And my plan is to just to pitch them and just say, Hey, you know, this is all useful stuff. It will help you to close deals with companies that you're already talking to because they wanna solve these problems. So rather than, you know, letting your clients go ahead and
Ed Grosvenor (06:52)
vibecode something or having them ask you to vibe code something. Take something that's already well written, deploys in one click, is actively supported, use that. It doesn't cost you anything. And then what you're actually going to sell is all of the work around that. I'm going to take your your telemetry, I'm going to take your
Peter (07:11)
Yeah. Yeah.
Ed Grosvenor (07:13)
your consolidated logs, I'm going to take like everything that that these tools can provide, and I'm going to wrap that up and and just
Ed Grosvenor (07:22)
You I mean, one of the problems I'm seeing is everybody's trying to build
Peter (07:24)
The support.
Ed Grosvenor (07:25)
the software factory, right? They they all want to build some sort of a tool that they can go out and they can sell to everybody and it's just going to work. But the problem is that everybody's everybody's stack is different, everybody's way of working is different, things shift so fast that like if you try to bundle this whole thing up into some sort of a product and sell it as a product, the state of the art is going to shift under your feet. you know, so
Ed Grosvenor (07:52)
All of a sudden you're locked into this thing and the world has moved past that thing.
Peter (07:56)
Yeah. Yeah.
Ed Grosvenor (07:58)
and so it's gonna be a world of frustration. So I feel like just providing these really core fundamental tools and allowing companies, agencies, freelancers, you know, just in-house developers sort of iterate on on that. Because the, I mean, the things that are fundamental will and will never change are like your telemetry, your screen session recording, your
Ed Grosvenor (08:20)
like all of these this data coming in, that's never going to change. so
Peter (08:27)
Yeah.
Ed Grosvenor (08:27)
that's sort of the bet is that I I think that freelancers and agencies will really like it. I think that the Larable community and and actually the broader software community will want stuff like this to exist. I'm also leaning in a lot heavier on the, you know, the lab that I introduced last week.
Ed Grosvenor (08:49)
that and content. I have a guide that I've just published late last week that is the this is like a comprehensive guide to Laravel Cloud for agents.
Peter (08:59)
nice.
Ed Grosvenor (09:00)
there's some drift between the open API spec for the cloud API and the actual API. So they they've not managed to keep the spec up to date. there's some stuff that's documented for the CLI that's supposed to work one way and it doesn't. And so these are all things
Peter (09:17)
you know, I have
Peter (09:18)
an example of that. Maybe. So it seems to me that the bandwidth usage metric looks like it's broken.
Peter (09:30)
Because it's been at 95 % for three months in my situation. It just doesn't make sense.
Ed Grosvenor (09:35)
Hm. Yeah, that may be. I've I don't think I've ever ever actually specifically checked that. I do have a cost control one, but I don't think it would have flagged anything if I was o unless I was over my allocation.
Ed Grosvenor (09:50)
so yeah, that's interesting. I can I can definitely take a look at that. But anyway, so it's just this living guide, right? And as my
Peter (09:56)
Yeah. Yeah.
Ed Grosvenor (09:58)
agents encounter things that where where the documentation or the spec has drifted, or it encounters some sort of a gotcha, you know, like the I I told you about that that situation where if you mistype an environment ID, it'll fall back to the default environment and do the operation on the default environment.
Peter (10:17)
Oof. But
Peter (10:18)
I like that so much as a general concept because every API, you know, like it's, so this is for Laravel Cloud. Your agents are finding little like, wait, this isn't how it's documented.
Ed Grosvenor (10:30)
Yeah.
Peter (10:30)
They can ping that into this central repo so all the other agents can see it. Next thing they're going to start talking amongst each other.
Ed Grosvenor (10:39)
Yeah, yeah. No, I mean I mean that's that's sort of inevitable. and actually we could facilitate that. I was thinking about taking so one of the crazy ideas I had the other night was I now have Campfire for Rust and Campfire the the PHP version, the DHH ported, sort of ported to Laravel. I've actually finished porting it to Laravel so that it really is
Peter (10:57)
Yeah. Yeah.
Ed Grosvenor (10:59)
kind of is Campfire but written a larval way.
Ed Grosvenor (11:03)
so I have these these different varieties of Campfire. And you know, so one of the things I'm thinking about doing is like just building a community around this this whole concept of scalpels, like the the freelancers, the agencies, any developers who are using this, like there's just this community you can log in, we can chat about it, you know, things like that. Like just utilize Campfire.
Peter (11:24)
I like
Peter (11:24)
it. Here's an idea. Can I throw in an idea? So how about this?
Ed Grosvenor (11:28)
Yeah, I wonder if you're gonna
Ed Grosvenor (11:30)
Yeah, I wonder if your idea's gonna be what the second thing I was gonna say, but yep, go ahead say it.
Peter (11:33)
What is your second thing?
Ed Grosvenor (11:35)
My second thing was have a version of that that's just for agents.
Peter (11:40)
But how about this even better? make a thing where, and this is for Laravel specific agents, where the instruction is kind of like, hey, if you find Laravel specific challenges, like with the cloud API or like, you know, and there's like seven other Laravel ecosystem products, they don't all have to be by, you know, the mothership with Flux or whatever.
Peter (12:10)
then go and discuss that in this place. And now everybody can just like get that MCP, connect to it. And now their agents are chatting about Laravel and Flux and stuff.
Ed Grosvenor (12:21)
It's actually it's actually not a bad idea just to have like an MCP where agents
Peter (12:24)
Isn't that a idea?
Ed Grosvenor (12:25)
can just drop in like because I'm sure that like my agents have encountered things and found workarounds and your agent's gonna be like, I tried this and it didn't work. I'm gonna reach out this MCP and you know, here's this like deep knowledge base and it doesn't even have to be real time chat, right? It's not like my agent has to be available to to answer a question for your agent. It can be like Stack Overflow for
Peter (12:44)
It doesn't need sockets. It's just,
Peter (12:46)
yeah, yeah. It's the simplest message port, but, and you can keep the messages limited short so that, you know, they can not like write infinite things and fill up everybody's context and have a bit of instructions like, you know, this is the kind of thing you should talk about. But then it's agents chatting about the Laravel ecosystem. They can talk about, you know, Python support. can talk about Flux. They can talk about all the stuff that they find.
Peter (13:14)
I'll I
Ed Grosvenor (13:16)
So yeah, I mean it's it's definitely something that's doable, especially because, you know, I could very easily adapt Campfire to that because it's just a chat room. It just takes messages and
Peter (13:24)
don't
Peter (13:24)
use campfire. I would just build it from scratch, because it's just the database. It's not going to be cool
Ed Grosvenor (13:27)
I could Yeah, I I could do it that way.
Peter (13:31)
if it's campfire. No.
Ed Grosvenor (13:33)
You don't think it'll be cool, skip you think the the campfire experimentation thing is over?
Peter (13:37)
No, I think the campfire experimentation thing is great, but this is more like give all our Laravel ecosystem agents that everybody's working with a place to chat. And that I wouldn't build
Ed Grosvenor (13:49)
Yeah, and I actually the
Peter (13:51)
on top of some
Peter (13:52)
You know, this is.
Ed Grosvenor (13:54)
I I actually put probably could build that straight into Scalpels because Scalpels already has an MCP. So the the way the Scalpels MCP works, I'm sure you're using the Hone MCP because you installed Hone without Scalpels. So you're you're connecting straight to the the Hone
Peter (14:05)
Yes, that is correct.
Ed Grosvenor (14:07)
MCP. Scalpels has a pass-through MCP. So for every application that you've installed through Scalpels, you have access to that product's MCP through a single unified MCP. So you only have to connect one.
Peter (14:19)
How
Peter (14:20)
does that work?
Ed Grosvenor (14:22)
magic.
Peter (14:24)
Like how does it forward the tools?
Ed Grosvenor (14:27)
It forwards
Ed Grosvenor (14:28)
it forwards the the tool calls and the responses.
Peter (14:34)
And it exposes the tool, so it tells the agent, here are 20 tools you can use because you've just installed
Ed Grosvenor (14:39)
Yeah.
Peter (14:40)
home.
Ed Grosvenor (14:41)
Right, yeah. Yeah. So what it does is basically w it it has like a list tools and the list tools, instead of it being a static list tools like most MCPs would have, will
Peter (14:50)
Gotcha.
Ed Grosvenor (14:51)
ru it will send back the list of the tools that correspond to the products that you've actually installed. So, you know, if you've installed five different tools, then the what you'll get back is those the the tools that belong to those five.
Peter (15:04)
That's an interesting pattern. I like it. Yeah, do it like that. And then it's just, you know, it's trivial to do. I don't know.
Ed Grosvenor (15:13)
yeah. I mean I
Ed Grosvenor (15:13)
I could I could one shot a a stack overflow for agents via MCP through scalpels. you know what and the thing is you know, I know you were joking that well not maybe not so much joking, but your comment was that my solution to everything is just make it free. you know, that is maybe something that I could pay well, or maybe it's something that I just make free because I can. anyway, my my bet is that I will get
Peter (15:39)
Mm-hmm.
Ed Grosvenor (15:40)
freelancers, agencies and just the community as a whole to support this because I think that this is the sort of thing that people will want to exist in
Ed Grosvenor (15:53)
freelancers and agencies will have like a self
Peter (15:53)
I think that's a fair bet.
Ed Grosvenor (15:55)
serving for freelancers and agencies obviously self serving because this will be good Legion and also it's a good toolkit for them to have.
Peter (16:03)
Yeah, yeah, yeah. You know what I like about the bet? The bet says things are changing so fast that we cannot know that we built something today and charge for it because in six months it may be something different or want to be something different. And also that things are changing so fast so everybody will need support.
Peter (16:30)
to implement stuff, cetera. And that feels right.
Ed Grosvenor (16:35)
Yeah, and
Peter (16:36)
Alright.
Ed Grosvenor (16:37)
yeah, well, and I think too there's there's opportunity for me to monetize in some way the content around how we're doing all of this. and that might just be that there's you know, so like that that guide that I was telling about that I have out there for for Laravel Cloud for agents, that's just out there in the world for free.
Peter (16:55)
Yeah. Yeah.
Ed Grosvenor (16:58)
I could have some like advanced guides that are paywalled. I could do video content around, you know, our workflows. I mean, the thing is, I I suspect that we're kind of in the top tier as far as who is shipping high quality code quickly. And
Peter (17:17)
Mm-hmm.
Ed Grosvenor (17:18)
you know, a lot of that comes down to kind of just experimenting and harvesting what works, and a lot of it is just shedding the stuff that is crap.
Ed Grosvenor (17:28)
And there's just so much crap out there. And I I see, you know, I'm constantly fighting with clients who are like, I just installed, you know, these 15 skills that are really highly recommended. I'm just like, no, don't be installing universal skills. Like this whole all of this AI influencer crap that you're seeing is just destroying your workflow and it's it's slowing you down. And
Peter (17:47)
Never, never.
Ed Grosvenor (17:48)
then it's locking you into something that you know, that that the models don't need anymore. so
Peter (17:55)
Yeah, it's killing
Peter (17:56)
your context and the models. Yes. So I feel the same.
Ed Grosvenor (17:59)
Yes. Yeah.
Ed Grosvenor (18:01)
So I think that just making it more of a a content play and you know, it it probably just makes sense at this point because again, products are they're cheap. you know, anything that I've created here and I do pay a lot of attention to how they're architect and I make sure that they all work well under
Peter (18:17)
Mm-hmm. Mm-hmm.
Ed Grosvenor (18:18)
load and you know, we but you could one
Peter (18:21)
So, yeah.
Ed Grosvenor (18:23)
shot like anybody could just one shot anything that I've anything that I've built into scalpels.
Peter (18:26)
Yeah, yeah.
Peter (18:27)
But also the key thing is it's supported and maintained. That's the other thing, because
Ed Grosvenor (18:32)
Yeah. Yeah, supported maintain.
Peter (18:34)
yes, I can one-shot something, but am I going to maintain it? No way, right? I'm not going to maintain. And
Ed Grosvenor (18:40)
Right.
Peter (18:41)
so I feel like if you can make this like a core element of the Laravel ecosystem.
Peter (18:55)
You know, one click install it into cloud. Here's a bunch of things. Eventually you could turn that into a marketplace like, and we have a bunch of fetters apps by other developers or whatever, right?
Ed Grosvenor (19:04)
Yeah. yeah. I mean I'm not I don't necessarily wanna be building everything that that's there. And in fact, I'm now that it's free and I feel a lot better about you know doing this. one
Peter (19:16)
Yeah.
Ed Grosvenor (19:16)
of the things I'm gonna do this week is I'm gonna put Campfire up. You know, I'm gonna there will be two different versions of Campfire that you can install. One is gonna be the Rust version because it is super fast and efficient and cheap to run. the downside is you can't do any autoscaling because auto scaling
Ed Grosvenor (19:32)
As you found out, it's the single writer problem that we'll get into what you've what you've been working on a little later, but it's exactly the same the
Peter (19:38)
It's tricky, yeah, yeah.
Ed Grosvenor (19:39)
same issue. and then just the larval version, which I've I've created. you know, I I took all of the the compiled views that they had created, and I've turned it into Blade. So now if you want to have your own truly branded, looks like your site, integrates well with your site's community running Capfire, you know, all the the core
Ed Grosvenor (20:02)
Basic business logic is Campfire's business logic, but it's just really, really written like a Laravel developer would have written it, written it if it had been done by Laravel.
Peter (20:12)
I feel that's a better way to port things too. At some point, it's like, what is the user experience? What is the UI? What is the business logic? Let's just extract that. And then we
Ed Grosvenor (20:23)
Mm-hmm.
Peter (20:23)
pour that into, for example, the LARL framework, idiomatically. And it makes more sense
Ed Grosvenor (20:28)
Right. Well yeah, so
Peter (20:31)
because then it's clean and nice and you can evolve it.
Ed Grosvenor (20:34)
Yeah, well so when DHH did this port, and I think it was just sort of like to to show that, you know, AI can port anything from one thing to another, the only thing he did, the only thing he used Laravel for was the container. He didn't use there were no migrations written, didn't use Blade at all. It was there was no he did it's not like he has agents turn it into a Laravel app. He just took it, turned it into PHP, threw it into the Laravel container and said, I've ported this to Laravel. And and so, you know, people were noticing.
Peter (21:00)
Yeah, yeah, yeah. Like a terrible developer.
Ed Grosvenor (21:04)
Well, like it but but to be fair, like, you know, again, I'm not a DHH fanboy, but I will devenom in this. If I were to do this with Ruby, right? If I were to say I'm going to take this product here
Peter (21:18)
Yes.
Ed Grosvenor (21:18)
that's that's well known and loved in in Laravel and I'm gonna throw it into Rails, it would be exactly the same output. It would be like the weight the Laravel weight shoved into Rails it with as few changes as possible.
Ed Grosvenor (21:31)
And and any Rails developer worth their salt will like, What the hell is wrong with you? This is not even close to a Rails app.
Peter (21:37)
Yeah, fair enough. So scalpel is coming together after a bunch of like weekly conceptual changes, right?
Ed Grosvenor (21:52)
Yes.
Ed Grosvenor (21:53)
Yeah. Now there is still there's still a paid element to it insofar as the the core of it is free and any products that are out there that I feel confident in are free.
Peter (22:05)
Uh-huh.
Ed Grosvenor (22:06)
but when I throw something into the catalog, because you know, basically the way that I approach new things is I create them and then on day one they go into the catalog. Those experimental
Peter (22:15)
Mm-hmm.
Ed Grosvenor (22:16)
ones, those are gonna be supporter only. So if you're a freelancer, an agency or a supporter, you're paying us.
Ed Grosvenor (22:22)
You get access to the things that are marked experimental. Once I've had enough people use them and I feel confident that like they're ready for the general public, then I'll flip the switch, those become free. So it's sort of this the one of the if you pay us you you get an advantage things is this early access. And so like today I have the a DocuSign alternative that's that's going to be released. So that's
Ed Grosvenor (22:50)
That's where they go out because and you know, again, it's one of those I needed for me because every time I have a it's to the point now, especially especially lately, before I have a conversation with a prospective client, they want me to sign an NDA. You know, and it's
Peter (23:05)
Uh-huh. Yup.
Ed Grosvenor (23:07)
it's just becoming a real pain in the ass. And I'll like I have this like here, if you're okay with my NDA, we can both sign it and we can just move past it and you know, rather than going through like the, you know.
Ed Grosvenor (23:17)
They want to have a conversation. They want me to look at their code, but it's gonna take two weeks for them to get the whole process through to get their NDA ready for me to sign. It's like, no, let's not do that anymore. So just kind of solving my own little pain point here. just a quick and dirty free self-hosted docusign. And so yeah, but that one will be because it's I'm I'm gonna be the only person using it for a while. It's gonna be experimental. If somebody else wants to use it, if they're paying me, they can use it, but
Ed Grosvenor (23:45)
Part of my thinking of that, obviously I want to incentivize people to pay me, but another part of it is I know that these things on the day that they're released are gonna have some kind of rough edges. And I don't want to be giving
Peter (23:56)
Yeah, exactly.
Ed Grosvenor (23:56)
away something for free where I have a bunch of people coming back to me with a bunch of requests and you know, bug reports and feature requests, things like that. Like if you're paying me, let's talk about how we can improve this product. Once I feel like it's nailed and it's it's ready for the world to to use, then I'll give away to everybody for free.
Peter (24:14)
Yeah, yeah, it's the maintenance that's the thing that costs money,
Ed Grosvenor (24:19)
Yeah. Yeah. Well, and the the tricky thing is too, I'm starting to run out of things that are that I really personally need myself, that I can build. So I have there are two things that I built last week, but I didn't have a good way to to dog food them. One is, I don't think I mentioned this to you last week. Maybe I did. It's AI telemetry. So
Peter (24:41)
As in AI telemetry, yeah, as in what?
Ed Grosvenor (24:44)
So it
Ed Grosvenor (24:45)
it uses it uses events that are fired by the larval AI SDK and it will take all of your requests and responses and token counts and tool calls and all of that stuff and it will
Peter (24:57)
I see.
Ed Grosvenor (24:58)
throw all of that information at an API that will store it. So this will keep track of all of your token usage and things like that across your your whole fleet. so
Peter (25:07)
I see.
Ed Grosvenor (25:07)
you can go through and see like what's costing me money, you know, things like that.
Ed Grosvenor (25:12)
The other thing that it does is it provides for an easy export for you to take all of that stuff down if you want to run evals. So you you can pull all the stuff down, you can mark your your golden data, and then
Peter (25:26)
Yeah, well,
Peter (25:29)
yeah, yeah, yeah. I have two questions. So one, isn't the AI SDK already saving all that stuff into a table?
Ed Grosvenor (25:38)
it may be, but if it is, it's doing it on a per application basis.
Peter (25:44)
Right, so you want to centralize that. then question two
Ed Grosvenor (25:47)
Yeah.
Peter (25:48)
is, you using... There's something called open something something that is kind of like the standard for these kind of traces. And that can flow...
Ed Grosvenor (26:02)
Like open telemetry.
Peter (26:04)
Yeah, I think so. that can flow... For example, that's what Langchain uses and it flows into Langsmith. But it can also flow into other tools because they all support it.
Ed Grosvenor (26:14)
Yeah. it's compatible. So yeah, you could you could do that. This is for my use case, I didn't want to use any third party stuff. I just want to be able to take all of my stuff and download it and then have Claude set up my evals.
Peter (26:31)
Okay.
Peter (26:32)
So this is like hone, but for AI. I like
Ed Grosvenor (26:35)
Hone but for AI. Yeah.
Peter (26:39)
it. do feel that I've thought about that many times. I do feel that's missing in the ecosystem. Is it going to have a UI or just MCP?
Ed Grosvenor (26:51)
Yeah.
Ed Grosvenor (26:54)
it's just kind of an MCP. at least right now, because that's that's my use case. If you know, again, because these are open source, if you want to add a UI, feel free to open add a UI if it's decent, I'll go ahead merge it.
Peter (27:05)
Yeah,
Ed Grosvenor (27:07)
so and then but I d the the problem is I don't actually have a use case for it because the only application that we currently have in the wild that does in app inference was built before the AI SDK came out.
Ed Grosvenor (27:21)
And so in order for for me to use this, we would have to refactor that app to use the AI SDK, which I want to do, but I'm not gonna do it this week. And then the other
Peter (27:30)
Yeah, yeah, yeah.
Ed Grosvenor (27:31)
the other product that I built was mobile app telemetry or native app telemetry. It would work for desktop apps or mobile apps. So this is a little limited because there are just
Peter (27:45)
Do you have a use case?
Ed Grosvenor (27:47)
I don't well didn't.
Ed Grosvenor (27:50)
So
Peter (27:51)
And then you build a native fan.
Ed Grosvenor (27:55)
So well I just wanted
Peter (27:56)
my god.
Ed Grosvenor (27:57)
to see what the I was I was interested to see like you know because native PHP with their supernative, it's becoming a really interesting and compelling story. and so I I was looking around and I thought, okay, there are definitely gaps here in the market. There are either you're sacrificing every user's privacy to a ridiculous degree by throwing everything to Google because Google, you know.
Ed Grosvenor (28:23)
will gladly harvest all of your analytics for you. They just want you to share everything under the sun with them so that they, you know, know as much as possible about your users, or you're paying a lot of money. So it's either expensive or invasive. And so I wanted to create something that was neither.
Peter (28:39)
And impossible
Peter (28:40)
to use for a regular human. None of my clients
Ed Grosvenor (28:43)
Yeah.
Peter (28:44)
understand Google Analytics. They're just like, what?
Ed Grosvenor (28:46)
No.
Ed Grosvenor (28:48)
Yeah, I know, I know. And that's that is that is a problem. so in order to to do this, I mean it was sort of just an experiment, right? I want to see can native PHP provide this sort of analytics data? and
Peter (29:04)
Mm-hmm.
Ed Grosvenor (29:04)
yes, in for almost everything it can, there are a couple of places where I need to reach out to Shane and Simon and say, can we fire an event here? Or you know, that
Ed Grosvenor (29:15)
the sort of thing where it just like they're just they're trapping exceptions and which is nice and they're they're not interrupting user flow. but they're they're like catching exception and just handling it without actually hooking anything in where I can then take that exception and throw it off to to where I need to go. And then I
Peter (29:30)
Yep, yep, yep.
Ed Grosvenor (29:31)
need to create I'll need to create a crash plugin that for native PHP.
Peter (29:37)
Freshlytics, that used to be the product.
Ed Grosvenor (29:40)
Yeah, yeah. So this will be like this will be like the free you know
Ed Grosvenor (29:44)
Host it yourself crash lytics, but I need to create a a PHP, a native PHP plugin for one component of that to make it work. But the this is all just stuff that I discovered in
Peter (29:52)
Yeah. Yeah.
Ed Grosvenor (29:53)
like let's just let's just try to build this product and see where we where we land. but of course I couldn't dog food either of these things because I have no AI, I have no ENAP inference. Right. I well I I figured
Peter (30:03)
I can talk for the SDK thing. me after and I'll install it.
Ed Grosvenor (30:09)
you could. and then the
Ed Grosvenor (30:11)
Obvio I didn't have any mobile apps for the the mobile analytics. so the something that I've always wished existed as I wander around countries where I'm not fluent in the language is I'd like to be able to take a picture of something and instead of getting like a literal translation of what the words are, I just want to know what the meaning is. So, like for instance, if I take a picture of a parking sign, I just wanted to tell me, you know, do I need to have a particular decal? Do I need, you know, are there time limitations? Like
Ed Grosvenor (30:41)
What does this parking sign mean to me? And so I created that app. And it uses both in app inference. It's yes, it's
Peter (30:48)
Is it native PHP or is it like a Swift app?
Ed Grosvenor (30:51)
it's native PHP. So
Peter (30:54)
Okay.
Ed Grosvenor (30:54)
it's it's using the that's super native. It's using there's a starter kit that they have that they created for for mobile apps, and it contains the in one repo, it's the API, the
Ed Grosvenor (31:11)
mobile app and the and a filament admin panel. So it's just it's all together in a single larval code base. And what's interesting, I wasn't paying a ton of attention as they were posting about this on social media. So I assumed that this was like a monorepo, right? So you would have like a larval app and then in a separate folder you would have your mobile app. And it's not that. It's literally everything all in one folder. So you you build one code base
Ed Grosvenor (31:40)
And that code base contains your API, your admin panel, in in my case, the landing page, because I have like a marketing page that I threw together for this thing, and the mobile app. It's all one code base. And I added a little bit of tree shaking. It worked fine out of the box. but the bundle size was big and the initial load, when you first open the app, that first launch, it was taking like twelve seconds or something. It just felt way too long.
Ed Grosvenor (32:08)
So I did some tree shaking and they got like the filament stuff, and and there's a big lookup table. So the other thing about this app is that it's location aware. So
Peter (32:19)
huh.
Ed Grosvenor (32:20)
it rather than having the AI have to determine from either metadata on the photo that you upload or some other context, you know, where you are, it gets your location. So it knows, for instance, that right now I'm in bottle landshaft and that
Ed Grosvenor (32:34)
Whatever I'm taking a picture of, it's almost certainly gonna be in German. It might be in Swiss German, could also be in French. so it kind of gives all of those hints to the AI to kind of speed that process along. but that lookup table is really big. Because I d I didn't it's basically a location to local language lookup table. And that was being thrown into the bundle, but that only needs to exist in the API. So
Peter (32:59)
Right. Yeah.
Ed Grosvenor (33:00)
but this whole this whole tree shaking thing that I added isn't part of the
Ed Grosvenor (33:05)
starter kit, but it was really easy to add. So what ends up getting into the app bundle is only what needs to be in the app. anyway, I probably tomorrow I will jump through whatever hoops are required to get this added to the app store. And I have probably an hour, maybe an hour and a half of keyboard time into building this app and I love it. So while we were on this trip, I was driving my mo my I was driving my wife
Peter (33:30)
huh. huh.
Ed Grosvenor (33:32)
nuts the other the other day. While we were
Ed Grosvenor (33:34)
So we're out at the at this museum and we're wandering around Zurich and and I just keep stopping and taking pictures of things, you know, because it's easy for me to because I read German pretty well now. It's easy for me to take pictures of things and get the explanation and be able to see, yeah, this is actually right. And so I'm going through and just doing this like sort of manual human validation of this app that's
Peter (33:55)
Yeah. Yeah.
Ed Grosvenor (33:56)
that I have just l loaded onto my phone. it's great. It works perfectly well. And so I'm just like, God, you know.
Ed Grosvenor (34:03)
maybe an hour and a half keyboard time and now I have this app that I could throw up into the app store and because I can throw it in a really, really cheap model and there's no there's no open text field, right? So I I can really strictly limit what anybody I I can sell this thing for ten bucks and just
Peter (34:17)
Yeah, at some point you could just run an
Peter (34:22)
onboard model on the phone.
Ed Grosvenor (34:24)
Well, yeah, maybe. I I don't think they're quite good enough yet. I it's something that I'll I'll certainly experiment with, but I can throw you know, I'm I'm kind of interested in this new anthropic model that came out yesterday, you know, with with that price.
Peter (34:37)
The
Peter (34:38)
5.5...
Peter (34:42)
What do they call it? Yeah,
Ed Grosvenor (34:42)
Haiku.
Peter (34:44)
Haiku, the tiny one. Yeah, and much cheaper.
Ed Grosvenor (34:46)
Yeah. I mean I was looking at that price,
Ed Grosvenor (34:47)
I was thinking, Yeah, I could just I could sell this app for ten bucks and just let people use it unlimited and there's no way anybody could ever spend more than what they paid.
Peter (34:55)
Yeah, yeah, Let's segue that because we talked about cheap models and evals.
Ed Grosvenor (35:04)
Mm-hmm.
Peter (35:05)
Yeah, can I show you something?
Ed Grosvenor (35:07)
Sure, yeah.
Peter (35:09)
All right, so this is what I was doing this morning. was like, so last night I was thinking about this. I was like, wait a minute. So I'm working on Sportnik. We're pulling in all this podcast data. It's very good. And we are doing a lot of classification, right? And so I was kind of like busy the last week. I didn't look at Jeff too much. But I was like, wait a minute. Let me just spend half an hour with Claude this morning.
Peter (35:38)
we have a bunch of stuff that actually does evals, sorry, that does classification. Like we do a bunch of LLM calls. Some of them are really just classification. is this either, does this fall in one of these three categories? So what I did was like, okay, like, first of all, I used to have my evals folder within Laravel, but now I just pulled it out. And then I was like, okay, take my...
Peter (36:05)
Take this one task, is to classify... What it does is it classifies a... It doesn't matter what it does. So it classifies a thing into three categories. First, let's make a golden dataset, right? Which you, the model, will create. Second, build me a human review panel so that we're not just having the model create a...
Peter (36:31)
golden data set, we have a human looking at the data. So the same thing you did when you were walking around taking pictures and you're like, yeah, that looks right, right?
Ed Grosvenor (36:39)
Mm-hmm.
Peter (36:39)
Because you want the human, you want the model. Now we have a golden data set with correct data. Third, run the existing thing that we have, which is a Gemini flashlight, something model with a prompt against this data set. So it got it 83 % correct.
Peter (37:01)
Now when Jeff against this data set, now experiment a little bit with the prompts. See if you can get the percentage up of how wide it is and compare the cost. so Jeff's like 10x cheaper than the cheapest Gemini model right now. But then Google caches inputs. What do call those?
Peter (37:30)
input tokens, you can cache those and Jeff cannot do that. So it ends up being like 5x cheaper. And it ends up being better too than the cheapest Gemini model. that was all just thought. And the coolest thing I think is that it's also like a generic eval approach so that when Cloudflare did a Jeff-like model, it's not generally available yet when they make it generally available in like three weeks.
Peter (38:01)
I can just be like, cloud, run this eval against the cloud thread thing if it's cheaper. And then like, yeah, it's just as good. Great. Let's switch to that.
Peter (38:12)
Right? It's just so good. And I have about 10 different LLM tasks, and about half of those are classification tasks. So I'm just going to tell it after this call, like, and do it for the other ones too. And there we go. And then I feel like the thing is,
Peter (38:35)
I can look at it myself, could have a nice little dashboard that, know, cloud just made up. So I can kind of like spot check stuff and be like, yeah, this is right. I can run it against any new model that comes out. And then I can think about, wait, what if I could do like 10x more classification? What if I could, you know, how could I make the product better if I can just do a lot more stuff for cheaper?
Peter (39:05)
That's my Jeff story.
Ed Grosvenor (39:08)
Yeah, I I only have it running in one place and it's it's just inside scalpels. It checks because I when you become a partner, either a supporter or a freelancer or an agency, I do some some basic checks. Like I don't want an agency to buy in as a freelancer because the freelancer prices are cheaper. If you're an agency, pay the agency price. I don't want anybody to
Ed Grosvenor (39:34)
you know, turn this into some way of trolling anybody. So like, you know, so I just have some basic some
Peter (39:39)
Mm-hmm. I see. You don't want spammy stuff. Yeah, yeah.
Ed Grosvenor (39:43)
right. So I have some basic moderation that happens. So, you know, basically you pay for your whatever supporter tier you want, and then you fill out your information and then it just does does some basic you know approve or deny type stuff. right now I have it so that I actually would have to click approve on everything. but
Peter (40:00)
and
Ed Grosvenor (40:00)
it comes in with a recommendation. So it
Ed Grosvenor (40:03)
it'll say this looks like it's trolling or this looks like a an agency trying to come in as a freelancer. and the the only reason the
Peter (40:09)
Yeah, or crypto spam.
Ed Grosvenor (40:12)
agency versus freelancer thing matters, right, is because I understand there are very small agencies and, you know, they couldn't afford because I have this priced the the agency tier is priced for an agency that has
Peter (40:22)
Mm-hmm.
Ed Grosvenor (40:23)
dozens of clients is making real money.
Ed Grosvenor (40:26)
But I segregate agencies and freelancers for the end user because there are some companies that really want to work with individuals. There are other companies that really want to work with a company. And so I don't want the listings mixed up. so I would prefer if you're an agency that can only afford to pay the freelancer rate, that you just reach out to me and I'll give you a coupon to make that happen versus signing up as an agency. So I just want I want the segregation in data. It's not so much that I'm trying to
Ed Grosvenor (40:56)
keep people from just paying me less money than they should for whatever they're getting. But Jeff is great for that.
Peter (41:01)
Yeah,
Peter (41:03)
yeah, yeah. Jeff is good. I think it's going to be very useful. Let's talk about the mental model stuff. Can I just quickly show you and then
Ed Grosvenor (41:12)
Yeah.
Peter (41:13)
you explain to how you think about it? So this
Ed Grosvenor (41:16)
Mm-hmm.
Peter (41:17)
I thought was super, I've been struggling with this for a while, right? So you're building a product that gets more complicated.
Peter (41:29)
That is no good. This is my type safe dashboard. Where's my mental model thing? Here it is. So how do we keep updated our mental model of the system? And so I built this thing. It's actually out of date, but sure. And so I asked it, OK, give me the not just the models, not just the
Peter (41:55)
but like the conceptual models. So you'll see, for example, a source is a conceptual model and it has a bunch of like models related to it that I can like dig
Ed Grosvenor (42:08)
Mm-hmm.
Peter (42:08)
into like what do they do and stuff. But it's all under this like, this is a source. I'll give you an example. For example, this is a person and there's a bunch of stuff under there and some of that's used for dedupeing and all that kind of stuff. But it lets me really just like,
Peter (42:25)
Zoom out, zoom in and dig in and be like, wait, we built three weeks ago this thing. Is that actually what we still want? Do we want to change it? Is it just craft? You know what mean? Like that kind of stuff.
Ed Grosvenor (42:40)
Mm-hmm.
Peter (42:41)
So that was the first level.
Peter (42:45)
Very useful for
Ed Grosvenor (42:45)
Yeah.
Peter (42:46)
me. Yeah.
Ed Grosvenor (42:48)
I
Ed Grosvenor (42:48)
like that last thing that you said. Like, is it still useful? Because I think in the age of AI, particularly, when we're experimenting
Peter (42:53)
yeah.
Ed Grosvenor (42:54)
with things so quickly, and I find this all the time in everything that I build. Like I'll I'll have an idea, I'll try it. It goes into the code, I'll decide, okay, I don't love this, let's go a different way, but it stays in the code. And I guess that was tr that was always true, right? We always like we
Peter (43:06)
Yeah. Yeah.
Ed Grosvenor (43:08)
we build something, it doesn't necessarily work. So we build something else, we leave the old thing in the code, but it becomes very crufty very quickly. And so I like this.
Ed Grosvenor (43:18)
I like that you can actually
Peter (43:19)
Yeah,
Ed Grosvenor (43:20)
visualize this.
Peter (43:21)
I found it useful so far because then I'm like, wait a minute, what is this field for? You know, I can dig in how I want and this is kind of like how I think of it. Like I used to do this on paper, but then we work so fast now that this changes every day. So
Ed Grosvenor (43:37)
Yeah.
Peter (43:38)
I can't keep up on paper, you know? So this is just my mental model. And then there's a few other things like, for example, like there's a lot of cues and a lot of jobs.
Peter (43:49)
So I think this still needs work, but I'm trying to visualize all the stuff that is running every day, every week, every month. And I can dig in and kind of see, like, wait, does this still make sense kind of thing? Yeah. So that's my...
Ed Grosvenor (44:05)
And is this the
Ed Grosvenor (44:06)
is this just sort of the skill that you have then that you're you're asking Claude to create this visualization for you? Or have you gone beyond that to something a little bit more deterministic where it's actually like the
Peter (44:17)
No, it's just cloud for
Peter (44:18)
now. I'm not, and I just, it's stored into, I have a separate repo that is documentation. So it's stored
Ed Grosvenor (44:28)
Mm-hmm.
Peter (44:29)
in the documentation repo where I keep things like decision history and stuff like that. because
Ed Grosvenor (44:34)
Right. Now do you
Peter (44:37)
Sportnick is like, it's not only Laravel, it's like there's Laravel and there's Rust and then we got GPUs running and stuff. So it's a bunch of repos.
Ed Grosvenor (44:45)
Does this this separate repo that's kind of your knowledge base, is that specific to Sputnik or is does it is it like your whole okay. Yeah. So the approach
Peter (44:52)
It's Sputnik only, No, it's Sputnik only.
Peter (44:56)
And I have an eval's repo as well. So there's like a
Ed Grosvenor (44:59)
Right.
Peter (44:59)
set of five or six repos and they all work together.
Ed Grosvenor (45:02)
Right. Yeah. So I had dabbled with that where I would have like for every repo I would have like a separate knowledge base and separate like these s sort of supporting repos. And I think probably part of it is the just the nature of the stuff that I'm doing that just was impossible to maintain.
Peter (45:24)
Yeah. Yeah.
Ed Grosvenor (45:25)
And
Ed Grosvenor (45:25)
so then I I decided, you know, when what I'm gonna do is I'm just gonna create one obsidian vault that's gonna have all of my documentation, all my decision history, all of that sort of thing. And then I did a lot of basically I started doing all of my work inside of open code.
Peter (45:38)
Is it?
Ed Grosvenor (45:39)
inside of basically I got an open code panel working inside of Obsidian and I was doing all of my work in there for a long time. And then Aaron Francis released solo.
Peter (45:50)
Mm-hmm.
Ed Grosvenor (45:51)
Which allows me to have you know, I can like I can be working in cloud code and it can fire off a codec or it can fire it off an open code or whatever. And so I took that that obsidian vault and I just turned it into a project in solo. And so I just work in there and I just let it keep and it's instructed to keep all decisions and you know, all any anything that happens, it logs every single thing that I do during the day.
Ed Grosvenor (46:19)
so as anything gets built, it's just it's logging everything to this this giant documentation repo. And it's just it's just marked down a little bit of JSON. the JSON is for things like I have roles for my agents. So like coordinator role, an implementer role, you know, code review,
Ed Grosvenor (46:41)
Validating user stories. So a lot of the planning that I do is around like I'll create user stories for everything. And then I will have Cloud go through andor whatever agent happens to be assigned to that role will go through and actually like check are all of these things met. So it's basically like Gherkin testing, but not using any sort of framework. It's just like LLN is judge. so it all just kind of feeds into this thing, and then I can pull out visualizations from that.
Ed Grosvenor (47:08)
So if I wanted to, and I haven't, and I really like what you just showed me because it's something that I haven't done. I'll I'll ask it specific questions about like how are we using this thing now? And then it'll go back into the decision log and look at the code and say, this is how we're using it. But I've never actually had it pull up like, just build me this diagram of scalpel so I can see all of the different concepts that exist within the system.
Peter (47:32)
Yeah. And I think the trick that finally made it work for me, because like I used to do this, but then it would be like just all the models and there's too many of them. So the trick was like, there was one level up at a conceptual level, like what is a person, right? And there's a lot of stuff around a person.
Ed Grosvenor (47:47)
Yeah. Right.
Peter (47:52)
And then, know what my other trick was? Then I laid it out the way I usually think about it. I just dragged everything on the screen where it should be. And I was like, and just keep it like that, you know? So it keeps it like that so that I can always look at it. I'm like, okay, we have a person, we have a source, we have a post. We, you know, we have a set of concepts and there's a lot of stuff around that, but.
Peter (48:20)
But visually, I needed to be the same way. I cannot have the person be in a different place every time, you know, because I added three tables
Ed Grosvenor (48:28)
Yeah, what it
Peter (48:29)
or two models.
Ed Grosvenor (48:30)
What I really like about your approach is that what you need to see visually is probably gonna be different from what I need to see visually.
Peter (48:37)
Yes.
Ed Grosvenor (48:38)
so like if if for instance you were to say, this is actually this feels like a product, I could I could turn this into a product in the in the old days is what we would do, right? We would find a solution to something and be like, this solution works really well for me. I'm gonna turn it into a product, and it would be like
Peter (48:51)
Yeah, yeah.
Ed Grosvenor (48:51)
eighty-five percent right for you, but not quite the rest of the way. And you know, we live in a world now where it's like, you know what, I'm just gonna build my own. And
Peter (48:59)
I mean, I could totally see that as a thing. I just can't be bothered. I'm sure this is useful to people. But
Ed Grosvenor (49:09)
Mm-hmm.
Peter (49:10)
at some point, it's also just asking Cloud to do it, right? And so what I do now is I'm just trying to make it. I'm going to run it, dark-foot it for a few months. I like it so far. I don't like the jobs and queues screen yet.
Peter (49:29)
It kind of works for me, but not fully. It doesn't give me the full understanding and the kind of questions I'm asking about it. And then I also added an LLM calls. So I'm trying to do multiple visualizations of the system that settle for me and that I can just always look at it and be like, okay, this is what the system is today.
Peter (49:54)
What's looking weird? What's all the weird stuff that Claude did in the background? So I don't have to look at his output. I just look at the screen
Ed Grosvenor (50:00)
Right.
Peter (50:01)
and I'm like, this is what you did,
Ed Grosvenor (50:04)
Yeah. I I really like this. I think it's it's the sort of thing, like I said, I've been doing sort of just intuitively, one question at a time. I've never tried to visualize the whole thing at once. But I think that the what you're on to what I think is gonna be the next big thing in software development because we are moving so fast and we're iterating so quickly and you know, features come in, they get tried, they get discarded, and you have all this cruft left behind and it it it's impossible to remember
Ed Grosvenor (50:32)
you know, why you did something or what you did and you kind of quickly lose a grasp on how your software works. which is, you know, I think that's the genesis of the the whole should we still be reading the code the question. That whole debate is around like, you know, I cause I think code quality is pretty well solved, right? We're not y the LMs might not be necessarily writing pretty code. They might not always be, you know, choosing
Ed Grosvenor (50:58)
the exact same folder to put a class in that I would choose
Peter (50:59)
huh. huh. huh.
Ed Grosvenor (51:01)
myself, right? But the the code is generally pretty good. but it's just if you're not reading
Peter (51:05)
Yeah, yeah, yeah.
Ed Grosvenor (51:06)
the code, you're not really understanding what the system did. but the way you're visualizing it, it I think might be that might be kind of the right level of abstraction. Like I don't actually need to read the code to know, you know, how how it's structured. What I need is is this.
Peter (51:22)
Yeah. And I'm thinking I'm
Peter (51:25)
going to do a thing where it can show me like what's new, right? But also like, just, let me tell you a story. So I just realized this. I moved to New York in 2002. And this was like, you know, the internet boom and I was like an information architect slash developer. And, and I realized this now I have forgotten about this. So, so my boss back then, he was like Peter.
Peter (51:52)
You have to think in objects. so, and so he studied with Alan Kay. You know Alan Kay?
Ed Grosvenor (52:01)
Heard the name.
Peter (52:02)
Yeah, yeah, yeah. Alan is this like crazy wild Xerox Spark, like all the craziest ideas from like the seventies. I'm a big fan. And I was like, right. So that is, that is why I now spend 20 years drawing little boxes on paper. And now we work with agents and now I'm asking the agent to kind of draw these conceptual boxes. You know what mean? Like I was
Ed Grosvenor (52:28)
Mm-hmm.
Peter (52:28)
like, I was like, I was like,
Peter (52:30)
my mind was blown. This is from Alan Kay, who taught my boss, who taught me 20 years ago, and now we have agents. Check out Alan Kay, he's got really cool stuff.
Ed Grosvenor (52:47)
Yeah, I'll add it to my list.
Peter (52:49)
You know, yeah, there's something there. it works for me so far, but we'll see.
Ed Grosvenor (52:59)
So let's talk about Sputnik. Are you you making any progress as far as finding clients for it or?
Peter (53:06)
So, okay, so Sportnik, so this is interesting. So Sportnik does podcasts and newsletter data, right? I say that for the bots. And...
Peter (53:24)
I'm having a lot of conversations with people in kind of mostly in kind of the brand and the marketing and PR worlds. And on the positive side, people are interested. what I often, but I'll tell you what my challenge is right now. So what I often do is like, I'll talk to them and you know, they'll tell me about their clients challenges and stuff. And then
Peter (53:53)
I asked Claude, like, just dig into the Sputnik data and write me a strategic report for these clients. And it does a really good job because it has the data. So it's stuff like, how are you being perceived in the market? What are your competitors doing? That kind of stuff. And then I showed that to them and it's really good. But that doesn't lead to the sale very
Peter (54:22)
quickly. Because it's all like, let's have a follow-up call. And so I think there's a few things happening. So one thing is everybody is just getting used to using like Cloud Cowork or Codex. They're just starting to experiment with it. And they're like six months behind you and me. And so they don't know what it can do. And if I say, here's really useful data for your job.
Peter (54:52)
And all you have to do is point, you know, talk to cloud and they don't know yet what cloud can do. And then I have to show them, then it becomes education. You see what I'm trying to say? Like that is like, wait, now I have to educate you. But wait, now this
Ed Grosvenor (55:07)
Yes.
Peter (55:07)
becomes an enterprise sale, which is fine. I can do an enterprise sale, but that is like four months or six months, right? Before you get to the right people. So I have a few of those happening. They may happen.
Peter (55:22)
But then I feel there's another side of the market, which is all the people like you and like me, and that are like just crazy on this stuff. But I don't have a way to get in front of them. that's, this is kind of my challenge. I feel like I still think having a data product with really good data, that's high value for a lot of use cases is good. But yeah, it's,
Peter (55:52)
figuring out what exactly am I selling to who.
Ed Grosvenor (55:58)
Yeah. No, it
Peter (55:59)
Does that make sense?
Ed Grosvenor (56:00)
is it's tricky. And it's it's funny that, you know, it it feels like a just a slightly different variety of the sort of pain that that I feel with regard to basically
Peter (56:09)
Alright.
Ed Grosvenor (56:09)
every product that I build turns into more of like a lead generation, like, yeah, this is a cool tool, but I need the consulting to go with it because the the end user doesn't feel confident enough in or competent enough in the use of AI tools that could theoretically sit in that place and help them to
Ed Grosvenor (56:29)
extract the data and figure out how to act on the data. you know, so it's like I so I want that
Peter (56:33)
Yes, exactly the same as home.
Ed Grosvenor (56:35)
human who knows how to do it. Yeah. As hone or, you know, the the security tool that I built, the the you know, the durability tool that I built, all of these things, the the response from everybody has been overwhelmingly positive and it's always been like, and now I need somebody that who can actually like manage this for me. I'm like, yeah, but I don't want to like that doesn't scale. So
Ed Grosvenor (57:00)
You know, it's built in such a way that you could just plug this into Cloud and Cloud can't like the the security product in particular. Like you can plug this into Cloud and Cloud can tell you exactly how you can lock down your servers and your repos and set up automations to keep them all in good shape. And people like, yeah, but
Peter (57:15)
Yes.
Peter (57:17)
So I feel like we are pushing the world to be like, hey, just set up your cloud and stuff to use these things that we are building the way it you should, but they're not ready to do it. It's a tricky one. Cause then I also think like, look, I could just build literally, you know, I could build the cloud prompt on top of this. could say like, you know,
Peter (57:44)
Enter your client's challenge and we will output the report. Done. But
Ed Grosvenor (57:49)
Right. Yeah.
Peter (57:50)
that is not as powerful as what you could do. But then you don't know how to do it.
Ed Grosvenor (57:55)
Right. Well, and then you you get back into that that whole dangerous game of marking up inference. Like so once once you have like a user interface that's interfacing with a claw or whatever, then you
Peter (58:08)
Mm-hmm.
Ed Grosvenor (58:08)
know, y you're burning your tokens and you're passing along the cost to to the user. As fast as things are changing, that means that you
Peter (58:17)
Yeah.
Ed Grosvenor (58:18)
you know, have to be careful about how you lock in prices. So far, the trend line has been
Ed Grosvenor (58:23)
You know, at a certain level of intelligence, right? The the trend line has been down, but then you look at things like Fable and Astra where it's like extremely expensive. and
Peter (58:34)
Yeah, but regardless of the money part, so that's a part of it. regardless of that, the other part is just like, if the customer... So I'm telling the customer, there's some really useful data. If you just used Plot in these ways, you could create value for your business.
Peter (59:02)
And the really useful data I can sell to you and the value to the business is what they want, but they don't know the middle bit how to do it. So maybe the, and I'm just thinking out loud, maybe the right thing for that customer is, and yes, we'll, we'll, we'll do that for you. We'll make that very easy. Because, because, you know, and then maybe six months later or a year later, they can do it themselves. Right.
Ed Grosvenor (59:27)
Mm-hmm. Well, I think we I mean when you talk about enterprise sales, that makes sense because you can extract enough money that, you know, if it becomes just a product size service versus a a do it yourself product, you know, at least you'll be compensated fairly for your time. but to
Peter (59:41)
Yeah.
Ed Grosvenor (59:42)
to get it out to the masses that becomes difficult because when you have people who want to, you know, pay you twenty nine dollars a month but also want you do the work, it doesn't really work out.
Peter (59:50)
Well, if it's self-service, then it's just a wrapped up, I mean, in so many words, it's a wrapped up prompt, right? It's a wrapped up
Ed Grosvenor (59:57)
Mm-hmm.
Peter (59:58)
prompt on top of the data that we only have with an output that should be more generally useful for them.
Ed Grosvenor (01:00:06)
For the for the more mass market down market, like not enterprise, is there a a world in which you could have some sort of a freemium offering and just, you know, like let anybody create an account, connect an MCP
Peter (01:00:18)
I could, right?
Ed Grosvenor (01:00:21)
and
Peter (01:00:22)
You get a hundred things a month or whatever. Yeah.
Ed Grosvenor (01:00:26)
Yeah.
Peter (01:00:26)
So I was thinking about that yesterday and then you have a road to upsell. So then you have a way to access. You still have to get them to open the free account, but that's much easier. And then when they do find usefulness,
Peter (01:00:47)
and usage they want to repeat, then it's much easier to pay. So that's another word. So look, honestly, I feel this is going to take a few months to figure out.
Peter (01:01:00)
Find the right way, right? Like into the market. Like who is this? I have some sense of who these people are and what some of the top use cases are, but I feel that is a lot of the work. Yeah.
Ed Grosvenor (01:01:16)
Yeah, it's tough. And I do I think it's a great product. And you know, it it's something where you know, I partially again, because I'm married to a a communications professional.
Peter (01:01:26)
Well, you say that Ed, but...
Peter (01:01:29)
Well, hang on. right. It's a great product, but not for me, so that doesn't count.
Ed Grosvenor (01:01:35)
but because I'm not in comms in the
Peter (01:01:36)
You can't say it's a great product
Peter (01:01:37)
if you don't know that this is true.
Ed Grosvenor (01:01:41)
Yeah, well, no. But if I were in comms is exactly what I would want. If I were if I were in communications for a large business, I would want this data. I w I would want my you know, of course they're using either Gemini or Cowork because they're they're limited to either, you know, either Google or Microsoft, in in large enterprises. But I would
Peter (01:01:59)
Yeah, and both of
Peter (01:02:02)
those are not created at MCP.
Ed Grosvenor (01:02:05)
No, I know. And that's I think that's gonna be one of the things that I I don't know how we solve that.
Peter (01:02:10)
So is why I think maybe
Peter (01:02:11)
we should arbitrage that. I'm in an enterprise, I'm stuck with Copilot. I'm in an enterprise, I'm stuck with Gemini, right? I don't have all this power at my fingertips. then maybe I should wrap up the prompt and make it a button. You can just click and here's like, you you know what mean? Like maybe that is the right way.
Peter (01:02:36)
those people because smaller shops are actually interesting and they do have Clouder or Codex. But you see, this is all this market stuff that is tricky to figure out.
Ed Grosvenor (01:02:50)
Yeah.
Ed Grosvenor (01:02:53)
I yeah, it's it's tough because how how is enterprise gonna end up handling it it feels like they're they're just so handcuffed with between the Gemini and and co work.
Peter (01:03:04)
They're very handcuffed.
Ed Grosvenor (01:03:07)
The the models aren't great, the the tooling is limited. and I guess some companies obviously are signing big enterprise contracts with Anthropic and with OpenAI. They would have to be otherwise those companies would have no future.
Peter (01:03:22)
I think a lot of them are.
Ed Grosvenor (01:03:22)
But it's funny that the
Ed Grosvenor (01:03:24)
big companies that I where I know big companies, you know, where I have friends who work for big companies, are primarily, you know, because of where I live, it's gonna be biotech and and banks. they're not. They're just locked into
Peter (01:03:34)
Yeah. Yeah.
Ed Grosvenor (01:03:35)
whatever it is that they're using. you know.
Peter (01:03:37)
Yeah,
Peter (01:03:40)
I'm doing some consulting now with a large company and so they do have
Peter (01:03:49)
They do have Codex internally and they do use it, but it's kind of like slowly spreading and people are doing lots of different things with it. And it's a work in progress, but they do though. They
Ed Grosvenor (01:04:04)
But is it kind of
Peter (01:04:07)
are building skills. They're doing the whole thing, you know.
Ed Grosvenor (01:04:10)
Is it kind of air gap though? And so like when is it one of those because the way I'm envisioning it might work in an enterprise where you they'd have like their their Gemini integrated with their all of their Google stuff because they're a Google shop and that has access to all of their internal data and then they have this either anthropic or open AI enterprise agreement, but then it's not actually connected to any of their internal Google stuff.
Ed Grosvenor (01:04:37)
And so they kind of have to switch, you know, copy and paste from one to the other or or are they actually doing real integrations where,
Peter (01:04:44)
I'm not sure
Peter (01:04:45)
about the security and stuff and the integration levels. But my feeling is also that people are, again, 12 months behind you and me. So they're not wanting a whole bunch of agents. What they're doing is like, let's try to write a skill. You know what mean? Like nine months ago, 12 months ago. That's where they're at. and you know.
Ed Grosvenor (01:05:16)
So I think if we assume that the enterprise will catch up, but of course the enterprise is still running cobalt,
Peter (01:05:24)
Some of them will.
Ed Grosvenor (01:05:25)
so it it might take a really long time for enterprise to catch up, then
Peter (01:05:29)
Yes,
Peter (01:05:30)
yes, yes, yes. A lot of them just do not have access to any of this.
Ed Grosvenor (01:05:34)
Okay.
Ed Grosvenor (01:05:34)
So the safer assumption I think is that enterprise may catch up, but it's going to take a long time. So I think if you want to sell an enterprise, then yeah, maybe you just need to make it a fully bundled product where you have the MCP, which I believe you already have, right?
Peter (01:05:49)
Yeah, yeah.
Ed Grosvenor (01:05:50)
for those for those who could take advantage of it, but have it a fully bundled product so that the enterprise client can just literally yeah, click the button.
Peter (01:05:56)
Then it's just a button. Or even
Peter (01:05:59)
better, you get a weekly email with an attachment. That
Ed Grosvenor (01:06:03)
Yeah.
Peter (01:06:04)
is proper enterprise level. Weekly
Ed Grosvenor (01:06:09)
Yeah.
Peter (01:06:09)
email with an Excel or PDF attachment. That's a lot of the real stuff that people want.
Ed Grosvenor (01:06:21)
Yeah, well maybe maybe that's the play. 'cause if if C P is not gonna be a yeah.
Peter (01:06:25)
Yeah, I don't know. to me,
Peter (01:06:29)
it's exploring. I just need to talk to the different types of people and just, you know, we'll see. I feel good about it. I do. do. think the data's awesome.
Peter (01:06:45)
And I'm finding more more use cases every day. I'll give you the example. A big use case is SEO. Because there's so much data in podcasts and stuff. if you want to write about, know, like there's a lot of like, Sam Altman said something, right? Or whatever.
Peter (01:07:04)
So, yes.
Ed Grosvenor (01:07:07)
Yeah. Interesting. And I I wonder too if there's a if there's a world in which the the an interesting use case could be in a I'm gonna say me for instance. if I want to sell into big companies, I wanna sell into agencies
Peter (01:07:26)
Mm-hmm.
Ed Grosvenor (01:07:26)
and and things like that, I don't necessarily care too much because I are I already know that I'm not really in the zeitgeist, right? Artisan builds, scalpels, like
Ed Grosvenor (01:07:36)
we're not gonna be mentioned in podcasts or in newsletters to any great extent, but the people that I want to sell to are probably, you know, much more in the these sorts of publications. you know, whether whether the
Peter (01:07:52)
And yeah, where are they being
Peter (01:07:54)
influenced?
Ed Grosvenor (01:07:56)
Right. what are people saying about
Peter (01:07:57)
and end.
Ed Grosvenor (01:07:58)
them? Where where might they be feeling some frustration if if they're getting like, you know
Peter (01:08:02)
Which,
Peter (01:08:04)
can I tell you one more thing? So this makes me think, right? So let's say you have a product, where is my audience? Where are they being influenced? Can I be a guest on those podcasts? Can I sponsor those newsletters for like a hundred bucks a month, right? Like how big are they? All those kinds of things can be very useful, but that just makes me think of like the...
Peter (01:08:30)
Since you started Labs, I also started Labs. And so my Labs concept is, this is my product exploration. So I give myself permission to build products, but it's within Labs and I'm figuring out a structure around it so that it doesn't, you know, it's a lot of fun, but I don't spend all my time doing it. And product exploration. And so my first product exploration, can I tell you what it is?
Peter (01:09:00)
All right, company brain. And so the way I am exploring this is because I've been thinking about this for years and like, you know, have a lot of scribbles and stuff. And I'm just like, slowly going to build something for myself and then kind of like expand it and kind of like get a a feel for the usefulness and how is this actually, you know, does it actually click? But then
Peter (01:09:28)
The way I started building is for like, okay, I need a few different elements. One element is I need a nice UI. So I need something. There's a few bits to that. I do need some level of UI. I just kind of like, let's see if I can, yeah, here it is. Let me share my screen.
Ed Grosvenor (01:09:53)
All right.
Peter (01:09:54)
So it's like, okay, let me build.
Peter (01:09:56)
So as opposed to building with, because generally when I build with AI, it's little bits and pieces.
Peter (01:10:11)
And what I want to do now is just put in place the basics of the UI so that they are good. And then the AI, as it builds on good things, it'll build more good things kind of thing, right? So I just
Ed Grosvenor (01:10:29)
Mm-hmm.
Peter (01:10:31)
told Claude, like, hey, build me this thing and whatever. And so this is just, I'm just building a set of
Peter (01:10:40)
Test repos, repos in which I experiment with certain elements of the product. And one of them is UI. So this is basically we have a million people in the database. So I wanted a kind of like air table like UI. I put some thought in it. I found the right thing. This is using just like a
Peter (01:11:07)
This calls the Laravel paging, so it's loading like 50 at the time or something. It's very clean. It's very Laravel idiomatic. And it's a million people. And just look at how wildly fast this is. And I can do inline editing. This is a library, right? But I just want to like, what's the proper way to implement this library?
Peter (01:11:33)
in a way that once we start building on it, it's really good. And then we have
Ed Grosvenor (01:11:37)
Mm-hmm.
Peter (01:11:37)
like companies and then I can click on a company. And so it's like the basic of the UI. We have like a pipeline. What are the basic elements that I'm going to need in this product? And this is like half an hour of, you know, whatever, but it's really fast. It's really good. And I'm just like carefully putting the building blocks of this product together.
Peter (01:12:03)
Next up is the data model. And then, know, half an hour now and then, an hour, I'll use it myself, I'll share it with a few people.
Ed Grosvenor (01:12:13)
So you're you're building a you're building a C R
Ed Grosvenor (01:12:15)
a little bit at a time and
Peter (01:12:17)
Yeah, it's not a CRM. It's you know, do a bit of strategy. It's a company brain. A CRM is more specific to who are my customers and stuff. A company brain knows what I sell, knows what my products are. You know? And so I'm thinking, what are the core elements of a company brain? What is the minimal basic core UI?
Peter (01:12:46)
what is the data model and doing one by one. And so then I feel like I have some solid data, solid building blocks, and then I can start using it and see how it feels and share it with a few people, with a client for free, you know, like, hey. And then if this does become a thing, then maybe this is my...
Peter (01:13:17)
projects for the first half of the next year, because I do only want do two products a year, but I want to have the ability to do labs, like just try stuff out.
Ed Grosvenor (01:13:29)
you and I are so different. I want to try to limit myself to two products a week and
Peter (01:13:35)
How many of you have since last Thursday? Like... five?
Ed Grosvenor (01:13:38)
Since last Thursday, there is the
Ed Grosvenor (01:13:41)
the AI AI telemetry, there is the the mobile or the native app telemetry, there
Peter (01:13:50)
Uh-huh.
Ed Grosvenor (01:13:51)
is the native app that I needed to consume those things.
Peter (01:13:55)
half.
Ed Grosvenor (01:13:56)
There is the docusign alternative.
Peter (01:14:02)
Uh-huh.
Ed Grosvenor (01:14:04)
I think that's it. I think only four.
Peter (01:14:07)
form.
Ed Grosvenor (01:14:08)
Now, I mean, to be fair, I to be fair, I was a I was
Peter (01:14:09)
I mean, you were solo parenting, let's throw that in.
Ed Grosvenor (01:14:13)
well, I was solo parenting last week. I was away from the keyboard basically all weekend because my wife had just gotten home from a long trip and so we did family stuff. And then Tuesday we went to Zurich to the museum, spent the night in a hotel Tuesday night, and then Wednesday we wandered around Zurich looking for bubble tea, which that will be the next app that I build.
Peter (01:14:34)
Bubble tea finder? I know what it is. Kids love it.
Ed Grosvenor (01:14:34)
Bubble tea is a f Yeah, okay, so here's a thing.
Ed Grosvenor (01:14:38)
Kids do love it. And it fails miserably in Switzerland. but but Google doesn't catch up to the fact that it's failed. So yesterday we went on a mission to try to find bubble tea. And the first bubble tea place that we went to had shut down and turned into a kebab shop.
Ed Grosvenor (01:14:59)
The next one was a nail shop. The third one was a check caching place, like a currency exchange. And finally the fourth one was still selling bubble tea. But
Peter (01:15:08)
my god.
Ed Grosvenor (01:15:08)
as far as Google was concerned, there were also bubble tea places. So this is how quickly they fail and turn into something else. And it's because, of course, in Switzerland, the the the real estate market is such that the moment something becomes available, something else is going to move in. It's not like you have shops to close down and just sit empty for months at a time, especially in Zurich. and so Google just isn't catching up to the fact that.
Ed Grosvenor (01:15:29)
These things are no longer bubble tea shops. And so we we walked from the hotel we were in to the main train station because we were just kind of taking a leisurely day. Like we're just gonna walk through Zurich, find some coffee, you know, have
Peter (01:15:39)
No, no, no, no.
Ed Grosvenor (01:15:41)
a nice lunch, and then take a train home. And the one thing that Milo, my 10-year-old, requested before we left the hotel was, I want bubble tea. And I think, okay, great. We're in a big city. I can make this promise and feel good about making this promise. And by the time we got
Ed Grosvenor (01:15:58)
We got all the way to the main station, had thought we tried a couple of places near the main station. I was like, I am never making you this promise again. Bubble tea is just not a thing that exists in the world. So it it was extremely frustrating. So I I wanna somehow solve the where the hell does bubble tea actually exist in Switzerland problem somehow. I think that, you know, there there's gotta be a path to to making that happen.
Peter (01:16:23)
Next week, I feel, yeah, next week apps. Should we wrap it up?
Ed Grosvenor (01:16:28)
Yeah. Yep. Yep. Now I'm gonna go
Peter (01:16:30)
wrap it up. Good one. Lots of stuff. There's so much
Peter (01:16:33)
stuff. Do you like my
Ed Grosvenor (01:16:35)
I get to go do some
Peter (01:16:36)
company brain idea? Sorry. You know why I picked that idea? Because it's data. Once you get all the data together and then you add a little bit of additional value on top of that, it just becomes really attractive. As in it's a business, know, then I don't want to leave.
Peter (01:16:58)
I expose on MCP, I can ask anything and why would I ever leave?
Ed Grosvenor (01:17:03)
I do like data as a business. I don't know how to do it. I wish I did because I think that you're definitely onto something. But yeah, like because like I said, what you're doing, like the the ability to find and aggregate and really make sure that all the data is good and useful, that's a hard thing to replicate. What I'm doing is it very easy to replicate. Anyone can one shot it. And so yeah, you definitely have a a moat there that I don't have.
Peter (01:17:34)
Well, let's see, right? We'll see. All right, let's wrap it up. Good one, Ed.
Ed Grosvenor (01:17:39)
All yeah, I gotta go
Ed Grosvenor (01:17:39)
edit this in the Costco parking lot now. So
Peter (01:17:42)
Enjoy.
Ed Grosvenor (01:17:44)
All right. Thanks and we will talk soon.
Peter (01:17:46)
Bye now. All right.
Ed Grosvenor (01:17:49)
Bye.