Subscribe
Share
Share
Embed
AI After Dark is a podcast hosted by Alex Gras, venture capitalist at Mercury, focused on how real companies are built once the hype fades and the hard decisions begin. Through candid conversations with founders, CTOs, and operators, the show cuts through buzzwords to talk about people, systems, risk, and the trade-offs that actually matter. Alex brings a background as an operator, founder, and revenue leader, with a belief that technology matters, but people come first. This podcast is for builders who care less about trends and more about what lasts.
00;00;00;07 - 00;00;29;12
Unknown
Actually, I want to start with that. I was like, you started with like ChatGPT. Point five all right. Right. And and just that progression. I'm not just like seeing it, but also incorporating it and and building with it. Like, how is that effective what you do with cook list. It's fundamental. Even before ChatGPT, you know, really the kind of interest started in 2013 when DeepMind was releasing the first results with Atari.
00;00;29;14 - 00;00;56;19
Unknown
And then a couple years later with AlphaGo and, you know, this is a promise that we've had since the 50s that we're going to have AI that can think like a human. And we've kind of had, you know, several, hype and winter cycles. But, yeah, around 2015, 2016, it seemed like we were on the verge of another breakthrough with AI.
00;00;56;21 - 00;01;20;07
Unknown
And I don't know if for some reason, to me, it just seemed very profound that if you could learn to build and apply AI, that can then be applied to anything else you want to do. And so, it's one of the most fundamental skills to try to develop as a builder. And then, you know, with cook List, we've kind of stayed on that frontier as the models have gotten better.
00;01;20;07 - 00;01;39;15
Unknown
That's allowed us a new build to build new capabilities. Has it, has it changed like how you've designed your team? Have you had to basically reorg? I had a I had a buddy that he's like, well, we're we're not an AI native company. We you know, we don't necessarily we use it, but we don't necessarily need to be truly AI native.
00;01;39;17 - 00;01;55;09
Unknown
And then like in a heat of despair, calls me late at night. I was like, dude, I like, I have these young, you know, engineers coming to me, you know, saying, hey, I've worked on everything you asked me to work on. I'd love a title bump. I love a, you know, hey, Bob. He's like, none of it.
00;01;55;09 - 00;02;12;10
Unknown
None of it matters anymore, right? And so I can't fire them because they've done everything I've asked them to do. But I also can't keep them because they're not right for what it needs. You know, an organization in the AI world needs to look like. Yeah, I mean, it's got to have happened to you 3 or 4 times over.
00;02;12;12 - 00;02;46;09
Unknown
Yeah. I mean, in the early days, a lot of what we were doing was we had humans that were doing like data labeling or matching, essentially a lot of core of what we do is to understand how products can be used in the context of recipes. So like what is a product? And, we had a team of people who would kind of review very like simple matching techniques, like looking at just string and substring matching and kind of doing this very manual process of tuning those matches and finding negatives and things like that.
00;02;46;12 - 00;03;09;25
Unknown
And now, you know, I can do that better than a human can reason through it. From a developer standpoint, though, and I think this is more broadly of when we talk about how is AI changing, how a company operates or what the structure should be. There's kind of levels of using AI is a tool to kind of assist existing workflows and make them more efficient.
00;03;09;28 - 00;03;36;08
Unknown
And then the companies that, you know, kind of go one step beyond that is it's more of an AI first workflow where, you're truly trying to almost remove the human, bottleneck from the processes that the company needs to operate. And then that really kind of opens up a new paradigm for how you design the company's internal systems and processes and everything.
00;03;36;11 - 00;03;58;16
Unknown
I will say on the junior engineers, though, you know, they're some of the best adopters of AI. They can be, at least from what we've seen. And, you know, we have one on our team who uses more tokens than anyone else. So do you have a leaderboard? Leaderboard of who uses. Yeah. We've we've you know, we we keep an eye on it.
00;03;58;19 - 00;04;30;18
Unknown
And it is interesting. Right. It's it's not like, it it's almost, it's a proxy metric. Right. Of course you don't want to just say we're incentivizing you use as many tokens as possible, like making as many commits as possible to try to measure a developer's productivity. But it is directionally, you know, there is some signal there as far as, how much work or effort, how much energy are you consuming, right, to put towards the goals of, advancing like what the company is trying to build?
00;04;30;20 - 00;04;58;21
Unknown
Yeah. So one thing I would have a doubt, as maybe I say this as a parent, more so than like a builder is when programing is so easy or cheap. Let's call it, how do you make sure that you have you're hiring people or building critical thinking skills? Yeah. That is so a saying that we've had and this goes back even kind of pre AI days to previous companies was never delegate understanding.
00;04;58;23 - 00;05;20;12
Unknown
So what I allows you to do today is you can outsource the thinking, you can outsource the work. But we try to kind of build a culture around it to never delegate the fundamental understanding of how and why something is happening that we're building and is actually great at helping you get to a level of understanding faster.
00;05;20;12 - 00;05;49;10
Unknown
Right? You can come into a new code base that you're not familiar with, and instead of the traditional way of, you know, developers kind of having to document things, and then those are always out of date. You can ask the question of explain how this works and then just start diving in and saying, like, okay, explain, you know, this piece, in more detail and then you can kind of use it as a mentor and a tutor, to test your understanding, to say, like, am I thinking about this, right, that this works in X, y, z, right.
00;05;49;16 - 00;06;11;13
Unknown
Ideally, you want to have those fundamental kind of skills to be able to go look at the code yourself and do that verification yourself. But I think that's the key piece is making sure that whatever it is, the path you're going down with AI that, you is kind of the orchestrator of these AI systems has that fundamental understanding.
00;06;11;15 - 00;06;31;14
Unknown
I remember the first time I met, we were talking about cook list as a platform, right? Which at the time was like, hey, we're you know, we're helping people understand what they're consuming. There there was like hardware component to it, right where they're scanning in their, or any barcodes of anything that they've eaten, all in an effort to be a Trojan horse to the data.
00;06;31;14 - 00;06;52;16
Unknown
Right. It was like people don't seem to care about privacy when it comes to their food. And so how has that helped right now, in a day and age where, like, proprietary data set is everything? Yeah. I mean, I wouldn't say people don't care about the privacy. I think privacy is important and has a place. But yeah, I put words in your mouth.
00;06;52;23 - 00;07;26;14
Unknown
Yeah. I think, I think what, the interesting thing though, is, is the utility that can come from bringing your data together. And so, you know, for a household that's busy and they have lots of demand on their time and, they have goals, right? But, it takes a lot of work to put together a meal plan or, you know, do this very, like, meticulous tracking of everything you're consuming, having ways to kind of bring data together from multiple sources and aggregate it.
00;07;26;14 - 00;07;55;18
Unknown
All right? There's a lot of convenience and benefit to that. As an individual, but then also as a business to have, first party data that you can then use to create really personalized experiences and, build something where it truly feels like, they understand or you understand these, like very nuanced food preferences because, you know, traditionally you'd have to kind of be like, okay, out of these five diets, which one are you on?
00;07;55;21 - 00;08;14;12
Unknown
How many people are in your house? And I don't know, like pick from ten cuisines. But if you look at the reality of how people eat, there's so much more nuance, right, of these subtle things of like, I don't eat fish at home, like I like fish, I'm not allergic to it, but I just prefer not to eat it at home.
00;08;14;12 - 00;08;48;27
Unknown
I don't want I want it to smell that way. Right? Or, you know, we have kids that are home sometimes. So the amount of groceries we have to buy depends on what our schedules are going to look like. Yeah. And so I think that's what's kind of always been our driving vision is, aggregating the data together and then building AI systems that can extract insights and really start alleviating, the work and the mental load from people as they're making these decisions about what groceries should I buy this week or what should I cook for dinner tonight?
00;08;48;29 - 00;09;08;03
Unknown
It's got to be the jump from like, ML to AI, right? Because it's it's your point when it's nuanced, there's not. The logic is a lot more complicated. Right. And it's not consistent where X goes to Y. At what point did you, did you really feel that unlock or was the unlock happened to be prior to Ben building cook list?
00;09;08;06 - 00;09;51;10
Unknown
But, I mean, we do still use some traditional ML and, you know, I think the, the most recent unlock has been this step changing capability of the models to operate in an energetic fashion. And so, you know, from very early days, we kind of you can use traditional email for doing kind of classification. And then even some of the smaller, like GPT models are, were good at classification, much better than traditional ML and a lot more flexible, where we've seen an unlock kind of over the last 12 months has been this, ability to run in a genetic loop and actually have, an AI system that can have multiple tools
00;09;51;10 - 00;10;10;03
Unknown
at its disposal and then intelligently decide which tool should I use to handle this response from a customer or a user? And, reason over the response that the tool returns. Maybe it's going to search for products. It sees that some of those products that were returned, like none of those are a good fit. So do decide to do another search.
00;10;10;03 - 00;10;25;27
Unknown
Right. And this is truly where you differentiate between like kind of a traditional system or even an AI workflow. And then a generic system is you have this open ended loop or you're not actually sure what the agent is going to do or how many tools it's going to call before it's ultimately returns the response back to the user.
00;10;25;29 - 00;10;50;27
Unknown
And, yeah, seeing the reliability and capability of that over the last four months has just been incredible. Yeah. Which is fascinating. I know you all started maybe more B2C now it's B2B, especially in a legacy industry like grocery, or food retail. It feels like the buyer actually had a preference towards a model coming back and saying, I don't know.
00;10;51;00 - 00;11;17;25
Unknown
That faded very quickly. Now people expect outcomes to be almost perfect. Is that what you're sensing? Yeah, it's an interesting design space because, in traditional digital products, we have a UI that shows you what your options are. And now with these kind of AI firsts, or especially the ones that are using natural language, the interface is just a text box.
00;11;18;00 - 00;11;45;16
Unknown
So any question that is possible could be asked or submitted to their agent. You know, whether it has the capabilities or not. And this is a big part of what we've learned of how we bring these systems to these large enterprises. And then our deploying them at massive scale is what are the appropriate, kind of guardrails, both on input and output.
00;11;45;18 - 00;12;04;29
Unknown
How do you build the confidence at an enterprise that this system is going to do what we say? It's going to do? And, you know, it's it's inherently, you know, stochastic, right? So we're going to get it could be right 50% of the time. It could be right 80% of the time. It could be right 95% of the time.
00;12;05;01 - 00;12;34;24
Unknown
What's been a really big learning is how to build the confidence and basically build the eval suites. So we can give a number to say, okay, this model's getting, 95% pass rate. And here's the cases that it's failing on. And when it does feel it fails in this way, you know, we do have this huge kind of, tailwind of the hyperscalers putting tens of billions of dollars, hundreds of billions of dollars really into, building more and more capable models.
00;12;34;24 - 00;13;01;27
Unknown
And that is showing where we're seeing way less cases of hallucinations and, the ability for the model to navigate, scenarios that we didn't previously prepare for sometimes in really creative ways, too, which is always fun to discover. But so, like you as a CTO and co-founder builder, have your expectations changed? Expectations of what? I mean, like, yeah.
00;13;01;29 - 00;13;29;07
Unknown
Fair. So in one vector, like the underlying model you've described, how, you know, those have been a big unlock in terms of what's capable with your product. But I guess you could also look at it from like an organization size perspective. You could also look at it from, a go to market perspective. I feel like there's been just so many transitions and it's that cycles happening so fast that I'm getting to a point where, like, I don't know what to expect anymore.
00;13;29;09 - 00;13;47;05
Unknown
And and it makes me nervous that I don't have as, especially as an engineer that don't have like some site to sit on and say, okay, I just need to target this and build towards that. And once I'm there, I can, you know, look over that peak and see what the next valley, you know, that I get across looks like it just feels like peak after peak.
00;13;47;07 - 00;14;13;04
Unknown
So starting in a point where, you know, even with the Atari game, you have a good understanding of like, what good should look like? Try now where not only is the expectation that it needs to be great, but you don't really have an idea of what great could look like. How do you build for that? Yeah, I mean, it's it's actually going back to that, been fairly incredible.
00;14;13;07 - 00;14;44;29
Unknown
Ray Kurzweil, you know, I had a few books that came out and plotted relatively similar to Moore's Law. Right? Plot of these relatively simple algorithms. But if you follow the exponential, the the outcomes can be very counterintuitive. And so it is surprising that, you know, I think his original prediction for singularity was 2029. And we seem to be, it's going to be fairly close to that.
00;14;45;02 - 00;15;05;22
Unknown
And so, I don't know, I think, you know, this goes back over the last decade, we've seen this same exponential playing out, and it doesn't look like there's a sign of it slowing down yet. And so I think that's kind of the thing to bet on as far as what to expect, is that the models are going to continue getting better.
00;15;05;24 - 00;15;29;25
Unknown
Our intelligence per dollar is going to continue increasing at this like exponential pace. And so then you kind of have to think about as you're building the product, the company, the business, will this be disrupted by the model being ten times better or will this be enhanced? Is this a tailwind to us, or is this going to be something that kills us?
00;15;29;27 - 00;15;53;00
Unknown
And when you, you know, making those kind of key decisions for the company of what direction are we going to invest in and what path are we going to invest in. It's that one where there is going to be defense ability, in this case, for the models ten times, 100 times better. It does create some really interesting scenarios that were at some point it's like, is this thing just going to consume all software?
00;15;53;00 - 00;16;13;24
Unknown
Is there any room like is there any opportunity if you have this ultimately superintelligent, you know, AI system. So yeah, it does feel like there's this quote, right. It's like business is just bundling or unbundling and it goes through these cycles. It feels like we're trying to do both at the same time. And and you have people specializing in certain things.
00;16;13;24 - 00;16;40;05
Unknown
And then but you have these foundational models that are just like absorbing, absorbing, absorbing, and not just from a technical capacity, but from a financial scope. I mean, all the money is flowing to them, right? And so when they, they don't have the competency, they could just acquire it. And so I, I, I think this is a challenge that every founders facing, every company is facing, how do you feel like you approach it.
00;16;40;07 - 00;17;04;17
Unknown
Yeah. And this, this kind of, you know, as part of the cook list out the original promise of it was fine recipes to cook with the groceries you buy. Right? So, I think I've seen actual ads from Anthropic of. Take a photo of your fridge and we'll, tell you some recipes to make. And I think, you know, we've kind of seen this play out, right?
00;17;04;17 - 00;17;26;25
Unknown
The smartphone displays dozens of different household appliances. It's a similar kind of analogy that's happening just in the digital space. But that core promise of the original Cook List app, you know, you can build now, you don't even need to build it, right. The models just do it by uploading a photo. And so then the question is like, okay, well where's their defense ability?
00;17;26;25 - 00;18;08;28
Unknown
And I think that's where one thing we've identified is, the physical groceries still have to come from somewhere, right? There's incredibly huge and complex supply chains. That actually, you know, move from the ultimate farm where things are produced through the CPGs, distributors retailer to the consumer's home. And so shifting our positioning where, we're now building these AI tools, but applying them in the environment of a retailer, kind of no matter how good any of these AI systems get, we're still going to need the physical atoms of groceries to come from somewhere.
00;18;09;01 - 00;18;39;07
Unknown
And so if we can, kind of be that partner to a retailer to build the orchestrator, to help a household decide what to buy, we think there's some lasting value there. And then on the other side, you know, we we also are doing some things, with some appliance partners for helping establish better tracking at home and kind of think of we think of it as a way to provide context to your AI assistant that's doing your meal planning or shopping.
00;18;39;10 - 00;18;59;13
Unknown
So think of like a smart fridge that is keeping track of what you have in your house. So, you know, you don't have to go and take a photo of it yourself every time you want some help figuring out what groceries you need. Yeah, yeah. It's interesting, I think of, parallel with, smart. I see another company in town, and so I had met Rob larder on and he was talking through.
00;18;59;16 - 00;19;19;11
Unknown
Right. They started as this consumer device that you could put into your AC vents to measure all these different variables and quickly realize, like you and I, as homeowners, don't care of 23 different variables. We just care whether it works or doesn't, you know, or saving money. You're like, yeah, exactly. I don't need. And, but the person who does the Hvac contractor, right.
00;19;19;11 - 00;19;57;21
Unknown
Because they need to have a better understanding of us as homeowners to especially when we have maintenance contracts to know when they actually need to roll a truck versus when they can resolve it, you know, from, from afar, remotely. And, and when they can preempt situations. Right. And, and build better customer loyalty through that. Similarly, what you're describing is like, hey, we can build this infrastructure layer, this orchestration layer that helps, us and, you know, given our understanding of the consumer, be able to provide, a layer of, you know, knowledge, let's call it, or judgment on the intelligence about the consumer to all the supply chain that is going to
00;19;57;21 - 00;20;16;14
Unknown
be providing these items to. Right. Is that first? Okay. Yeah, that makes a lot more sense to me now. And, and and to your point, I, I, you know, I can't, I can't eat compute. So I definitely, yeah, yeah. One day I know, I know, I your will be in the matrix and and just absorbing that.
00;20;16;17 - 00;20;57;08
Unknown
That's fascinating. So, when did that realization hit? How has it been fairly recent, or was it something that you guys had starting to see kind of percolation of early on and started building towards the realization that these models are kind of consuming all software? Yeah, it it's definitely, I guess, been as the coding capabilities have progressed to such a high degree, where on the one hand, it's accelerated our team's ability to create new software, and move at a much faster pace.
00;20;57;10 - 00;21;22;10
Unknown
But the thing to keep in mind is everyone else also has those tools. And, even some of these consumer facing sort of experiences where it's like prompt to app, right. If your thing that you're building is an app, anyone can get an app from a prompt. You know, it seems like maybe the, sustaining value of your service, unless you have some other.
00;21;22;15 - 00;21;51;05
Unknown
Right. There's other, defense ability things you can have, like network effects or, but ultimately the age we're in is the price of any content is going to zero. And then the quantity of content that's out there is unlimited. So if your business is content, if it is, you know, a recipe that is like delivering that to the right person, it seems like that's going to be a hard business to be in.
00;21;51;11 - 00;22;10;20
Unknown
Yeah. There's just there's unlimited competition. Yeah. Well, that's what I keep seeing. These apps that provide, you know, Surface Insights. And I was like, insights are cheap now. Right. And at first I would say even like three months ago I would find the best use case, at least for me in a chat based LLM is like, hey, help me identify things I'm missing, right?
00;22;10;20 - 00;22;32;04
Unknown
Like, I think I know what I'm think I know and show me what I don't. And that was great because then I could, you know, pare my judgment on to that, those insights. To your point, I think then being able to disintermediate the judgment pieces, that next layer that is scary to think about that in a foundational model could do that.
00;22;32;06 - 00;23;00;25
Unknown
Because, yeah, I'd like to think that, oh, only I have had my own experiences. Only I understand the dynamic amongst my peers, my family, my professional career, whatever. And therefore only I should be making decisions about what to do with this. These insights. But maybe not. You know, maybe right now that's the shift, right? Yeah. Of using it kind of is like a copilot where you're still the one analyzing the data and then taking the action to make a decision.
00;23;00;27 - 00;23;34;29
Unknown
And then now we're kind of, you know, with, this era we're entering to that's more a genetic is you're actually delegating that decision making process where the agent is the one taking actions on your behalf. And you know, you can see this play out across many different areas. But fundamentally right. That's like what's happening is instead of it being that copilot that's kind of giving you more context and then you're still the one that's actually acting, you're kind of setting the higher level goal and then having an agent act on your behalf.
00;23;35;01 - 00;24;11;23
Unknown
So how do you train understanding going back to your point, right. Like, like if, if we've we've disintermediate intelligence potentially judgment. You know, next layer is understanding. How do you. Yeah. How do you train for that? I think you have to build it as a part of the culture to, question each other's understanding. So if someone tells you something to not just take it at face value and to ask the why and then seeing, are they able to produce ROI, that makes sense.
00;24;11;25 - 00;24;41;11
Unknown
Or you can usually fairly quickly realize that they just have the answer. They don't know the why. And so then that's where, you know, we try to teach that as the learning opportunity. When we when we talk about how do we train kind of our team and just to build the culture around, spending the extra mental effort to get to that point of understanding because, yeah, you know, the, you know, who knows, maybe we won't need to know that one day.
00;24;41;14 - 00;25;00;13
Unknown
But, as long as is, we want humanity to have some, you know, agency in the direction our future is going, it's important for us to understand why. It's why the things that are happening are happening. Yeah, I, I was I was talking to someone, and, and we were talking about, like, right, like, what do we think?
00;25;00;20 - 00;25;17;14
Unknown
Like, the most popular major in college is going to be in ten years, right. Because, like, maybe in five years, it's still ML engineering of some flavor or something like that, you know, what would it look like in ten years? And I think we kind of so is like like philosophy and art, like humanities like that.
00;25;17;14 - 00;25;45;23
Unknown
Those will be the most critical things that help us question why, understand why, grow from that, that mental effort? I think it's it's very artistic and poetic to say it, to think that, I'll be curious to see if that's that's the direction we go. Yeah. I mean, if you go to Midjourney homepage, the art there is incredible.
00;25;45;25 - 00;26;08;10
Unknown
And it's actually I go to some museums now and kind of disappointed, we're not able to keep up with how good the, the AI art is, is, but do you get, like, emotion out of that art? I'm the same way. Yeah. Okay. I mean, it. Well, emotion is all tied to stories, right? And so it's the, the story that's behind the why of, like, why is this impactful?
00;26;08;12 - 00;26;37;24
Unknown
And so, I guess, you know, is there there is a lot of human story that has carries a lot of meaning and is able to connect with people. But at the end of the day, I don't know, I kind of view AI as a tool. It's like if it's great artists made that, with some finger paints on a cave wall, or if they made it with Photoshop, or they use Midjourney to make it, what's the story behind it?
00;26;37;24 - 00;27;00;25
Unknown
Right. What inspired them to enter the prompt? Or, I mean, now it's so much more than a prompt, right? If you're really trying to achieve a specific creative outcome, it takes an incredible amount of work. Just like you could give a paintbrush to, a four year old or, you know, a great artist. The tool is the same, but the outcome can be completely different.
00;27;00;25 - 00;27;19;24
Unknown
And with many of these AI models, it's a similar thing. You know, at the base level, yes. Someone with, can enter a very simple prompt. You get something that looks decent, but the gap between, their capability and then some of these people who are the best in the world at using these AI tools, is extremely vast.
00;27;19;28 - 00;27;50;06
Unknown
And that's also kind of where there is still, I would say some degree of, of opportunity. Right? Is essentially there's many companies and we're one of them that specializes in taking the foundation models that are kind of trying to, be a general intelligence that can be applied to any task. But then engineering a harness that allows that same foundation model to achieve a much higher level of capability.
00;27;50;13 - 00;28;14;29
Unknown
Right. And this is where the, the harness or the context engineering. And you see this with even some of the coding models where, you know, cursor will say that their, their harness is better with, you know, clod than, the cloud code harness. And so, yeah, I think that's, you know, that's a question of like, how long is that going to last?
00;28;14;29 - 00;28;36;03
Unknown
But that has been a, you know, the AI wrapper kind of thing has been a saying for, I don't know, 3 or 4 years at this point. And there's still a lot of really big, really defensible companies that you could argue are an AI wrapper. But, you know, SAS is also an AWS wrapper and Nvidia is a sand wrapper.
00;28;36;03 - 00;29;03;26
Unknown
So yeah, I mean, yeah, yeah, yeah, it was a great analogy. Well I, I yeah. So I guess said differently which things these financial models go, take the concept of an inch deep, a mile wide and maybe they're, you know, ten miles wide and a few feet deep. And so we just if we just zoom out and, like, think a little bit more exponentially, with this, like, mindset of abundance and like, yeah, you should be.
00;29;04;02 - 00;29;30;03
Unknown
It doesn't matter if, you know, these financial models eat the world. That just means that we have the capacity to maybe go deeper than we've ever thought before with the right harness, with the right perspective, with the right context. Maybe. Or maybe they get so good it doesn't matter. Right? That's what they are on that trend. So, you know, just like you kind of have, the frontier labs and open source labs are lagging behind.
00;29;30;03 - 00;29;55;01
Unknown
Maybe, you know, 6 to 12 months, a fully optimized harness. And, you know, Ark AGI is probably like a good example of this, where people run the Raw model and then they have specialized harnesses that run on the same eval. And you can kind of see that differential and like, what difference is the harness making. But even the raw models are making huge improvements, you know, no harness attached.
00;29;55;03 - 00;30;29;28
Unknown
So it's it's a, it's a there is opportunity there. But how long it lasts, you know, TBD. And why does it like if there's an improvement in the raw model, why doesn't that also improve the harness equally as much. It it can in some cases, I think, you know, a big part of it is like typically the harness is trying to identify where there's gaps in the model's capabilities.
00;30;30;00 - 00;30;52;03
Unknown
And then either, provide the right context or kind of right direction where the foundation model, you know, maybe makes some sort of silly mistake. Right? Is, you know, simple one. Like one of the latest ones, has been, you ask, should you walk or drive to the car wash that's 150m away, right? Most of the models say walk.
00;30;52;06 - 00;31;15;20
Unknown
So you could build a harness that says, like, when you encounter this situation, say, drive, right. It a simple analogy. That's essentially what a harness is doing is it's for a specific, problem domain, kind of giving the model the context it needs to make the right decision. And so in a similar way like this is where you kind of have to understand like what is the harness doing.
00;31;15;20 - 00;31;41;23
Unknown
Like what gaps is it filling. There's other other things a harness can do like bring in company specific domain knowledge that is maybe proprietary. So the foundation model would never be trained on it. Or, you know, in the case of grocery, right. We're bringing in the product catalog and recipe catalog of like real time inventory. Right. And a model, a foundation model is never going to know the real time inventory available at a grocery store, because it's always changing every hour.
00;31;41;25 - 00;31;57;15
Unknown
But those little like, you know, if the main thing your harness is doing is filling that knowledge gap of just saying when someone asks you about a car wash, say, drive, then, you know, a later generation of the model is probably going to figure that out on its own. Yeah. But I mean, like you mentioned understanding, right?
00;31;57;15 - 00;32;11;04
Unknown
Like what's the purpose of going to the car wash? Right. If it is to wash your car. Yes. But if you're going to pick something up or meet a buddy, I don't know, like, you know what I'm saying? Right. There are layers to this, which goes back to your point of, like, asking why? Like, why are you going to the car wash?
00;32;11;04 - 00;32;39;22
Unknown
What's, you know, and, and and I guess what you're saying is no foundational model is going to get to the point where it's asking, why am I not saying that? I mean, the the latest models, you know, and this is part of, this is like an example of a good harness, right, where the coding models, instead of just taking whatever prompt you submit at face value and going off to the races, some of the harnesses will now actually hold the model.
00;32;39;24 - 00;32;58;19
Unknown
Take into account what the user asked for, but then identify, you know, 2 to 3 areas where there could be ambiguity or, you know, the intent maybe isn't crystal clear. And so then the model asks you, what are you trying to do? So before it goes off and builds it, reasons over that. Thinks about what?
00;32;58;20 - 00;33;22;25
Unknown
What are they trying to do? Why are they trying to do it? And then we'll actually come back to you and to consult you to say, hey, like, what did you mean by this? Like, which way should we go? So I think that models absolutely have the capability of asking why. I think if you're the human who is directing and orchestrating the model, though, you know, you need to know why those decisions are being made, in some cases.
00;33;22;25 - 00;33;49;27
Unknown
Right. There's, you know, very, of how, you know, some bits are allocated in memory. You don't need to know how the compiler is like optimizing those decisions of why at that level. Right. So, I mean, yeah, if you're doing some kind of crazy performance optimization, maybe. But, for most people who are building kind of these higher level business services, you know, that that right level of understanding, I guess, is, is important.
00;33;49;27 - 00;34;26;12
Unknown
Do you feel like the human or the user is now the bottleneck for the reasoning of the model? Yes. Yeah. I mean, you should try to be removing yourself as the bottleneck. So in this case where, right. Like, well, what we're seeing is this capability where the, the systems can kind of go, like, if you push the autonomy slider all the way to Max and you say, like, how long can this thing make meaningful progress on its own?
00;34;26;14 - 00;34;51;27
Unknown
And what a lot of these like, latest eval show is that even after, like wall clock time of, many hours or even dozens of hours, if you just keep letting the reasoning process run and the model has a good, objective that it's optimizing against, it's essentially able to do like what gradient descent does, where it can just keep pathfinding.
00;34;52;04 - 00;35;15;16
Unknown
And I think actually, yeah, there was, you know, a recent, auto ML library that came out, which was essentially this encapsulated as a harness or a prompt of, if you give a problem like, you know, finite optimizations to make this run ten times faster, the model. And as long as the test suite, it doesn't let it cheat.
00;35;15;18 - 00;35;31;13
Unknown
The model can opt out. It can sit there and churn away, try out different strategies, and actually, make a surprising amount of progress where if you were trying to insert yourself into that loop and each time it wanted to try an experiment, if you had to say yes or no, right, you're going to dramatically slow that down.
00;35;31;16 - 00;36;06;18
Unknown
So as long as you can clearly define the outcome of of what you want the model to do, and then just send it and let it work away. You know, this is kind of a shift that has to happen. And like how I think as product builders and developers, we think about structuring problems. But, you know, at the end of the day, it just allows you to move up these like layers of abstraction where, you know, we're not trying to be in the loop of, like, reviewing every turn of the model, with a code change suggestion.
00;36;06;18 - 00;36;28;18
Unknown
Right? We want to describe a high level feature or optimization, let it work for a day, and then come back and just, you know, review the final PR and understand, what actually worked right after trying out 100 experiments. Yeah. You've just unlocked something. I, I going back to my humanities comment, right. In terms of like, you are engaging with these models in a very natural language way.
00;36;28;20 - 00;36;46;04
Unknown
The model is limited in its understanding, in your capacity to explain what you want. Right. And so the more you have to iterate on that, the longer it's going to take you. And so in a world where things are speeding up, the faster you can help the model understand what it is you want to accomplish. Yeah. The better.
00;36;46;04 - 00;37;15;02
Unknown
Therefore the better you can communicate, the faster you can produce. Yeah. Which is wild to think about. Yeah. I mean, this is it's also this, interesting trend that, markdown is essentially becoming the new programing language, which is just an English textile document, essentially. But it's, it's, you know, portable. And there is, a lot of like, it works with every elm, it works with lots of tooling, and it's human compatible.
00;37;15;02 - 00;37;30;25
Unknown
Right? The computers learn to speak our language. Not everyone's gonna learn to speak the computer's language. And, yeah, I think that's absolutely it is. If you can be. There's kind of two things, right? One, you need to be able to be concrete in your goals and in what it is that you actually want the model to do.
00;37;30;27 - 00;38;04;21
Unknown
But then the other important piece is understanding what the kind of jagged frontier of model intelligence looks like. Like what can it do well, and what can it not do well, and this is something you only build up an intuition for, I think by being a practitioner and actually using the models and trying different things out. The crazy thing, though, and like the running joke for, I know the last five years probably with our company has been whenever something doesn't work, just wait six months and then it'll start working.
00;38;04;23 - 00;38;26;02
Unknown
Yeah. And so but truly, like that's happened time and time again where we are kind of pushing up into, product experience or workflow that we're, we're trying to build and use AI for. And it maybe it works like, you know, 10% of the time or 20% of the time. And then 1 or 2 model generations later, now we're at, you know, 80, 90%.
00;38;26;02 - 00;38;55;01
Unknown
Yeah. I shipped, let's call it and, you know, in quotes, something this morning for Mercury, you know, on the VC side, which was like we, you know, every fund, gets a ton of inbound, you know, from from people especially, I think macroeconomic people are finding entrepreneurship to be like a form of job security, almost. And so you're getting more and more creators, obviously, they're able to build better things or just build period in a way they weren't beating before.
00;38;55;04 - 00;39;16;00
Unknown
And so we have this like shared inbox and everybody kind of shares, you know, swords in their with their comments about a company. And then like I, I would go put that into our CRM and first it was me and then I like outsourced it to you know I offshored it basically and then it was like, okay, we can do it, you know, with, with a mix of internal teams to kind of cut that cost.
00;39;16;02 - 00;39;32;11
Unknown
We tried and then to try to build some kind of, you know, almost Zapier type connect, to feed it into that air of all kinds of stuff. And, and finally, I, I was like, this is just dumb. Like, this is a huge waste of brainpower. Like, we're just like, how long does it really take to, like, put this stuff in?
00;39;32;11 - 00;39;54;13
Unknown
But candidly, we were limiting how much data we were capturing because it was still being manual. In a weekend, with cloud code, I now have it integrated to, you know, between Microsoft into our CRM. And then to be able to scrape any attachment, any, you know, context from the body. Barb, link it to the right things.
00;39;54;13 - 00;40;09;06
Unknown
And now, like, not only have I gotten it to work, but it's open my mind, just like, what are their attributes should we be tracking to accomplish other things? Right. Like, what am I missing? What did like when when I looked at an opportunity, I didn't invest in it and end up doing really well? What you know, what did I miss?
00;40;09;06 - 00;40;34;14
Unknown
And, like, what else should we be thinking about from like, you know, that are more qualitative. That isn't just quantity driven. And how someone writes their email and how they put a deck together. And it's, it's just like, to your point, just like, look back and I'm like, I sent a note to the team was like, I'm rationally excited that this is that I just built this because we spent a year trying to put this together.
00;40;34;14 - 00;41;00;07
Unknown
And now I could do it in a weekend, you know, like, yeah, it's insane. Yeah. The thing you you said there too, about where your mind went, after initially getting to work for the existing workflow to work, I think is a really important thing for, people to consider. Is, what work is there that was previously just prohibitively expensive or time consuming?
00;41;00;08 - 00;41;26;02
Unknown
Tedious? That does have value, but it just it wasn't worth the cost of having someone on your team do it before. And this is where there's, I think, also a really big opportunities to start expanding that, that, mindset out of now that we have very cheap intelligence. You can start doing things that just you would never have spent the human time of doing it before.
00;41;26;02 - 00;41;52;04
Unknown
And, you know, I think in this in business, there's that case. But then I think also, you know, in, you know, your, your personal like, life, like we think of this in, you know, how people make decisions about their food is sure. There's some people, you know, celebrities or whatever that have, nutritionists on staff that kind of plan out their meals and help them decide, like, what they're going to eat to optimize for their goals.
00;41;52;06 - 00;42;16;07
Unknown
But most people, you know, can't spend that kind of money on, doing that for their, you know, personal lives. But everyone would probably want that if it was affordable enough. And so I think that's something else. Is like, almost the market expanding for things that just weren't previously possible because it didn't make economic sense. Yeah.
00;42;16;09 - 00;42;24;07
Unknown
So what, when in your life did you realize, like, you were going to be building software?
00;42;24;09 - 00;42;56;08
Unknown
Later than I, I wish I had realized if I could, could go back. I've always been a builder, so from, you know, 2 or 3 years old, I guess, like taking things apart and, building, like, physical things. And I was always kind of drawn to technology and electronics. But I didn't actually ever kind of have anyone around me who, you know, I like built computers, but I like, built the computers to play games on them.
00;42;56;08 - 00;43;44;03
Unknown
It wasn't to, program them. So, you know, I didn't really learn to start coding until, like, much later, like, early 20s. And it was the I had attempted a few times to like, learn and, you know, had built e-commerce businesses and, kind of worked in web technologies. But it was really around that, 2013 time when it seemed like this next AI wave was going to come, that that's when I went kind of PhD mode, if you will, and, was just consuming lectures and reading whitepapers and, and, really the first thing that I learned from a programing perspective was how to work with data, train ML models, and
00;43;44;03 - 00;44;07;21
Unknown
then some of the, like, you know, very early kind of mNIST style of AI models. And then that that foundation then is what gave me the ability to, I think there was a website coding for entrepreneurs, right? Because I was already an entrepreneur. And then I had kind of started learning to code. And so I had the vision for the product that I wanted to have built.
00;44;07;23 - 00;44;36;11
Unknown
And was like, okay, now what do I need to to learn to be able to, to build that? I think this is actually something that, you know, retrospectively also, I didn't fully understand the value of software or building your own software early on. You know, I had read some books and they were all like, oh, well, you can just outsource any software you want, and have it built for really cheap on these, you know, kind of aggregator platforms that have freelancers.
00;44;36;14 - 00;45;04;08
Unknown
And so, you know, my initial, I guess, education around what software should be or how software should be built was it's something to try to just outsource for really cheap and, you know, I think if we look at, you know, the existence proof of, you know, how a lot of the greatest software in the world gets built, and, you know, why are all these software engineers paid so much in California?
00;45;04;10 - 00;45;48;16
Unknown
There is a legitimate value to, people who understand that the deep why of the problem of why are they building that software, especially in the early stages of an entrepreneur's, journey? I think, you know, having that direct like nothing is, is coming from doing a hardware product and then coming into software, the ability to, be using my own product and encounter a bug, and then immediately you just go to my computer, fix it, push that fix in under ten minutes is just, such an incredible feedback loop where if you think about if you have someone overseas who's working opposite time zone to you, you then have to not only,
00;45;48;18 - 00;46;12;28
Unknown
try to communicate all the context about what that bug was and why it matters and how the how the fix should look, to them and then wait for that iteration time. It just really slows down the ability, to innovate and, improve your product. You strike me as someone that, like, never wants to feel like you say, I don't know the answer to that.
00;46;12;28 - 00;46;33;25
Unknown
Like, right. Yeah. You never delegate understanding. Yeah, yeah, yeah, yeah, yeah. I've been recently in a few meetings where, I, I as I'm learning, you know, through a firehose a little bit, had to do that. It's it's frustrating as hell. And it it's, it's, it's awesome to be able to, like, have all I remember everyone when I, you know, internet obviously.
00;46;33;25 - 00;46;58;08
Unknown
But like mobile phones especially the iPhone came out I was like, oh, you've got all this knowledge at your fingertips. But it's still there was still friction towards accessing it. Right, right. And and now, I mean, it's just so frictionless that it's, understanding what to do with all this knowledge, I think has been such, an amazing, you know, exercise in, in just brainpower, but also in, like, where you put it understanding the deep.
00;46;58;08 - 00;47;23;23
Unknown
Why that's cool. So, like, we touched on it a little bit, but like, what is quick. Let's do it now. Today. Yeah. So today cook list builds, an urgent ecommerce platform for grocery retail. So what that means is we have, kind of three core products that sit under or are powered by our platform. The first is an AI shopping assistant.
00;47;23;23 - 00;47;46;10
Unknown
So this is, you know, in a similar way to ChatGPT. It's, conversational based and it has tools where it can access the product catalog, access the recipe catalog, even retrieve a lot of kind of facts about, you know, how does the loyalty card work or how do you do a return. And, you know, some like customer support, kind of questions.
00;47;46;13 - 00;48;12;25
Unknown
And then one of the most important things it has is, the ability to personalize its responses based on your transaction history and your previous interactions. So we have a memory system where if you tell it that, you know, you're a vegetarian or you don't like mushrooms or, you know, all of these kind of very nuanced things, but that are important when making decisions about what groceries you would like.
00;48;12;28 - 00;48;38;21
Unknown
The, the system stores that information, and then it's included in the context for all the future conversations. And so, you know, just by initially being able to, kind of reason over a shopper's transaction history, we can build up a shopper profile that helps understand some of the dynamics in the household and what their preferences are, and kind of, you know, to your point, we've had Google search, but, is hard to consume, right?
00;48;38;24 - 00;49;01;25
Unknown
You type in something and it's like, okay, here's 100 million search results. You're like, great. But like, what's the what's the answer to my question? Yeah. And so in a similar way, you know, a retail retailer's catalog maybe has 70,000 products that are available, you know, maybe 50,000 in like one location, but you probably want to buy somewhere between five to, 30 of those.
00;49;01;25 - 00;49;36;29
Unknown
And so the question is, how do we filter down that search results of 50,000 down to the, you know, 10 or 20 that you're actually going to purchase today? And because the agent has that context, we can kind of offload that work of searching through and reading all the attributes and making all these comparisons. And, you know, if someone can just come into our service and give the high level request of, I want three high protein dinners for this week, and the assistant will know what proteins you like, what you're cooking, experiences.
00;49;37;02 - 00;49;59;19
Unknown
You know, what other factors like price sensitivity or brand preference. And we'll assemble those together for you. So we kind of say instead of doing the traditional search and select where you're typing in the keyword, seeing the whole list of results kind of passing through them, and then, you know, figuring out which one you want, to know what the experiences that we power our review and accept.
00;49;59;21 - 00;50;20;06
Unknown
So the agent does all that hard work of doing the curation and searching. And then you as the kind of orchestrator or the driver of the agent, just get to review the work that it did. You can give it some feedback if you want something changed. But then you accept, you know that. And you, you've saved yourself a lot of time.
00;50;20;08 - 00;50;38;13
Unknown
And so that's our conversational AI assistant experience. Sorry. Real quick on that. Is there a difference in build when you talk about a human to agent interaction versus like maybe not. So a future state of like agent agent, right, where I might just end up having an agent that like, to your point, gets absorbed into this cloud environment.
00;50;38;13 - 00;51;01;19
Unknown
And it's like I go into Claude actually do all my grocery shopping, and then that agent goes off and figures out which grocery store is the right one. I mean, build wise, it's the same or it's surface ability, right? Yeah. At the end of the day, the assistant that's trying to return back those meals to you doesn't care whether it's a human on the other end or another AI assistant who is just abstracted.
00;51;01;19 - 00;51;21;06
Unknown
Right. There's still a human. Ultimately, that's kind of we're trying to bring groceries into their home. And so this is something that's beneficial. And this is a question to you, I think that a lot of business are going through is we've seen through the digital age, right. Everyone kind of has a website. And then everyone had an app.
00;51;21;11 - 00;51;57;07
Unknown
And we have these digital services that you meet your customers and like deliver your products and services through to your end customers. But in this next generation, that's going to be an agent potentially in a lot of cases. And so we have, lots of businesses that are sending up either an MCP kind of server that can expose, direct kind of contexts and APIs and data directly to an agent or just going a even a simpler route and having a CLI right, where you can kind of, just the, the resurgence in these terminal interfaces has been really interesting to see.
00;51;57;07 - 00;52;16;19
Unknown
But, you know, that's what I systems are. They don't care, right. They don't need it. It's easier for them to read a terminal output than it is the pixels in a screen. Right. Or cheaper at least they're getting pretty good at reading screens now actually. Yeah, yeah. Fair. Okay. So conversational agent, you said two more products, right?
00;52;16;22 - 00;52;39;17
Unknown
The other ones are a genetic search and then a genetic merchandizing, and, I think a really good parallel to think of this is similar to the coding tools where you maybe we originally had like line autocomplete and you have this autonomy slider. So it's like complete one line. And then it's kind of like copilot complete one function or do this like change that.
00;52;39;17 - 00;53;00;06
Unknown
I'm going to be in the loop and review. And then you have these like long running your cloud agents where here's an objective work as long as you need until it's done. Right. And that's kind like the three levels of the autonomy slider. And so, the conversational AI assistant is kind of that one in the middle where you use it as a copilot, you know, to alleviate some of the work.
00;53;00;06 - 00;53;24;27
Unknown
And but you're still very much in the loop. Then we have a generic search, which is kind of the lowest level. And so this meets most customers where they are in a grocery e-commerce environment, which is entering a search term into the search bar. And so if you put in something like Taco night, the traditional search bars, which are keyword based, they're either going to key on the words taco or night or taco night together.
00;53;24;27 - 00;53;47;06
Unknown
Right? That's all they have to work with is doing things based on keywords. And so you might get some taco shells. You know, night could surface a wide range of products and some of them are definitely not good for tacos. But that user intent of I want to make taco night. So what I really need is I need ground beef and or or ground chicken or ground turkey, depending on my preference.
00;53;47;06 - 00;54;19;16
Unknown
Right. And then I need either a hard taco shell or a tortilla, depending on what I like. I need some cheese and lettuce and all these things to have a taco night. And what taco night means to each family is different. But the way that our a generic search works is we take in that search query in the search bar, and then we use an LM to kind of decompose it into all the subcomponents that describe customer intent, and also make an evaluation of, should we surface recipes for this search query or not?
00;54;19;18 - 00;54;37;26
Unknown
So, you know, we do handle non-food queries as well. Right? So you could be asking for a skincare routine, or you could be asking for, you know, some stuff for your plants or whatever. But in the cases where, we're working with a retailer that does have a recipe catalog, it can make that distinction.
00;54;37;26 - 00;55;04;17
Unknown
And then we'll also surface recipes for taco night as well as all the subcomponents. And then all those subcomponents are personalized to that individual shopper using their transaction history and state. Yeah. It's how much how much context do you have to build? I mean, how much does it feel like a harness versus, what you described as like just a better understanding of the customer?
00;55;04;19 - 00;55;30;28
Unknown
Well. The key part about the harness is, a few things. So in the search experience, we need it to be extremely fast because the traditional search results that are keyword based, those are returning in, you know, 100 milliseconds, a couple hundred milliseconds. And if you submit a query to your output, it goes into thinking mode. It might be five minutes before you get your answer.
00;55;31;00 - 00;55;57;26
Unknown
Right. And so that's something that, you know, we optimize both, a performance and a cost standpoint is, for a retailer who has millions of customers on their platform. How do we make this something that they can justify the cost for? And so we use in this case, right? We use specialized inference hardware, to, to get extremely high tokens per second.
00;55;57;29 - 00;56;25;13
Unknown
That helps cut down the response time. And then additionally, what our harness does is, if we need to return back a list of products to the customer instead of, and we want to format it. Nice. Right? So it has an image, it has a title, it has the price. And instead of having to elm rewrite like because we're going to put into the context right where it's like it does a search products tool call it sees all the context about the search results.
00;56;25;13 - 00;57;02;21
Unknown
And like, you know, that information that's helpful for making a decision. But then when we need to return that information to the client, we don't want to have that all. Re transcribe out an image URL or, you know, the description of the product, the title, all these things. And so what our harness does here is a lot of validation where all the LM has to do is write product ID one two, three and then in real time as that stream from the provider is coming out, we basically have a stream processor which is looking for product IDs or recipe IDs or other kind of rich content type coming from the live stream.
00;57;02;24 - 00;57;26;21
Unknown
And when we see a product ID like product one, two, three. Well, first check and make sure like is that a hallucination? Because limbs still do occasionally hallucinate. And we don't want to show one retailer's product on another retailer's experience. And but we do know ground truth. What was the data returned to the limb like? What was it in its context when no, when it was making its response back to us.
00;57;26;23 - 00;57;46;27
Unknown
And so we'll check. Is that product ID real. And then like, yes, if that was in the context, then what we do is we can do a database lookup. And actually usually we actually just read it from memory because everything's already in memory from performing the search. And then we'll enrich that stream before it goes to our app or website and basically populate all those details.
00;57;46;29 - 00;58;07;11
Unknown
So if you were trying just to have a raw elem provider do this, it's going to spend five minutes. Re transcribing out all of that product data and information. There's no checks for hallucinations or other guardrails. And that's kind of one of the key parts of how our harness helps retailers optimize for cost and speed trade offs when building with alarm systems.
00;58;07;15 - 00;58;28;10
Unknown
I love it, man. Okay, I know you got to get to your next thing, here at one. So one last question. What's like a controversial I take. You've got that you're down a hill for. Controversial. I take,
00;58;28;13 - 00;58;52;19
Unknown
Yeah. I think we can actually blend to two hot things. I in crypto is I think we we are going to see these, you know, we still don't know what consciousness is, but I think we will see things, where we'll have, And there's some early signs of this even, I suppose, but I still think most people would argue that you can't have, like, digital consciousness.
00;58;52;22 - 00;59;34;04
Unknown
So I would say that we're going to have these kind of self-sovereign AI systems that, operate autonomously and kind of, use these, blockchain technologies as having their own, sovereign individuality of. Right. They can have their own finances, they can work in the economy. And then it's self-directed. They can kind of decide how they want to allocate their resources, and they can spend money to create art or poetry or, build things in the world that, you know, is going to be kind of independent of human control to some extent, which is going to be potentially really interesting.
00;59;34;06 - 00;59;51;29
Unknown
And we already have these like smart contracts, right? They're deployed once it's deployed, you can't, can't really stop it. Unless you, you know, you have certain guardrails built in. But yeah, I think, we're not at the peak of, until, like, humans are not the peak of intelligence. And I think the exponential is going to continue.
00;59;52;00 - 01;00;12;16
Unknown
Yeah. Well, truly enter this video game mode in life, right? Where we're engaging with NPCs that are actually NPCs. Yeah, yeah. Digital intelligence is. Yeah. Right. Yeah, yeah, yeah. That's awesome. How long until we get there? I'll. I'll go with curse of all I can say, 20, 29.
01;00;12;18 - 01;00;17;05
Unknown
Well, Daniel, thanks so much, man. This is awesome, I appreciate it. It's been a lot of fun. Thanks for having me.