Subscribe
Share
Share
Embed
We follow the journey of Faith Forster as she creates an AI native tech startup & product.
Welcome back to Building Out Loud, a weekly check-in with Faith and Randy to see how everything's going in the world of building Faith's new AI agent. What are we calling it this week?
Faith Forster:I've been calling it product intelligence as a working project title.
Randy Silver:We've done one episode so far just kind of introducing what we're doing and why. We should probably talk about the idea itself and where you're going with this. So we're both come out of product thinking. So I'll ask you the most obvious product question. What is the problem that you're trying to solve?
Faith Forster:As product leaders and product managers, we do a lot of work to gather different data points work out, therefore, what does that mean and what should we be building? So a lot of this work is done by product managers as part of a planning cycle. They should be looking at competitors, any changes in the market if you're in a regulated space, what regulatory changes are coming, also what customer feedback we're getting like sales, win and loss analysis, feedback we're getting on prospect calls. There is a lot of information that we gather and normally sits across a whole bunch of different tools. It's usually not an easy thing to capture and work out what does this mean.
Faith Forster:Even if you do have a customer feedback platform, someone manually has to go through every single comment anyone submitted, work out what are they saying, what does that mean, merge similar ones together into themes, but there's still no way of actually connecting that to the product team. So the product manager hasn't got an easy way of just looking through what are the things that I need to be thinking about this month or this quarter and then prioritising those. Actually, AI agents are quite good at gathering large volumes of information. They're also quite good at analysing that information. So if we could use AI agents to gather that information both from internal sources but external sources as well to understand what our competitors are doing, what changes they're making, what does that mean for us, and should we therefore be doing something different.
Faith Forster:But doing all that analysis quickly and on a very dynamic basis. So it's not just something you do once a quarter or once a year.
Randy Silver:What's different than what people are doing today? Are they doing it on thoughts and vibes and perception, and this is to give them evidence back? Is that the thing that you're solving?
Faith Forster:I think we're all a bit guilty of recency bias with this stuff. Whoever shouted the loudest or the powerful person within your organisation has got wind of something and that's therefore the most important thing. Product managers often have bias as well. They've been working on a certain feature. They've gotten to know the customer problem really well.
Faith Forster:They've gotten to understand all the option needs to fix that feature really well. And so they want to just keep going rather than stepping back and thinking, actually, we've got this to a certain point. Is that enough for now? Should we focus on something else? We do have inherent biases in the way that we assess the option needs.
Faith Forster:That's partly because the data is not easy to access, to understand, to assess. So in the absence of good, robust data that's more comprehensive, we tend to focus on recent conversations or recent discovery work. That's one part of it. There's a slightly separate step in aligning that to your product strategy or your key metrics or the goals of the organization to make sure that you're always doing the things that are gonna have the biggest impact on your commercial and customer goals.
Randy Silver:Okay. So you've got this as an idea. Where do you go from there? What's the first step? I know you've been talking to people about it and you've been playing, but what's your process?
Faith Forster:My real worry is this is sensitive information for any organization. I had very important questions around the architecture and how you set this up so that it actually works. I spoke to an engineering friend. What he suggested was simpler than I expected. He said you can create the front end on a tool like Lovable.
Faith Forster:Lovable had just released their functionality that could create a database. When I started using it, it broke pretty quickly. It didn't work very well. He suggested having a third party database that you can connect into. You also need your agent manager.
Faith Forster:The other part was the API management, a way of being able to manage and authenticate access to all your internal tooling. He suggested a tool called Pipe Dream, which could do that as a white language solution within your platform. So a bit of a different architecture to what you would normally set up with a full stack product, but it felt achievable.
Randy Silver:Do you jump straight from idea stage into architecture stage or were there things in between?
Faith Forster:This is where I made a mistake quite quickly. And, a common one, you've got this AI tooling, which makes it really easy to build stuff at least get part of the way through building stuff. The reason I started doing this was to learn the AI tooling. I wanted to play around and work out how it works. What I hadn't considered was all the viability and desirability questions around is this a problem worth solving?
Faith Forster:If other people got this problem, would they pay for it? How would this fit within their existing sort of product tooling stack? There was a lot of bigger questions that I didn't really properly consider.
Randy Silver:This was desirable for you at this stage because it was solving a problem that you had, but then you left that organization and you decided you wanted to keep playing with it and developing it. So where did you go? Who'd you talk to? How did you start evaluating the other questions besides just feasibility?
Faith Forster:I created a version of this product in Lovable and shared that prototype with a number of product managers and product leaders I know to get feedback. I got some really great feedback. There were definitely some suggestions on other features or more things we could do with it. There was definitely curiosity like yes we're trying to do this for the agents but I totally get it's not easy and actually having a central place where all of this data sits that I can then have an AI agent sit across all of that and ask questions about the product strategy would be really useful. At the moment, the agents are single threaded when you create them to query one tool that comes forth, but it's not then understanding other information that you're getting from other sources internally or externally.
Faith Forster:So it was a bit like, yeah, this is interesting, but it wasn't a need to have reaction. It was a this could be nice. This could be fun to play with. But that that was enough for me to keep exploring it.
Randy Silver:I think the next step was you talked to a mutual friend of ours, Dave Colleen.
Faith Forster:I was having a chat with him about what I was doing, and he gave me a prompt that he had created, which takes you through a three step process. You use ChatTT to effectively act as your product manager, product designer. And so it steps you through a bunch of questions to create your product spec document, user flows, and a PRD. I got these three documents, which seemed amazing. I spent quite a bit of time going through those documents, tightening them up and thinking, is that right?
Faith Forster:Is that what I want this to do? Like, is that the core functionality? And I actually took those and thought, let's try out the other tools. I've got a much better basis of understanding what I want to create now. Let's test that out across all the different tooling and see what happens.
Faith Forster:So I started again in Lovable and uploaded these three documents saying create this. And I used that prompt and put it across a number of the different tools. So I did that in Bolt. I did it in VO. I did it in Claude.
Faith Forster:I also did it in Replit and another tool that's quite early stage London based startup, I think called Solid. Exact same prompt, three documents, put them in all the tools and just had a look to see what happened.
Randy Silver:What happened?
Faith Forster:It was really interesting and really good learning. Bolt ran out of credit before it even created anything. And I had a look at how much it cost to buy credit. That was 300,000 credits and to buy them was like £25. So that gave me a million credits.
Faith Forster:If I got nothing for 300,000 credits, am I going to get more of nothing? So I scrapped that one straight away. Was like, that's going be too expensive. Likewise, Claude, read through all the documents, said it had produced the product, but the screen was blank, the preview screen. And so I was like, well, what's happened?
Faith Forster:What do I do? So I got it to rerun it and then it ran out of credits. So again, I got nothing back. So I scratched that one as well. Was really interesting.
Faith Forster:VO did a much better job than Lovable of actually visualizing data. It was able to handle the complexity that I was asking for in this product much better than Lovable. In the way it's the styling and the design Lovable feels like it's geared more towards a consumer orientated product than a B2B product. So what I ended up doing actually was creating a version in VO and then copying the screen and putting that in Lovable and saying do this so that it would then visualise it better, more effectively. It could handle larger volumes of information but styled nicely because VO is black and white it doesn't even attempt to do the UI styling whereas Lovable is quite good at that.
Faith Forster:So again, I got it to the point where I had a more robust version of the product. But actually, thing I found was none of those tools handled user authentication very well. When I started looking at Replit, they had some good 101 training on how to get started with the product. I watched those videos and actually one of the things it said is, give the agents one feature at a time to build out and then they can handle it much better. And that's what I found.
Faith Forster:Trying to put a PID for a whole product into it, it couldn't handle it very well. So most of them, Lovable only was able to create the user login. It didn't create any of the onboarding flow or user features. Vio created the dashboard, like the landing page, but then I had to ask it to build everything out behind.
Randy Silver:This is definitely one of the lessons I've learned as well. You can come up with a whole concept document. You can have your whole PRD, but the art of writing prompts to effectively get things built is complicated. It's not impossible, but it's complicated. You can use Claude or Gemini to help you write the series of prompts that you would then feed into Replit or Lovable, breaking it down into manageable chunks to do so.
Randy Silver:It sounds like you learned that lesson the hard
Faith Forster:Yeah. Way as actually, I've ended up doing is just focusing on feature by feature. I originally asked what should the MVP be, And it was still quite complex. What I'm trying to build is a whole B2B SaaS platform. It's not a simple consumer app.
Faith Forster:What I've learned is just do it step by step yourself in the platform. What you need to work out is where you get started. I think that's where I kept going back and forth because I need an onboarding flow to ask the right questions to know how to set up the first version of the page, the dashboard. So weirdly, I started with an onboarding flow rather than a core product or a core feature because I needed the user input to then use that to then set up their dashboard properly.
Randy Silver:Alright. So now we're starting to talk about the experience itself. Let's pause there, and we're gonna come back in the next episode, and you're gonna walk us through where you got and show us how this actually works. Give us something to relate to. So we'll be back soon.