Subscribe
Share
Share
Embed
We follow the journey of Faith Forster as she creates an AI native tech startup & product.
Welcome back to Building Out Loud with me, Randy Silver. And
Faith Forster:Denise Foster.
Randy Silver:Every week, we get together. We talk about the new app, the new business idea that Faith is building. And we're at a nice point in this because, Faith, last week when we recorded, you had gave us a demo. You showed us what was going on. You've got an MVP, and you were just about to send it out to a whole bunch of people.
Randy Silver:So what happened?
Faith Forster:I have a working version of the product now that does the complete workflow. I've had some calls with some friendlies first to talk them through it and get their feedback and got some really useful feedback from that. There was a few different reactions. One was slightly overwhelmed, think similar to what you had, like just a bit blown away about how much is there, how much you can do with this AI tooling, which was really interesting and really useful because it made me stop and reflect and think about the purpose of each picture. Like, why is it there?
Faith Forster:Why you have it here? Why not somewhere else? What are we trying to achieve by having it here? And so for example, the timeline, the roadmap that you were a bit like, really? Do you really need that?
Faith Forster:I really thought about it. Was like, well, actually the reason you'd have it here, not Jira or somewhere else is so that you can understand the implications on value realization. Are you actually going to achieve that goal or has it been delayed? Therefore, what implication does that have on your roadmap and also the realization of whatever goal you're trying to achieve? And so I've rewatched that feature so that it's much more around the impact, the changes the dates is having rather than the actual plan itself.
Faith Forster:So it's really, really sharpened the thinking, the focus and the execution across a number of different features, which has been incredibly, incredibly valuable. I had reached out through a couple of product leader WhatsApp groups and said, I've been playing around this tool. The way I talked about it or positioned it was it's to help product leaders and teams better understand and align their product work to commercial and customer goals. That was all I said. Was literally one line and I've had 53 people, heads of product VPs or CPOs.
Faith Forster:I've checked them all and I couldn't believe it. Honestly, I thought I might get 10. It sort of shows that this idea of focusing on outcomes or driving product thinking and product work towards the goal you're trying to achieve is resonating. There's definitely something in that. I'm just doing some final touches, which we'll talk about in terms of the agents.
Faith Forster:And then I'm about to send invites to 60 people.
Randy Silver:Fantastic. So you've got validation that this is a problem and an opportunity space that's worth exploring. Now the question is how quickly are people gonna be able to respond to it to use it? Is it the right way of solving this problem? And then you have to figure out, are they willing to pay for it as well?
Faith Forster:I have even from the calls I've had so far, I've already had, so I say, our license for product board coming up. This is squarely where we want to be moving towards. We're pretty good at product process now, and actually we need to shift the conversation more towards the outcomes and the impact that the product teams are having. So the early indication from there is that that's real. They would be up from paying for this, which is quite cool.
Randy Silver:Yeah. This is something that's definitely been in the ether the last couple of years and the best product leaders I know have known this lesson for a while, but Matt Lamey is talking about it in the last year or two. Dave Washer talks about it. Rich Maranoff has a new book coming out about the idea of how do you connect the work you're doing to value created? How do you speak the language of the rest of the company?
Randy Silver:We're not doing this stuff just because product is a good philosophy. We're doing it because we think it is the best way of executing and how do we show real value.
Faith Forster:Yeah. Yeah. It absolutely has to about driving commercial impact. Product teams are a huge investment. If The UK base are 1,000,000 a year per product team.
Faith Forster:And so that needs to have some sort of return. Actually Matt Loud Building is one the people I spoke to last week. I a 100% agree with his Loss of Impact First Product teams, his new book. In that there should be a very, very, very clear and direct link between the goals of the product team and the business goals that we're all trying to drive towards. One of the things I mentioned, I think in our last call was I was really not quite certain about how the goals have been set up and whether that would work for lots of different organisations, because it's one of those things that's done differently everywhere.
Faith Forster:And so I really wanted Matt's take on it. He actually, he said it's one of the best executions of the SEC, which is lovely.
Randy Silver:You've gained indications both from people who are working as consultants, but also people who would potentially be buying this. That's that's all strong. Now let's talk about the build itself as there were lots of things you showed that were great. Yeah. There were a couple things you were a little nervous about.
Randy Silver:So you've been playing around with AI agents for the last week. How's that gone?
Faith Forster:Oh my god. There's definitely been moments where I'm like, what am I doing? It has been far longer and more painful than I thought it would be. Honestly, it has gotten to the point where I will Google Dex news, take the screenshot, put it in and say, This is what you should have found. I'm teaching Google how to find news.
Faith Forster:There's just certain things I thought that it would know how to do without me having to trade it. So it's taken a lot to really trade it. One of the things we've got in there for competitive profiles now we pull out their pricing. You can really quickly and easily reference your competitor's pricing. Finding pricing pages and then drawing the content off the price page is actually really not straightforward.
Faith Forster:You've got different regions with different pricing that has different currencies, instruction in very different ways across different businesses. There is a lot actually. I can understand why you would have to spend some time really training the agents and making sure they're finding the right information.
Randy Silver:B2B pricing is a whole thing as well, but that's often not very transparent at all.
Faith Forster:Yeah. So if it's not transparent, I will say that and say the sales led model, there's no indication. Although there have been cases where even though the pricing isn't published, you can find reference to it on reviews or there's been some sort of news update. You can only find something, but it's taken a lot to train the models to find the right content and pull it out.
Randy Silver:So where have you gone with it? Where did you start?
Faith Forster:I had some advice from some developers who have been playing around with these agents that Gemini was the best to use for web search based things, which makes sense because it's a Google product. I've kind of given up on Gemini right now. I always found it so frustrating. I've got to the point where I've actually completely refactored the whole agent part of platform. So it now all goes through a router or router, depending on what asset you have.
Faith Forster:The user can actually choose whether it goes through forward Perplexity, OpenAI, or Gemini. For things like finding the right news publications or finding the right review platforms. I think I'm going to use complexity even though it's a little bit more expensive. It's so much more robust at finding the right content and making sure it's properly cited. And I potentially might use that as default for the actual feedback itself, but that runs daily.
Faith Forster:So that could add up. I've just finished building that refactoring of those agents. I'm now going to go through and test what results I get back from the different LLM models and then decide which is the default one for each agent.
Randy Silver:You just mentioned a moment ago that you're running it daily. I'm curious, how do you decide what is the right cadence? When news hits, it's incredibly important, but pricing news for lots of things doesn't change very often. Feature news doesn't change very often. It's more alert based.
Randy Silver:Do I need to rerun this every single day?
Faith Forster:Most of the agents have a schedule that the user can change because that also has a direct implication on the cost. So something like gathering feedback on yours or competitor products, the default is that that's run daily. So as new comments are posted on different review platforms that will come through to you quickly. Something like scanning to see if there's any new competitors. So the way we identify a competitor is if there's been a user comparison through a review or a comparison site, just because it's got the same features doesn't mean it's a competitor.
Faith Forster:It could be in wildly different spaces. We only scan for new competitors. The default at the moment is once a month because they tend not to come up that often, but it's all configurable. So anyone can choose to change their schedule.
Randy Silver:And how often should I be scanning for this anyway in my mind? I don't wanna be distracted every day by some potential change in the market. If there's a massive thing, want to know straight away. But if it's a relatively mine thing, if I'm reasonably established and there's a new competitor just bubbling up, I probably have a month, a quarter, multiple quarters before it really matters and I need to change things. Might wanna start thinking about, but I don't wanna switch my entire strategy on a dime just because one review came out.
Faith Forster:It was one of the points that came out. One of my conversations last week, he was talking about competitors and the way they monitor and respond to competitors. So one of the things I've built in following that conversation is the ability to rate competitors by threat level. If it's a higher threat level, the feedback is prioritised higher. He was also one of the people who suggested that scanning of competitors, because he mentioned there are some competitors that he's aware of, but kind of dismissed for whatever reason.
Faith Forster:But they've got an example of an AI native startup they're finding quite threatening. Being able to jump in the platform, do a refresh, get that competitor, get all the information around the key differentiators, their pricing, their features, their integrations, summary of customer reviews so far, if there is any, and actually turn that into a battle card that you can then give to sales and do all that in minutes was one of the things that I got really excited about. That's really powerful. And in that battle card, having the comparison of their features to your features and how you would position yourself to be stronger was actually a really easy thing to do because we already had an AI assistant that sort of sits over the platform anyway. This button is literally just putting the prompt into the AI system.
Faith Forster:The AI assistant then says, what are you hearing from your salespeople? Are there any objections coming up yet? So that it can then tailor the messaging accordingly.
Randy Silver:Okay. So you've got an assistant over the top of the system. You've got agents running underneath. What's been the difference for you between doing something at the top layer versus these autonomous agents?
Faith Forster:It's really that training. The top layer is pretty easy to do. Have the conversation with the user and use all the information we've got and the feedback to have a richer conversation. The agents, you really have to sit there and go, this is what you're looking for. This is all the scenarios that could come up.
Faith Forster:This is how you make sure that it's true and reliable. You can make sure there's a proper link there so that we can source it. Yeah. There's a lot more effort involved in training the agents than there is just a general assistant.
Randy Silver:Have you gone down the road of using evals for this?
Faith Forster:Not yet. It is something I need to look into and work out how I do that. If I do that, I think I will need to.
Randy Silver:It's an interesting one because it's come up a lot in my feeds lately. It's been the subject of lots of conversations. And I understand that technically, they are their own unique thing, and it is a new skill set for people. But philosophically, I'm curious to me, it sounds like it's not that different from the idea of user acceptance criteria and test driven and behavior driven development. The eval was describing the end stage, the bad cases, the good cases.
Randy Silver:It's a very rigorous way of presenting the information and of doing the thought process. But I haven't gone down actually writing evals myself or really deep into it. I'm struggling on the difference between the philosophy and the application right now.
Faith Forster:I haven't done it, so I'm not sure about the practicalities of it yet. I think that's one of the things with the agents. There's not many areas in the platform where if the agent doesn't quite get it right, the user can't correct it. One of the things I've built into it is to use that feedback to then improve the prompt. So for example, feedback that's collected on either products or competitors is allocated to a product team based on their understanding of what they know, but a team member can reallocate that to a different team.
Faith Forster:That act of reallocating it to a different team is being used to learn so that the allocation next time will be better. There is ways that you can build it into the platform where it gets better over time on its own.
Randy Silver:That's gonna be an interesting one because there's all kinds of reasons why something might be reallocated. You know, is this a rule? Is this an exception? Is this based on people being on holiday or a one off thing? Do you assume that just as this was done once, that's the way to always do it?
Randy Silver:It's, yeah, it's gonna be fascinating to see what is the assumption, what is the initial approach, and then what is the reality as you start to get usage. So what's the next step? What are we checking in for with next week? Are you going to start getting feedback, you think?
Faith Forster:In the next twenty four hours, I'm going to be sending links in a feedback form to these 60 people. I'm sure I'll start getting some feedback in over the next few days, but it might take a week or two to get all of it back. That will provide hugely valuable indication of whether or not we're on the right track with this. As I mentioned at the start, I have already had someone say, please, can I be part of your beta? I will hopefully in the next week or two have an indication of what's left to get this into a place where it's actually being used in earnest within an organization.
Faith Forster:In the meantime, we're reframing the messaging at the moment. I've just had proper logos done. I'm getting our first LinkedIn posts so that we can get these podcasts out. So lots of fun stuff happening.
Randy Silver:Fantastic. Well, looking forward to seeing you, and we'll catch up next week.