Building Out Loud

Building Out Loud: Faith's AI-Driven Product Management Prototype

In this episode of 'Building Out Loud,' hosts Randy and Faith dive into the development of Faith's new AI project designed for product monitoring and management. 

Faith showcases her progress, highlighting the journey from a simple clickable prototype in Lovable to a more advanced version being built in Replit. The episode covers the project's core functionalities, including AI agents for data collection, sentiment analysis, and dashboard metrics for product teams. Discussions also touch on the challenges of sentiment scoring, the relevance of OKRs in an AI-driven environment, and the importance of setting constraints for productive development. 

Tune in to learn more about the project's evolution and upcoming features.

00:00 Introduction and Episode Overview
00:29 Initial Product Development
01:18 Lovable Prototype Demonstration
03:10 Replit Development Insights
04:02 Sentiment Scoring Challenges
04:21 Organizing Feedback and Goals
04:57 OKRs and Leadership Insights
07:30 User Engagement and Roadmapping
08:12 North Star Metric and Future Plans
09:04 Conclusion and Next Episode Teaser

Learn more about Discoveree: https://discoveree.com/

What is Building Out Loud?

We follow the journey of Faith Forster as she creates an AI native tech startup & product.

Randy Silver:

Hey. Welcome back to the Building Out Loud podcast, videocast, whatever this is. Actually, this episode is definitely video. We'll come back to that in a moment. I'm Randy and

Faith Forster:

Faith Forster.

Randy Silver:

And we've been with you for the past couple weeks talking about the work that Faith is doing, building out something new. And we've been talking about it in the abstract and teasing, but this week it's something different.

Faith Forster:

We're gonna look at the product itself, the pictures and how it's working, how I'm building it out.

Randy Silver:

We're starting off with the initial versions. This is when you were playing around in Loveable. Right?

Faith Forster:

As we talked about in the last episode, where I started was just building a basic clickable prototype in Loveable. And I shared that with a bunch of PMs and product leaders that I know and got some feedback. I'll show you that. And then I'll also show you where I'm up to at the moment. I actually decided to build this in Replit.

Faith Forster:

They've got good resources to help get started and plan out the work. But also Replit has, I think, a stronger ability to create and manage the database. Most importantly, they also have a developer community. Once So I get this to the point where I can't take it any further, I can engage the developer.

Randy Silver:

Okay, we can get into all that. I want to see this. We've been talking about your weeks. Let's see it.

Faith Forster:

So this is the version I built in Loveable. The original idea, as we talked about, was to have a series of agents who are monitoring feedback on your product that could be externally available information or from your internal tools, HubSpot, Salesforce, Intercom. So this is Lovable. Lovable is great at building beautiful UI and designing it well. I gave it a really brief overview of what I wanted this product to do and it created this.

Faith Forster:

So what I want to do is get some basic information around your product, your competitors, your adjacent products that we should be monitoring. But the idea is that you tell us your product, the website, and then we have an AI agent that actually goes and finds the information. You don't have to put it all in. So we've actually gone out and looked at what we think the competitors are and suggested some, and then you can add your own in as well. Again, for adjacent products, we look at your customer segments that you're managing and then we've got the agent to also go out and have a look at where we found mentions of your product on publicly available feedback forums or review sites.

Faith Forster:

So it's found mentions on these different sites and said should we monitor these? Yes. Then if you've got specific platforms that you want industry specific sites, then you can add those in as well if the agent hasn't found them. And then also linking it to business goals. So what is it that you're actually trying to achieve?

Faith Forster:

How have you set up your product teams? This one's assuming that we would then also be able to connect into your internal tools. So it's asking which do you use? And that then completes the setup. So we have here an overall product dashboard.

Faith Forster:

These are your business goals so increasing revenue. We can then see some of the opportunities. They can be prioritized and organized by product team if they're not already assigned to a product team. We've then got our other goals, overall sentiment score and then it compares it to our competitors. You can see how you're tracking against your competitors over time.

Randy Silver:

I'm curious about that one because this is something that's been promised for ages. And all my experience with these in the past was that it didn't deal with sarcasm very well. It was such a rough and inaccurate thing. I know this is just a prototype, but I'm curious, and what's your confidence that you can do something valid with Sentiment Score?

Faith Forster:

I have spoken to some colleagues who have worked with platforms that have sentiment scoring. And so I've got some learnings from them. There's different steps you need the AI agent to segment the information before it assigns a score. So it needs to identify the different products being referenced or features before it splits that out and works out the scoring. But it is something that I've been thinking about with this platform.

Faith Forster:

How do we enable users to refine the prompts, giving the users access to be able to refine and improve pictures like this in the platform over time? Keep going.

Randy Silver:

Yeah, keep going.

Faith Forster:

So we've sentiment scores against competitors and what the opportunities are to help improve the sentiment score of your product versus your competitors. We also look at the sentiment score by customer segment, key features and feedback we're getting for different customer segments that contribute to that. And then also feature level feedback here as well. The way I was originally thinking about this was to organize a lot of this information by product team. So it really is about enabling the product team to make better decisions on where they focus their effort.

Faith Forster:

So we've got here the core platform team. We've got some key numbers that they can be monitoring. You've got their target. And this is an interesting question you mentioned about OKRs, how you actually set this up in a way that's flexible enough. Different teams will also have slightly different competitors.

Faith Forster:

For key features, you'll have different products that you are competing against. Each team can then choose which competitors they're monitoring and see what the opportunities are to get better parity against that competitor.

Randy Silver:

Let's touch on OKRs for just a second. And I'm just curious, in your experience, OKRs tend to go around quarterly planning. With the type of work we're doing now, with the acceleration from AI, are OKRs still useful and relevant? Or do we need to approach them differently?

Faith Forster:

I think there will always be a role for leadership of any organization to set out what are the most important things we need to be delivering against. And that should be a combination of both commercial value and customer value. How that manifests in terms of the actual framework to set goals, I don't know that the framework changes. I think the nature of the metrics themselves will, though.

Randy Silver:

That aligns very closely to what I was telling someone else this morning. So glad to hear someone else smart saying that. I'm also curious every time I talk to someone who's got a lot of experience in leadership roles who then goes back to building something on their own, do they follow their own best advice? As you're doing this, you're early stage. Did you set up any formal or informal OKRs for yourself?

Faith Forster:

One of the things that I have found quite useful, I've created false deadlines for myself, and I find that's when I make the most progress. I'm talking at an event or I've just finished the Build with AI course with Hustle Badger and I wanted something to demo. I think sometimes having some constraints is really useful.

Randy Silver:

Giving yourself constraints, giving yourself deadlines, it makes the difference in prioritization what's truly a minimum viable version and not the nice to haves, but the absolute must haves.

Faith Forster:

In terms of this demo, the idea was to have an overall product dashboard that shows how the work of First product teams is contributing to our overall goals and how our product is performing against competitors within our customer segments. By product team organising feedback that we can get from publicly available information but also internal sources. And again, at a product team level, being able to work out, well, how does this contribute to our goals? How do we compare to our competitors within our space? What is the fixture feedback?

Faith Forster:

And then the ability to set a priority for some of these ideas, add it to your plan. You can then use all that context to create your tickets automatically in Jira.

Randy Silver:

So looking at this, it gives me clarity about who you think the users are, who you think the customer is, how you think they're going to be using it. Let's just be explicit. It sounds like it's the head of director, VP, whatever you wanna call it, a product, CPO, as well as the various teams who are going to be using this. Did this inform or drive how you thought people would be using this on a day to day level? Would they be coming in and doing it first thing in the morning?

Randy Silver:

Would they be doing it weekly? Would they be doing it in retros? Did this drive that kind of clarity for you?

Faith Forster:

Now that I'm actually building it out in Worklet, that's becoming more real. Like, how do you make this really dynamic a way that people do want to check it on a regular basis? One of the challenges with road mapping tools, people only look at them during the planning process and then they're out of date. And so it is one of the things that you want to drive that behaviour around. Yes, you have a roadmap, you have a set plan, but what if something changes?

Faith Forster:

How do you make sure you know that something's changed and then work out when you go to kick off your next project? Is that still the most important thing for us to do next or not? You'll see some of the fixtures starting to come through in the next version. It is a little bit more dynamic. If something changes with one of your competitors or adjacent products, you'll be notified.

Randy Silver:

So one more question before we go into the new version. Did this give you an idea of what the North Star metric for the platform might be? If people are using this successfully, you will see something as a leading metric that will drive value for your users. Were you able to determine what is the activity or activities you wanted that would be the indicator of value and success?

Faith Forster:

Actually, I've gone around in circles on this a little bit. Where I started this idea is like, you get an army of AI agents who are pulling all the information together for PMs and for leaders to make it much easier for them to work out what they should be working on next? It was always about helping product teams and product leaders have the confidence or assurance that they're always working on the most valuable things. But actually that gathering of the data is only one part of it. It's also how that aligns to what are you trying to achieve, the commercial value, the customer value, and having a way to systemize that is more important than the data gathering.

Randy Silver:

Okay. This seems like a good place to pause because we're leaving on a cliffhanger. I really wanna see what you've built in Replit, but we'll do that in the next episode. Is that right?

Faith Forster:

Yep. Sounds good. And we can spend time talking about what I've learned and how to build in Replit and make it work the way you want it to.

Randy Silver:

Fantastic. We'll come back next week for that one.