Subscribe
Share
Share
Embed
AI is reshaping product development and the future of work for every role in the tech industry. Every product team used to work the same way: slow discovery, designers as bottlenecks, agile scrum ceremonies that meant nothing because the rest of the company was still doing waterfall, and a measurement phase everyone skipped. The result was feature factories everywhere. In this episode, I break down what product development looked like before AI and what it looks like now — based on my own experience and conversations with founders and engineers building AI-native companies.
---
TIMESTAMPS
00:00 How Product Development Worked Before AI
01:06 Discovery: Slow Data, Slow Analysis
02:17 Design: Always a Bottleneck
03:43 Delivery: Scrum in a Waterfall Company
06:36 Measurement: The Phase Everyone Skipped
07:00 The Feature Factory Problem
07:43 How AI Changed Discovery
09:06 The New PM: Full-Stack Product Builders
12:00 How AI Changed Design: Three Approaches
15:44 How AI Changed Delivery: Four Levels of AI Collaboration
20:00 Tools: Cursor, Claude Code, Codex, Superpowers, nWave
22:22 How AI Changed Measurement
24:16 AI-Native Startups Are Finally Agile
25:37 Daily Sprints and Same-Day Feedback (Andsend)
26:39 The Convergence of Product and Engineering Roles
28:39 What This Means for Your Career
---
RESOURCES MENTIONED
- PostHog — product analytics platform with AI-powered insights
- Andsend — AI-native startup with daily sprints
- Cursor — AI-powered IDE
- Claude Code — AI coding agent by Anthropic
- Codex — AI coding agent by OpenAI
- Superpowers — AI agent framework for software development
- nWave — multi-agent framework for the full product development lifecycle
- Lovable, V0, Bolt, Stitch, Replit — AI prototyping tools
- Figma MCP — integration between AI agents and Figma
- MCP Protocol — Model Context Protocol for tool integration
---
CONNECT WITH PRODUCT ENGINEERS
Host: Peppe Silletti
LinkedIn: https://www.linkedin.com/in/peppesilletti/
Website: https://peppesilletti.io
Product Engineers
Website: https://productengineers.com
Newsletter: https://newsletter.productengineers.com
Community: https://discord.gg/4sMtRNSgU4
---
SUPPORT THE SHOW
If you've seen AI change how your team builds products — or you're still stuck in the old way — share this episode with your team. Drop a comment with what phase AI impacted most for you: discovery, design, delivery, or measurement.
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit
Product Engineers is the podcast about the convergence of product and engineering in the AI era: what's breaking, what's working, and the craft of doing both well.
Hey, guys. Welcome to a new episode of the Product Engineers podcast. In this episode, I'd like to talk with you about how AI changed product development.
I'll go through what was happening before in each role and in each activity, and then make a comparison with what is happening now based on my own personal experience, but also with
the conversations that I'm having with the people in the podcast and other people that I talk to every day. So yeah, let's get started.
My name is Pepe Silletti, and I am a product engineer, and also the host of this podcast, of course, where I'm exploring the convergence of product and engineering roles in, in tech.
So let's talk about the pre-AI world, 'cause that's very interesting. So I, I kind of lived both worlds. So I started to work professionally as a software engineer 10 years ago.
If we think about what happened before the pre-AI era, what would we see in discovery, for example, is that we had to invest a lot of time, first of all, into
gathering data, so both qualitative data from customer interviews, usability testing, and also quantitative data from, like, maybe surveys.
Um, all the competitor analysis, the market analysis, they took a lot of time to do because there was no deep research from Claude to help you do that.
So there was an actual person going on a search engine and looking for all the information they needed.
So there was a time-consuming gathering phase for data, but also the analysis phase was quite time-consuming because if you have all this data, then you need to invest some time into looking for all the patterns across all this data and trying to extract some opportunities and insights that could have been useful to meet our product outcomes.
If you think about design instead, we have very common thing that I've seen all along is that the number of designers in a company is always very low, and designers are always shared between different initiatives, and so they always kind of been spreading too thin.
And when this happens, of course, they become kind of a bottleneck because all the product initiatives depend on the design team.
Most of all, if designers were following, like, a more traditional way of doing, of putting together a solution, that could have taken a long time.
Add to that, that there was design reviews and then iterating over different versions. It really took a long time. So designers were kind of a bottleneck in the pre-AI era. Most of all,
w- it wasn't very common f- to involve a software engineer into the process.
So what usually happened is that once the, let's say, Figma file, the output was out, then once the developer started to implement the, the UI, the user experience, they found out that it was maybe too complicated to do.
So that's, that happens because there was not much communication between the two departments, let's say. Then we have, um, the delivery phase. So in the delivery phase, Scrum got very famous among development teams.
So you had all these teams implementing all the possible ceremonies from Scrum, from having the backlog grooming, sprints planning, poker games, retrospective, all those stuff there.
The idea was to implement Agile in development teams while the rest of the company was not agile at all. So we had this kind of part of the system that was super efficient, maybe producing a lot of output,
but that could have not been used properly by the rest of the company because the rest of the company was still planning making roadmaps for one year,
and they had very specific plans of meeting some, some feature, specific feature releases, which it's, by definition, it's what's called Waterfall, which is, it's completely the opposite of, of Agile.
And, you know, th- this created a, a few problems, of course.
One side, there was kind of psychological, let's, issue with developers because once you work like that, you don't really see the impact of your work because you're just shipping features, and you never see if that's been useful for the product or for the business.
But that's because the, the, the business its- itself is not actually checking for it. And
last but not least, engineers were also bottleneck for some reasons because if I was a PM or a designer that wanted to test multiple ideas for a new feature, for a new product, well, I had to borrow software engineers to create these prototypes and maybe to even integrate them into the existing systems as experiments.
But software engineers have been always busy, full of work, so that was not going to happen very easily. The measurement phase, even here, it's a bit like the discovery phase.
First of all, we needed to instrument our product to start gathering product analytics, and then we needed to wait for data to become statistical significant, which is still a problem today, actually.
But the analysis of this analytics was quite time-consuming becauseWe needed still a person to do that, and the same as discovery, maybe the outcome of this analysis was not always h- heard, and maybe it came too late into the process, so a lot of times it was just ignored.
And it's also for this reason that startups were the most prone to ignore or do less of the discovery phase and the measurement phase because of the time it took to do them.
And since startups are trying to survive the market, they try to cut as much as possible from thing that, at least in appearance, they seem useless.
So what's the main side effect of all of this, of this way of working?
Well, the first one has been the infamous feature factory, and this is a very sad way of working, of course, because you don't really see the impact of your work because all you do is just getting an input to producing an output, and then you don't know if what you have produced has any kind of impact.
But also it's not good for the companies because i- instead of focusing on the outcome, so on the behaviors that we want to change into the users in the system, they just focus on production ve- velocity and how fast we can deliver the features that we think are important for our customers.
So this is what happened more or less in the pre-AI era, and I've been there to see this, and this is what I've seen most of the time. Let's move to the present.
Let's move to what happened to companies that were born in the AI era because they have a completely different way of working, and it's really exciting actually.
So if we go back to discovery, so we still have the problem of gathering useful data, which always requires time, and AI here can kind of be of assistance. AI can help find maybe the right people to interview.
AI can help putting together the questions for the interviews, the questions for the surveys.
It can help with competitor analysis, with market analysis, and also the analysis part of the discovery phase is way faster because once you have all this data, then you can just give this data to an LLM, and they will extract
some good insights that then can be used during the decision-making process. Of course, it's not as easy as it sounds. You need to kind of tweak your agents, your prompts so that the extraction is actually working fine.
Um, otherwise you would have... It, it, it could happen that you will miss some good opportunities that you maybe would have found if you were doing ana- an- the analysis manually.
So there is a lot of fine-tuning to do there, but yeah, it's still way faster than before. The role of PM is also changing because PMs now have way more technical skills thanks to AI. They can be m- way more dangerous.
So for example, for research synthesis, going from a transcription or a survey or a ticket, um, to insight is way faster because you just give your things to Claude, and they can extract everything you need.
Competitor analysis and market analysis is easier, but also it, it's easier to augment the context of your discovery thanks to tools integration through the MCP protocol.
Imagine you, you have all these customer interviews that you have gathered.
You have all the transcripts, and then you can use an MCP s- server with Notion or Jira or Linear, Slack to kind of enrich the context so that the interviews are put into context.
[laughs] And this is pow- powerful, right? Validation, idea validation is way faster, right?
Because now you can put together a prototype with tools like Lovable, Cursor, CloudCode i- in super fast, super fast, and then you can create multiple versions of it. You can test it with your customers.
You can test with users, and it's super easy. Imagine this, this before would have taken probably a developer to help you.
With AI, you can code and experiment on a proto- prototype directly into the e- real code base so that the prototype looks like the actual product, and it's also used within the right context, not out of context.
AI also helps in the discovery phase by finding edge cases a bit easier, which is something that we usually find when we ship something to production because the users start to play with the tool, and then they find new things.
But AI can help with reducing the risk or not covering some edge cases that we have missed. That... It, it's not gonna find all of them, but it's still, like, a good bunch of them.
Another important role that PMs had all along is alignment of vision and strategy and the mission of the product around the company, but also communication b- across multiple stakeholders.
So AI can help kind of tailor the communication that a PM could have based on who's the target audience.
So if I'm talking with engineers, AI can help me communicate with them in a more technical and precise way, but if it's talking with the investors or executives, then it's more of a strategic talk, right?
This is how the PM role is changing, and now you're gonna hear about full stack PMs or product builders. The role of, of PMs is changing a lot, and they're becoming a bit more hands-on thanks to AI.
Let's g- let's get into design instead. Designers have been impacted a lot by AI. You know, think about all the tools like Lovable, V0, Bolt, Stitch.
They can produce working...A working UI, a working front end in a matter of minutes, and a lot of times it's not even that bad. Of course, it's not gonna be great because
it's gonna be like an average kind of design, but it's always good as something to start with and to explore different ideas.
So if we have to kind of categorize how designers work with AI, we can choose three categories mainly. The first one is vibe design.
Vibe design comes from the term vibe coding, but, you know, it's kind of different because I think vibe design suits the activity a bit better since the goal there is to use tools like Lovable, V0, Replit, all those there, to kind of brainstorm different solutions to solve a specific product outcome,
and it's something to look for inspiration, right?
And the good thing is that these tools provide you with a functional prototype that you can click and you can see how it works compared to something like on Figma that is completely static and doesn't really let you see that.
These tools also have an integration with Figma where you can export the design, and then you can tweak it a bit faster because I think that's the advantage of using Figma, that you can kind of tweak everything a bit faster compared to writing a prompt that changes the code.
And then from Figma you can also export it to code again. The second approach is actually the code first approach. So you will see a lot of designers now that start chatting with cloud code,
and they will brainstorm with it or even Codex or whatever.
And then once they have an idea, they ask cloud code to reproduce the design on Figma thanks to the Figma MC- MCP, and then send it back to cloud code for the actual implementation in code. And yeah, the...
Again, this is very powerful because they can have something working, right? And another approach that is actually my favorite is the one of creating a solution, like a new design directly into an existing code base.
So, and this works only if the code base has already a solid design system and has a very well-designed front-end components that can be reused to create a whole user experience.
This is quite powerful because it allows you to prototype directly into the existing product. It allows you to see how the new feature exists in the existing context.
I would say the main theme across all these approaches is that designers now can not only work on the visual aspect, but also on the in- interactive aspect of design a bit easier because it can be translated directly into code, and they can get feedback also from the users way faster because the best feedback we can get is the one we get from users using the product.
Figma or just static, static prototypes have alwa- always had this limitation, so people had to kind of imagine how things would work in the real world. Let's move to delivery.
So delivery is the aspect that touched me in first person because, you know, my background is in software engineering. And to kind of describe how things changed into delivery, I think it's important to
describe what are the four levels of collaboration between AIs and humans, so we understand how the software we c- can be built when AI has less or more control over the code production, let's say.
Well, I would say there are five. The first one could be vibe coding, but I don't consider vibe coding something that belongs to the de- delivery phase.
I see that something more as a tool for the discovery and design phase to explore different alternatives faster or experimentation.
Once we need to deliver in production, we need something that works properly and that it's safe and sound, and it's well designed. So the first level, it would be the AI assisted level.
In, in this level of interaction, we have an engineer who just uses a chatbot, and the chatbot i- is used like a rubber duck.
So it can suggest code, it can suggest solutions, it can review your code, and the human in the loop can still decide to write the code themselves or even ask the AI to write the code, but
they have full control over the intelligence, the co-intelligence.
The second level is the AI, AI integrated approach, which we start to use LLMs that communicate with external tools, uh, using, um, the MCP protocol so that to enrich the context that they can use.
So let's say I'm building a new feature, I can integrate with Notion or with Jira, with GitHub to get more information about what we're working on, and then the output that LLMs are going to produce is gonna be of higher precision, higher quality thanks to a better context, right?
There's also the concept of memory in this level.
So as you work on a project, you may be using different agents to work on the code base, but between chats the context is probably cleared, so you want a way to track the progress.
So the AI stores some information about what happened in the past into this memory that usually is just a memory.md file, and
so that agents in the future can pick it up and continue from where the, the previous agents left off.The third level, it's called the AI orchestrated level.
So in this level, what happens is that we start to create some specialized agents that can help us in each phase of software development. So we could have one that is specialized in planning.
So we give it some requirements, and it helps us understand how to integrate them into the system, so creating a plan. Then we have the actual implementation, so the, the implementer.
Then we have the tester, the one that does QA, the one that does the code reviews. The role of the software engineer here changes a lot because we're not the ones writing the code anymore.
We are the ones orchestrating all the agents and making sure that they work together in a good fashion. Our role becomes more like the role of a designer of systems and also coordinators of these agents.
The fourth level is the AI autonomous level, where we have agents that are completely autonomous, twenty-four/seven, and they can perform different tasks while we are sleeping, like code reviews.
They could look for bugs, updating libraries, looking for improvements. They can- we can get really creative with this, but the thing is they are just there. We're not orchestrating them. They are autonomous.
They are launched by a scheduling tool, and then, yeah, we can still review work they do, but we're, we're not the ones controlling them.
There've been a lot of tools created for software engineers that kind of place this- themselves in different levels, the ones I just described.
So if we take, for example, Cursor, the famous IDE, or CloudCode or Codex, they kind of span all the levels because they are used in different ways.
So in level one, they could j- just be seen as chatbots, but in level three and four, they become autonomous agents that can work in the cloud for you while you're away or while you're sleeping.
Then we have also tools that kind of fit into level two, between level two and level three, which are Superpowers, for example, or nWave.
These tools provide a framework that is composed by a set of agents that are specialized in each phase of software development.
Actually, nWave goes even beyond that because they specialize also in discovery, in the architecture design, the testing design. The thing about these frameworks is that they
are trained to follow specific rules, specific quality checks, specific way of working so that the output of this framework is actually pretty good.
It's been guided very well from the moment we give a requirement to the moment we have the outputting code.
For example, Superpowers, they have this initial brainstorming phase where we kind of chat about what needs to be done, and then once we get into the implementation phase, it uses a test-driven development approach.
I think what these tools are trying to do is kind of to bring some discipline in how AIs w- should work. And if you think about it, it resembles a lot how w- good engineers have been working all along.
They've been using good engineering practices that have been making their job quite easy.
So these tools are trying to kind of codify these good practices with agents so that they can be followed, and that's one of the way to reduce the problem that we can have with non-determinism.
I'd kind of invite you to try them because you would see the outcome is quite good.
The only thing that could surprise you is that they are a bit slower than using, like, a chatbot directly, but that's because they start this kind of loop where they have to go through all these checks before giving you something back.
So of course it's slower, but it's supposed to be like that, and that's why the output is so good.
So at the end of the product development life cycle, we have the measurement part, which, I mean, actually it should be done even before starting delivering something because you want to have the
tools integrated in your products that can actually measure what's going on in the product. But yeah, what changed
with AI is that now all these events that we are gathering, all these analytics that we are gathering, they can be analyzed way faster now thanks to AI. And LLMs are pretty good at pattern recognition.
I think that that's one of the best thing they can do. So now we can use AI a bit, a bit, a bit at the same as we do in discovery to prompt this amount of data and get some good information out of it.
Example, one of the companies that does this is PostHog, which pro- they provide a pro- product analytics platform. I mean, I'm not affiliated with them a- at all, but, you know, it's, it's just the tool I know best.
They help you query the data. They also help you ex- extract insights or maybe anomalies or automatic trends or even getting a summary of what happened b- between replay sessions.
A few years ago, when I had to watch a session replay from a user, it would have taken a long time because I had to see what w- was happening and see what behavior the user had.
But now I can just ask the AI, and it's going to save me a lot of time. It's probably from hours to minutes.
So analytics becomes more of a conversation with your data, and it's way more accessible to people, even to people that don't have a background in data analytics. So the AI native
startupsAt least from the conversation I'm having and from what I'm seeing, are the ones that finally are giving the right place into the world to Agile. So finally we can be agile again. As I said in the beginning,
we always kind of implemented Agile in the worst possible way. We just gave Scrum to delivery teams hoping that it would change something. But [laughs] we just thought, "Okay, everyone is doing Scrum.
We do Scrum the same way because it's just the way it is." But as I said, an agile delivery team in a non-agile business is completely useless.
So now with AI, we have reduced a lot of the bottlenecks that we had before.
We can design way faster, we can write code way faster, so we can finally put in practice the real teachings from Agile, which is to gather feedback faster from our users and from our customers. This is now
possible, and it is super easy, and if you're not doing it, [laughs] you're really missing out. I've actually interviewed one of these AI native startups, it's called Unsend,
and the CEO told me that they have this concept of daily sprints.
It's very fun because what they do is just that they meet in the morning, they have their own stand up, and then they think, "Okay, what's the hypothesis that we have to test today?
What kind of experiments we want to run?" And that's actually their day. So they think about that, they think how to approach it, and they build something, and you know, with AI this is way faster.
And they sometimes they get feedback even in the same day because of that speed.
And I keep saying speed and, you know, going faster, and I just want to make sure that you understand that when I say fast, speed, you know, velocity, all these things here, I'm not referring to the speed of
building something, which is also important, but it's more about the speed of learning. That that's the most important fun part in product development.
There is no excuse not to do it properly anymore and to be actually agile. There's also another trend.
I mean, if you are on this podcast, you probably know already about it, is the convergence of product and technical roles in tech.
This is something that I've been exploring for the last four years, and I've seen a lot of changes changing in that sense.
You now we have these hybrid roles called product engineers or product builder based on what background you have, and the idea is very simple.
Now everyone can start their job by thinking in product outcomes, thinking about what kind of impact they want to see on the product, and then starting their work from there, not from a GR specification or something that someone else told us to do.
And th- th- th- these new hybrid roles, also thanks to AI, can work end to end on product initiatives, so they can go through all phases all at the same time,
either alone, which is probably more common in startups, but also in teams.
So for example, in Unsend they have this concept of product groups where they have two or three product engineers working together to do a different version of pair programming, but for the whole product thing, pair product development, something like that, which is, is pretty cool.
This is very important that you keep in mind that this is happening because it's going to affect the careers of everyone in product and engineering, even though I consider product everyone at this point.
It's very important that we experiment more with these tools, that we find new ways of working. We kind of try to see what are the alternatives that we have nowadays to achieve our goals. That's gonna be a,
a big deal when it's about the future of our careers. That's it for today. I tried to give you an overview of pre-AI and post-AI, and present actually, not post-AI, it's actually the present. Hope you enjoyed it.
If you like the episode, please think about subscribing to the YouTube channel if you're on YouTube, or to follow th- the show on your favorite streaming platform, and yeah, to the next one