Software engineering isn't just strategy and sprint plans. It's ambiguity, pressure, trade-offs, and people.
Thriving In Engineering helps you navigate all of it.
In the newsletter, Alex Ponomarev distills 25 years of engineering leadership into practical frameworks to help you lead with confidence, stay calm under pressure, and build thriving teams.
The podcast goes deeper, talking to engineers, leaders, and builders across the industry about what AI transformation actually looks like in practice, past the headlines and hype. One guest at a time.
Hello. I'm Alex Ponomarev and welcome to the Thriving in Engineering podcast. In each episode,
I talk to people from across the software development industry about AI. Today's guest
is Elena Calvillo. Elena, welcome to the podcast. super happy to have you today. Can you please tell
our listeners and viewers a bit about yourself? Thank you so much, Alex. So yeah, I'm Elena
Calvillo. I'm a product manager. I have worked in different industries. I actually started my career
as a front-end engineer. So that has helped me a lot in the AI era to understand and communicate
to engineers. recently I have a newsletter called Prompt-Led Product on Substack where I teach a lot
about what I build and I do there to my audience. So yeah I personally love your newsletter. you
had a very interesting idea in the post about AI taking your job and that you're glad about it. Can
you tell a bit more about that and what changed for you since you know since the moment that you
felt things are changing? Yeah. Yeah, that's a good one. So what changed it is that I stopped
waiting for permission to build. So in the past and in my corporate life so much of the energy
went into titles, processes and many many same meetings. So instead of focusing on the actual
product and the people building behind it, right? So it was just an enormous time of wasting there
trying to align everything and everyone in the same room. So when I were independent in January,
I had no team, I had no engineers, it was all just me. So that felt very good actually. I built
DraftKit that app, the AI adventchalleng.com and still writing right. So the good part is actually
something very honest that I communicate in my newsletter and I think in corporate people spend
so much time trying to protect their titles and instead of encouragement encouraging the teams to
to complete the goal and do something valuable for the product. So once I went free from that
that bottleneck disappeared for me. I'm sure a lot of people in the corporate world envy you because
that's quite a transition. Can you tell a bit more about the AI advent challenge that you launched in
December as far as I remember? But I think it's it's such an amazing product and you got so many
people involved. How how have you managed to deal with this so quickly? So that is a community that
is a product. It was very challenging. I almost have right now at this point 900 people subscribed
into it and like 200 people actively using it daily to learn AI skills. Yeah. So this community
event that's supposed to teach in 25 days an AI skill on a different day for this audience turn
into something h very different from my original idea. I actually didn't expect that a lot of
people jump right into it. So this is something that I want to do each year now and my focus on
this year will be if I can get some sponsors and maybe talk to another tech companies to see what
happens in that space. But yeah I mean it was a huge experience for me. I think it was the first
application that I built entirely with Loveable and Cursor that really make this mind shift for
me. This is huge. I'm sure a lot of companies and people would love to get involved. What
about DraftKit? Is it related to the AI advent challenge or it's a completely different product
that you also built yourself? It's different but it was born because of the AI advent challenge.
So back in that time between November December of last year I was very busy building the AI
advent challenge. So, a lot of people approached me and say, "Hey, I love what you're doing. Let's
collab together on a guest post." And I was like, "No, I'm sorry. I'm so I'm so busy. Let's do that
back in, I don't know, next year or I reached back to you." But I felt that I needed a tool
to actually help me handle those collaborations, right? instead of just using my mind and tell them
well push them back. I wish I had at that point a calendar where I show my newsletter schedule and
probably have some requests instead of using just Substack messages. So that's how DraftKit worked
because of that friction that I had at at that point. And it turns out that it's really something
very different from what I did on the AI advent challenge because that was more like a community
tool. but DraftKit is something different. So it's a workspace for writers to write together.
And I had actually an assumption that the calendar for scheduling and coordinating was like the most
important part of the product. But turns out that after using after listening to my users feedback
it it was not. So that changing to a workspace solution so everyone can share their ideas, write
together before publishing on Substack that's what they value the most. So both are different
products but one was born from a necessity at that point. Yeah. Interesting. You're writing
about things like Product-Led pro - Prompt-Led Product. How are you using this approach or where
are you using this approach when building draft kit? So the idea behind Promt-Led Product is that
you're using AI to make decisions. So this is something that I have seen in the industry also
a lot of companies and teams are jumping into we are AI powered but it's just a label right so
their products hasn't changed a lot they are the same they're just trying to add a small AI feature
to tell everyone that they are AI powered but my approach and the ideal of a Prompt-Led Product is
different it's actually use AI to challenge your mindset to challenge your assumptions and validate
the ideas before building anything even if it's a prototype right so when I built DraftKit I started
to validating this idea with Gemini and Claude before writing anything and that's how I started
to write my product specifications So, like I said, the collaboration was the whole point and
listening to my users was actually something very valuable that I didn't expect to change
in the product. But that small piece of using AI to challenge your assumptions and listening
to the users helped me to redirect all of my energy into the writing experience. Yeah, it's
actually super important because now you can build anything and the the most important piece becomes
deciding what to build and you also write about technical clarity. What what does that mean and
how you use that when building your own product? So technical quality is basically that you're able
to explain how systems work but to people that are not technical heavy users and even how to
communicate with engineers, right? So it makes a lot of difference if your engineers understand
your requirements from the start instead of just guessing what you what you meant in that part. So
just to give you an example of how I do that with prompt engineering I don't for example I don't
write to Loveable or Cursor or any AI tool the API should do this right it's like very ambiguous for
anyone even for AI so I try to be more specific if I'm for example the MCP bridge that I recently
published I was like very structured with it. I need this output in JSON format to not waste
time. I need that the free users are able to read the titles only but the paid subscribers get the
full content of the prompt part. I need that the rate limit is I don't know 50 request requests per
hour or something like that. So that helps a lot into have a good output instead of just wasting
a lot of time and energy in a conversation with AI. That's the product brain MCP right that you
posted. Yes. yes a few weeks ago. Yeah. Yeah. I saw that one. It's It's a really great idea. I
love it. And again, you're setting the example for product managers to build build their own
tools. You you're building a lot and clearly you've learned a lot while building your own
products. If you like in a hypothetical situation, if you join the team tomorrow that you know
nothing about a typical engineering team, what learnings, what knowledge would you bring from
your own products and what you would try to fix in that team and and how they're building things?
So apart from trying to understand the team dynamics more specifically and to
fix something I actually have done this and the first thing that I do is read the documentation
and compare that to the product because I have found that many many times there's a gap in there.
There's a lot of things that are not documented and some things that are very different into how
the product behaves. So that's the first thing that I will do to understand where the friction
lives and what is the first thing that we can fix to to get the behavior right right because a lot
of the limitation it's not like mostly updated and that's that's a huge problem in the industry that
even before AI it has been there a lot of times. So that's how I will do also friction audits
that it's something that I just added into my services recently. So basically it's just stress
test these products and see where the real gap is. Interesting. And maybe you can tell us where
are the typical places to look for those gaps but can you give an example of such friction? Yeah,
for example, a lot of problems happen even in the login, even after start using the product. How you
are on boarding your users? If they are finding things easily and there's a lot of sometimes
disconnection with the premise, what the product supposed to do and what's the real value that the
users get. So for example, I could tell you about DraftKit again that I thought that the calendar
was the most important thing, right? But that was my assumption. So when I listen to my users, it
was like no, I need a place where I can actually collaborate with all other writers because if
I don't have that, then I will just use Google Docs that is free and that is easy to use, right?
And I also got some other feedback about I need to have comments in the in the draft and those things
actually shape what the product does now. So that's what I mean when you said that your premise
is different from what the product is is actually doing for the user. Yeah, that that makes so much
sense. Do you as a product manager, do you rely more on customer development and user interviews
than on product analytics like mix panel, post hoc? I try to boost to use both. I actually now
that I can build my own products something that I do is get the analytics by myself. So I'm not
relying into an external tool. I'm able to track the clicks in there and have a dashboard just for
me. And usually I have a funnel. Yeah. So I did that for the AI advent challenge and it helped me
a lot to understand how people enter into the platform but also what were the challenges on
different days that they were actually completed faster and the other ones that were like more
difficult or probably took more more time for them to to solve right to complete. So that was super
super helpful and I did that with inside level itself. I actually didn't use an external tool.
that is something very easy to to implement right now but I still like to get in touch with
users because there's a difference there between analytics which is raw data that you can interpret
in different ways and data can tell different stories right but your users will tell you
another one so I try to balance both. 100%. Yeah. Do you think what do you think about this general
idea that everybody is going to build software for themselves now? You know the SAS apocalypses
that people are talking about on one hand you've just built in your own rolled out your
own analytics for your product right so that is happening on the other hand you're you're building
products like DraftKit so what do you think what's your perspective for the next five years? I think
it's something challenging and also encouraging that anyone can build. Software has become cheap,
right? So, it's not more like liability anymore, but we also need to be very careful. So, one
thing is just vibe coding for the sake of trying a prototype or even using AI and see what happens.
But another is building a product. And building a product, it's something that you need to
orchestrate from beginning to end. And there's a lot of things around it. It's not just ship
anymore. It's distribution. It's how to monetize it. It's how to announce it. Is the marketing
around it. There's a lot of things. And you also need to be very careful with security and privacy.
because I have seen so many people from the non-technical background for example have started
to build their own products but I'm not sure if they are actually concerned about the privacy of
the users I mean anyone can just hack everything and get the user data and just to put a little
the quickest example So it's kind of worrying on that part, but it's also an opportunity for
anyone to to build and learn about those aspect that sometimes designers ignore in the past or
even engineers or non-technical folks, right? Yeah. Yeah, that makes a lot of sense. Is this
the era of a solo builder or the engineering teams are here to stay? What do you think as a product
manager? Yeah, I think solo builders had never been more powerful honestly and I have experienced
that. I don't think teams are going to disappear. They are needed. We need more engineers. I think
the market is actually opening more jobs for engineers and product managers. But also the tools
that let one person ship compared to what a team of five can do it's actually where the limit is
right. So there's a lot of things around it of getting users supporting them and growing over
time. Those are the basic team team problems that a solo builder will not face. So the solo builder
can now focus on anything around the the product itself and if the idea hits I think that's a great
opportunity. So that's actually what I'm trying to do with the DraftKit that we have now 70 active
users and I'm finding myself yeah I'm finding myself seeing more how to grow this platform and
even doing distribution better. So I'm pondering now within probably getting paid ads or how to
attract people that it's outside of Substack and that's something that I can solve myself
but I need to also be very conscious that it's investing in more time and probably knowledge that
I didn't have in the last year, right? So, growing a business alone, it's it's hard. It's hard. I can
I can say that. Yeah, as a founder myself, I can 100% relate to this. I'm sure that over time you
will find amazing people who will back you up. Before it was a requirement. you couldn't really
do much yourself because everything was harder and much more time consuming. Now you can get as
you were saying you can get pretty far ahead. You launched DraftKit just recently and you already
have 70 paying users. That's that's actually amazing. Congratulations. I'm so happy for you.
Yeah, thank you. Can you tell something about the intersection about of AI and product work right
now that is very exciting for you but underrated as in a lot of people don't don't realize that.
That's a very interesting question. I believe that's something that has been very excited
for me is seeing my new MCP protocol that I just built and I haven't seen a lot of people
doing that. Just a few that I know. So I think that's seriously underrated. And I can see how
that will be the future for a lot of us into how we build how we consume information and how we
search for a knowledge base. So honestly directly through the through an API key I think that's like
part of the future that we are living right now. So like I said, not a lot of folks have done that
with their products or even their content, but there's an opportunity there. So imagine someone
just that pays Claude and connects you through your thinking. That's like wow amazing amazing not
to get just generic AI answers from Google search or anything. So getting the specific patterns and
prompts from someone that you know it's an expert in the industry I think that changes the story
entirely 100%. Maybe it will be the next product you build. Sounds very exciting and useful. Yeah.
Yeah. Yeah. That's actually what I have with the prompt vault, but I just launched it like
two weeks ago. So, I'm just still getting some users in there trying to see who from the paid
subscribers are actually using it more and what are the prompts that they are consuming more. So I
just have a few users in that place. But hopefully it it will grow and I can see how what's more
important to them. Right? So what are the prompts that they are using mostly in their work if they
are trying to use a prompt for stakeholder communication or how to draft a road map
from scratch? I mean I can see how valuable will that be. Yeah, I I love how you're thinking about
this. You're immediately turning some need into a product and you're testing it and sounds like
the gap between people might have this pain point between that realization and like PoC of a product
or MVP of a product, it it is becoming very small, right? Yeah. Exactly. very very interesting times
100%. If someone is super early in their career today in their product manager career
what you would recommend them to focus on? I would say because building is becoming more of a common
ground for anyone. So not just try to follow a tutorial project but actually build something for
yourself. So something that solves a real problem even at work, right? So maybe there's a workflow,
a disconnection between tools that you can improve something that it's already generating friction
for your workflow. So try to build that and ship it and see who uses it. I mean, if it's solving a
necessity for yourself, that's fine. when you get the exposure of building in public that's another
realization I think it changes a lot of your mindset and especially for product managers
that we are constantly driving and getting user feedback that really resonates with the with the
work. Yeah, that's that's a good advice. To often product managers are focusing more, you know, on
assumptions, on personas, on scenarios, but those are hypothetical or interpreting the analytics. As
we talked, this is a totally different approach. Such a great advice. Thank you. What would be the
one thing you would want a person listening to or viewing our conversation today to to take away. So
the tools that we have now has made building so easier than ever and it's actually very easy to
build but it is not easy to ship. So again trying to reinforce that point of building in public
because it really helps. I think shipping means that you have already defined the problem that
you have tested with real users when something is breaking. So put that in front of people and
validate as you said the user personas if they if that is real and keep going with the conversation
on almost instant feedback right now and not just focus on providing a demo because demos are are
easy they are very excited but when you shift that to production into the real So that's what changes
everything. So at this point I would say stop buy coding and start shipping and building in public
more. I love this. a lot of people in product management are talking about building prototypes
by coding features and so on. You have such a unique and different perspective of just building
in public. It it's amazing. Thank you so much for this advice and we can all learn a lot from what
you're doing. I personally envy you to be honest because the because I'm you know I'm a former
engineer. I can build it a lot of things but it takes a lot of courage to to actually ship and
make something public. I would say it's not hard but it is worth it honestly. So take the
risk. Did anyone ever gave you negative feedback about things you shipped? Like very discouraging.
Not I'm not trying to get things personal because I know that well I'm a product manager. I have
always received any type of feedback, right? So even for your own products, don't treat them as
babies. I would say that like I'm trying to be very conscious if at some point I have to shut
down DraftKit for example. I have to to see that in the in the future that it's a possibility.
Product managers we always need to be thinking in that that way. And I won't say that that's
a negative mindset or something like that, but it helps you to be resilient because if you actually
need to kill a product at some point, you know why you know why it didn't work and you can
build the other solution that really satisfies that that need. So I have received a lot of
different feedback. I wouldn't I won't consider positives or negative ones. I'm just trying to
balance in the middle what really the user is trying to tell me. If something is not working
well why why it was not clear to me. Sometimes it's just changing the message in the front end,
just tweaking the the text to communicate the the right things. So that's how I see I see feedback
and I'm not trying to take it all personally. Right. Yeah. I actually love it because
users stakeholders they they just give us information. It's not positive or negative,
it's just information just like analytics is just information we have to interpret. That's
that's a really great point of view. Thank you Elena. Thank you so much for joining us today.
You gave I think nuggets, golden nuggets of advice as an experienced product manager.
We'll continue watching the advent calendar, the DraftKit and reading your newsletter. Thank
you. Thank you so much for having me. I had a great time. Thanks, Alex. Thank you. Bye.