Subscribe
Share
Share
Embed
The Question is a collaborative learning podcast about Design Systems. Smart people like you sign up, answer a few niche questions about design systems for each episode, and then we all get together to unpack the data we've gathered. Each week, I'll invite a new co-host to help facilitate the conversation. After the deep dive, the co-host and I record a recap of what we learned. That means, for each episode, you can listen to the recap and the full deep dive!
If you're a design system practitioner, subscribe today (https://bencallahan.com/the-question) to receive an invitation to each episode. This only works if the community joins in!
Stay in learning mode ❤️
Ben Callahan (00:01)
Here we go. Hello, system thinkers. I'm your host, Ben Callahan. Welcome to episode 073 of The Question. I'm very excited to welcome my friend Davy here. Davy, thanks for being here, buddy.
Davy Fung (00:12)
Thank you for having me.
Ben Callahan (00:14)
Long time listener of your show as well. So it's really fun to collaborate with another voice in the space, especially one as generous as yours. Thanks for all your time and energy, It means a lot to have you here. This week, you and I talked a little bit about, we had a long conversation about what should we cover. And we landed on this concept of AI for automation in design systems. And so...
we asked four questions of folks. The first two were very similar with just a subtle difference and they were, ⁓ when you think about your current design system workflow, what percentage of your time think could be automated away with AI today? And the other one was what should be automated? So the difference between what could be and what should be. And we gave folks a percentage scale to choose from. And then we asked two open questions. ⁓
The first of those was, in what areas of our work do you believe we should avoid AI automation and why? And the last question, in our world, craft used to mean beautifully and elegantly designed and implemented interface work. In a 2026 design systems context, what does craft mean to you? And so we sent this out to over a thousand design system practitioners. We got 101 responses this week, which is an amazing amount.
And as always, wherever you're watching or listening to this, you can click the link to get the raw data and dive into those results yourselves. Gosh, Davy, where do you want to start, man? I feel like we covered a lot of ground in an hour and you and I are just hopping back on right after the deep dive that happened this Thursday. So where do you want to begin?
Davy Fung (01:51)
Yeah, a lot of this I think goes towards that if you could zoom into that chart that has the bar, let's see the bar chart. Yeah, that one. Yeah, that what tipped this off was zero heights, design system report where they asked the question, do you have automation set up for your design system? And 37 % said yes, but cumulatively 63 % were in the no bucket. So 32 % said no, but we plan to.
And then 31 % said, like, absolutely not. Like, no. So what I wanted to gauge from our community, the question community, was how, for those that are not interested, like, why not? Is it a reluctance based on, like we mentioned, like, the avoidance and ⁓ changing the way we work? I know that we do get a lot of design leaders
Ben Callahan (02:22)
Yeah.
Yeah.
Davy Fung (02:48)
And even like above design leaders, like leadership overall in our company saying, how can we use AI? Let's interject AI into into all of this work. But there may not be a meaningful, just a practical sit down of ⁓ what can be automated. This should be automated. What should be manually curated and done as a group? know in the responses also, there was some sensitivity towards moving away from rituals that are ⁓
Ben Callahan (03:08)
Yeah.
Davy Fung (03:18)
help us connect as designers, right? So those are the types of things that I wanted to hear. That specific piece wasn't mentioned ⁓ in our session, which was ⁓ interesting.
Ben Callahan (03:21)
That's right.
Yeah.
Yeah, it's funny, man. When I think about this stat from the Zero Height Survey, actually am... The way that I would consider that through the lens of the data we just gathered is this group right here, which are the folks who decreased their answer from question one to question two. So question one here was like, could be automated with AI today? Question two is what should be. So folks who decrease, they answered, say, 50 % could be...
but only 25 % should be, that's a decrease between the two. And those folks, it was like interesting to see sort of look at the rest of their answers, right? They believe that AI could automate more than it should. ⁓ And they kind of got into this whole, like if you look at their answers to Q3, they had this sort of belief that something important gets lost when we automate things away with AI.
And so it gets it like that 63 % that aren't doing it, you know, from the chart you shared above. Like, why is that? Well, there's some interesting things that's always framed in the context of loss. So what are we losing? We're losing organizational context, institutional memory. We're losing learning through doing. We're losing relationships that are forged in the work itself. We're losing the quality control at scale, right? So all of these are the kinds of things they said in their later answers.
Davy Fung (04:36)
Mm-hmm.
Ben Callahan (04:58)
How do you feel about that? mean, do you agree with this?
Davy Fung (05:01)
I don't agree with the loss of organizational context because that's more of a ⁓ reliance on a single point of contact. Obviously, ICs, we all strive to be a point of contact for things. ⁓ We strive to be known in organizations for something. So either design tokens, documentation, automation, that sort of thing. But what if this is all documented? What if this was all brought into
skill. Does that take away my point of contact? Ness of does that take away my subject matter, expertise title? I don't think so. But it does make it more scalable. ⁓ I think a lot of the ⁓ especially like the language with the word loss is I think that it's a sensitivity towards like losing control and having the
Ben Callahan (05:36)
Yeah.
Yeah.
Davy Fung (05:58)
the AI or the automation do the work for us. ⁓ I don't believe the loss in learning is necessarily there unless you're coding automations and coding processes and having no ⁓ knowledge of how this should be done.
Ben Callahan (06:16)
Yeah, I think that that loss of learning one is really interesting. And a couple of folks in the deep dive brought up this idea that like, if you think about the maturity of your processes, so you start with a not I don't know how to do a thing. And then you investigate and learn how to do it. And you probably do it manually to figure that out. And then at some point in the maturity of that process, you say, I'm tired of doing this manually, and it's time to automate.
Their point, I think, was like the more mature a process gets, the more ripe it is for automation. And I like that sort of model of thinking because in my mind, that means we've done the work to learn the lessons that we get from doing the work. Now we can automate it, right? ⁓ I assume that we're going to get 80 to 90 % of the learning if we take that approach versus just starting with the idea of automation. Like, I don't want to do that thing I know has been lurking in my to-do list. So I'm just going to...
Davy Fung (07:06)
Mm-hmm.
Ben Callahan (07:11)
ask AI to do it for me. Well, gosh, yeah, then you are going to miss out. You you are going to miss on some things. so we did address that one. ⁓ Yeah, the organizational context one or being the point of contact or like the expert in an org. I do think people want that. And I also wonder, you know, that's not super sustainable for the organization, right, to have like an individual who's the person who's going to have that knowledge trapped in their own mind.
So maybe it's a better solution, right? To have it out of an individual and into a repository somewhere or something.
Davy Fung (07:45)
Yeah, it's a scalability issue, right? So like what if like my manager will always say like, what if you get sick or what if you go on vacation and who's going to be able to drive these like all like the the wacky stuff Davy's been doing? And then I would say, well, it could pause, you know, like they don't need to to run it. But it is in the org's best interest that I capture how to utilize all of this stuff. Obviously.
Ben Callahan (07:46)
Yeah.
Yeah.
Yeah.
Davy Fung (08:10)
Like we did talk about in the deep dive, sometimes we're creating tooling that is solely for ourself, which is fine. We're creating single use Figma plugins, NPM scripts to do specific tasks. ⁓ It would be great to be able to document those lightly so other people could use them. But I think as long as we're getting ⁓ use out of it and we're saving time and we find value of it, I think that's also OK.
Ben Callahan (08:37)
Yeah, I think that's a great point. And I mean, I was, I did this yesterday with one of my coaching clients. We were talking about a thing after the session. I was like, you know what? Like I could make that in about five minutes with Claude and I did and sent it to them and they were like, my gosh, this is great. And that's a single use thing. It's never going to be reused. It's just to solve a very specific problem, very specific context. And it's perfect for that. You know, it's, it's, it's so I think there are definitely those use cases. ⁓
this sort of like avoidance reasoning. this again is comparing the folks who decreased their score saying we should automate less and the folks who increase saying we should automate more. And the reason that they gave for
when they listed the things we shouldn't be automating on the decreased group was like philosophical. It was like a challenging my humanity to offer this up for automation. The other group who was more maybe biased towards the use of AI, it was more technical. It was about like, hey, we don't think AI is ready for this kind of thing yet. So very different reasoning there. ⁓ Really interesting to sort of pull the data part in that way.
There was so much that we could talk about. I actually want to ask you, if you wouldn't mind, to give us sort of some clarity, because I think in the beginning of the deep dive, I picked up on some tension in the chat and in some of the conversation around people conflating automation with AI. So how would you separate, how would you define those two terms in the context of our conversation here so that we can see that they're distinctly different?
Davy Fung (10:16)
Yeah, think automation on its own doesn't have to be tied to AI, like the methods like Cron and like writing scripts to do jobs, repeatable jobs, like that's been around for a long time. I think with the new tooling,
Ben Callahan (10:28)
Yeah.
Davy Fung (10:30)
Now, we could designers can use Claude and other agents to be able to do that. So I think the lens of this is looking at how can we utilize like our new tooling to create automations and be OK with letting go of some of the work that we used to do that is more ⁓ craft oriented or let's say like artisanal artisanal design, right?
Ben Callahan (10:55)
Yeah, yeah, I love it.
I love it. think that's great. And I think ⁓ the more we got into the conversation, I think people understood we were asking about the overlap of those things, right? Like using AI to automate. so a little nuance there, but it's always nice to just be really clear with those terms. I feel like that comes up in every episode. You know, it's when you're getting a hundred answers on a topic, people are going to kind of pick out specific things and like, you know, challenge the framing and all that, which is wonderful. That's why we love the community. So. ⁓
Let's see, we talked about the maturity of the process. What else here? Yeah.
Davy Fung (11:31)
I did want to touch on like the design,
like the physical design tooling portion of it, because we talked a lot about like Claude and new tools. Like we didn't talk about any of the other like new suite of prototyping tools or such, but there was ⁓ no talk about Figma and where most designers are still working. And like there's this narrative, obviously, like on X where ⁓ every few days, like Figma is dead.
Ben Callahan (11:50)
Yeah.
Davy Fung (12:00)
yada yada is dead. especially if you're working in a enterprise forum, obviously that is not true. And if we pull the data, it is not true. the time might fluctuate. And over the next six to nine months, the iteration time, the time spent to ⁓ modify components, maybe less. But there wasn't a lot of talk about. ⁓
Ben Callahan (12:00)
Yeah.
Davy Fung (12:30)
I like to maintain components and design them and build them on my own. Or I prefer to start ⁓ now with the new tools in like Codex and Cloud. I would like to see if I could use ⁓ Codex or Cloud to generate them. And then I go in and refine them. So I did go through that exercise just as a pilot to theirs. And I'm sure this is a case for many designers is.
Ben Callahan (12:34)
Hmm.
Davy Fung (12:55)
There's drift between code components and design components, or there is simply just no parity, like components are missing. So I wanted to see utilizing the new Figma MCP tools and in the future also using the Figma console MCP tools. How can I ⁓ code gen these components back into Figma? What is the quality like? How many passes do I need to do? And upfront, the quality's not great.
If you're not hooking in your own design system, MCP, the tooling is not quite there yet. But over ⁓ a few passes and writing skills to refine and knowing what the what the guardrails are, it eventually got there. And I created, would say, in these these two dozen components that I've created, 80 or 90 percent of the work was done through codecs. And then the 10 percent was through
manually using our plugin linting, applying the right tokens, applying the right instances, moving things. ⁓ But it's still needed that we need to then go and have the human in the loop. We still need to go back in and modify it, but I spent significantly less time ⁓ building them up front.
Ben Callahan (13:57)
Yeah.
Yeah, it's weird. think I am. I've done some of the same experimentation, David, you know, sort of I'm just curious, what trying it sort of repeatedly the same kinds of tasks over the last six to 12 months with different iterations of the models and tools just to see sort of how is it evolving? I feel like I don't it's hard. It's really hard in this space where things are constantly changing to have like a control, you know, like something to compare against.
because it's always different. And we were talking a little bit just in between the two sessions here about the explosion of tools and like the sort of the FOMO that comes with a new tool being released every week and a new model, a new version, a new whatever. And I think it's especially hard in larger organizations where a leader is having that same FOMO and maybe one month says, hey, I want everybody invested in X, Y, Z,
Davy Fung (14:47)
Mm-hmm.
Ben Callahan (15:13)
suite of tools and then they read some other thing and now we want to move on to this because it's better and we've invested so much time in building workflows around that tooling. I'm looking forward to a future where there's a little bit more portability of the workflow between the tooling. And I don't know how far we are from that but ⁓ man it would be nice because it definitely feels like I'm starting from scratch each time I go to do it.
Davy Fung (15:41)
Yeah.
Yeah. I mean, I think what the design leaders want to see is just the appetite to try and ⁓ introduce these new things into the workflow. So I think specifically for my project where I was working on utilizing the new Figma MCP tooling, they just wanted to prove a concept to know whether like whether it's there like right. It has the and like you said, like has the control improved?
Ben Callahan (16:05)
Yeah.
Davy Fung (16:09)
Like, and I would say the control hasn't proved. But if your stuff doesn't, if you have code components and there's no semblance of design components, you're not, you're not going to get to where you want. If there's some foundational components, you'll get there, but you need to, you need to, it's the, the always sunny in Philadelphia.
Ben Callahan (16:35)
Yeah. Yeah.
Davy Fung (16:35)
like the the meme, right, you have to
connect the dots and you have to help them connect the dots. But they want to know that we're trying and they want to know where it's at. And all they want to do is then report back to their their manager, right to say like, we've we've tried it. This is where it's working. This is where it's not working. I'm not sure this is a good use of time.
Ben Callahan (16:48)
That's right.
Yeah, it's so true. think the challenge of course, on the other side of that is, is exhausting for teams, you know. And we saw some of that in the, the deep dive, you know, Keisha shared this in the chat. I'm fatigued from the constant changes, ups and downs, shiny new tools, all of this, and lots and lots of folks kind of jumped in agreeing with this, you know. In my mind, this is one of the reasons we do something like the question or Redwoods, because no individual can stay current on all of these things.
Davy Fung (17:03)
Yeah.
Ben Callahan (17:24)
all the time. so having a group of folks who are, you know, sort of like when they have the energy experimenting and reporting back to the community is a really good way to, sort of be informed at least on what's happening and where we should be spending that energy. So, we asked folks, well, I actually kind of wanted to call out, you know, Bill, Bill had this sort of on the idea of like, what does it cost to automate things? liked Bill's perspective here.
He kind of said, you know, there are risks of using AI, right? And the one that he specifically called out using AI for automation, the risk that he called out is that in his experience, when he's, you know, this is kind of precursor to AI in his world as an engineer, when he was automating things, he oftentimes would find himself trying to automate a process that actually didn't make sense. And so the process of trying to automate it forced in his mind, the conversation of wait,
we should rethink how we're doing this. And I think he's saying AI is powerful enough to probably go and automate some stuff that you probably shouldn't be automating. And so we missed that check, know, that sort of the checks and balances that maybe come with a more manual approach. Is this something you've experienced at all or do you agree?
Davy Fung (18:40)
Yeah, there is a thread ⁓ from this and then another from another designer, Doug, where we were talking about ⁓ where we should interject some of this ⁓ automation. And I think the one thing, if you think that there should be some more oversight in the CI pipeline, let's say you wanted to do token linting or component linting or accessibility linting.
put that into the pipeline, write that as a reusable thing. And I had a discussion with one of the design techs on our team is it doesn't necessarily need to mean that you pass or fail the pull request. All we need is a signal. So that is another way for design system folks to gather data and understand how folks are utilizing the system. So I don't necessarily see that as a negative thing.
It's not a blocker by any means. You can make it a blocker, but I also think that ⁓ with all of this stuff like you see engineers now ⁓ sort of in the weeds with just the sheer amount of code reviews that people need to do. So we do need the help help there.
Ben Callahan (19:55)
That's true. Yeah.
Yeah. And I was just reading ⁓ an article that was talking about this idea that like, the more we sort of like bake AI and more and other automation techniques into the generation of code, the more responsibility we're pushing to the PR and it's, it's having to do things that it hasn't had to do in the past. And then it's actually never really designed to do. And so like being a little bit
smarter about like where we interject the human aspect of things earlier in the process means we can let the PR be the PR and not have to have like not vetting like did we even build the right thing, you know, by the time we get to a pull request. So I think there's some smartness there for sure. ⁓ We definitely we got a lot of folks to jump into the Fig Gym with us and share all the ways that they're already using AI.
to do automation. So there's a ton of that in the FigJam wherever you're listening or watching this, can, there's a link in the, in the description to get access to that FigJam file and you can see that long list of things. ⁓ I mean, lots of stuff about prototyping. ⁓ Randy was talking about what you mentioned earlier, Davy, which is like building like very sort of small purpose driven tools for like maybe even just a single use, ⁓ you know, automation for tokens, of course. ⁓
GitHub workflows, creating dashboards, lots and lots of stuff. So it's really cool to see how people are already using it. ⁓ Yeah, fun conversation today for sure. And there's tons more data in here. Is there anything else that stood out to you that you want to hit on here, Davy?
Davy Fung (21:37)
Yeah, I'm glad that it's beyond prototyping there because I think internally, as design system practitioners ⁓ at Alaskan anyways, there is a big push to move into a higher fidelity, utilize the prototyping tools to create ⁓ significantly better prototypes than just the noodley prototypes Figma used to produce. But I think there is a bit of a struggle with what
can we utilize them? What can we utilize ⁓ AI to help us with? I'm glad there's a plethora of things here going from the single use plugins, ⁓ auditing. Obviously, idea generation, I think, is the one that a lot more design generalists are doing. then also, think it's really, ⁓ LLMs are really great for crawling information. ⁓
Ben Callahan (22:33)
Yeah.
Davy Fung (22:35)
confluence or if you have zero high and you're able to let loose into stuff like that or even Google Docs, right? ⁓ It's great, like amazing for that. even we didn't talk about this type of automation that I'm really keen on, but Slack automation. So automating design system report, automating and ⁓ capturing specific key terms. So going into like your help channels and capturing things like
Ben Callahan (22:52)
Hmm.
Davy Fung (23:04)
Figma or breaking change or specific keywords that you think maybe dark mode dark mode is one that people like to gripe about right. And then utilizing slack mcp to bring those those tidbits in and I think the the problem with slack is like it's a double edged sword right it's easy to communicate in slack but there's a fire hose of information that's when you utilize the slack mcp to to pull these into post it's for you and act on them.
Ben Callahan (23:11)
Yeah.
Yeah, so good. And you're right. It's definitely I think prototyping was like the early thing. Well, we can do this with AI and it's it solves, you saves us a little bit of time. But I think the smart folks are looking for really thin sort of slices of work to just have AI help me all along the process. You know, I love that rather than trying to have some like very wide.
you know, like AI do all of this for me throughout the entire process. I wanted to help me in stages, you know. And that's a nice way, I think, to sort of evolve and as the tools mature, maybe we can allow it to do a couple of these things in a row. But it's nice to sort of break it down. And that's really what's represented in that FigJam with all of these ideas for ways that people are actually doing it. Yeah.
Davy Fung (24:15)
This is great. Yeah,
I'm so I'm so happy that it's beyond just I want to use cloud design to generate a dashboard or.
Ben Callahan (24:22)
Yeah, right.
Lots more information in the Fig Jam. ⁓ I captured some really fun quotes from folks, especially around around what craft is. I think it's a it's like a moment in the space, you know, for us to constantly evaluate this word, this idea of craft.
And some of these I thought were just really good, especially this one from Sam Samantha. She shared that, and this was just directly from her answer to, think, question four. She said, AI is a manifestation of the average. Craft is rising above the average. I love that comparison, you know, and it's, and it helps you. think statements like that can help us clarify where do we, where do we bring AI into the process, you know? So, ⁓ really fun. It's always great to hear from such smart folks in the community. Davy.
Appreciate you, man. You rock. Thank you so much for being here. Thanks for sort of bringing this question to the community. It's always fun to talk to you and I appreciate the generosity you have with your knowledge and experience.
Davy Fung (25:21)
Thank you so much, Ben. Take care. Bye bye.
Ben Callahan (25:22)
Yep, we'll see
you next time. Cheers.
Ben Callahan (25:25)
Thank you so much for joining us on this episode of The Question. Remember, you can get access to the raw data, the collaborative FigJam, and all of the recordings for this episode and many more on my website, bencallahan.com
If you or your team could use an outside perspective on your design system program, I'd be honored to support you in that way. There's more information about my coaching and consulting offerings over on the site. Thanks for being here and remember, stay in learning mode.