{ "version": "1.0.0", "segments": [ { "speaker": "Karl", "startTime": "0.0", "endTime": "12.48", "body": "You just need to think in terms of what would a potential buyer of our product or our solution actually ask ChatGPT. Like, it sounds kinda silly, but, like, as long as you're distributing it across multiple sites, you should see Lyft." }, { "speaker": "Jack", "startTime": "13.045", "endTime": "22.645", "body": "In today's episode, karl from draft.dev is sharing the results of a survey they conducted with DevTools marketers. Let's go. So what is the survey that you did?" }, { "speaker": "Karl", "startTime": "22.725", "endTime": "37.48", "body": "Yeah. Yeah. So, you know, with draft.dev, we're always look in the last couple of years, there's been a ton of change, to be honest, Jack. Like, some good, some bad, some I don't know yet. And so we're always trying to get a pulse on, like, where's the market going?" }, { "speaker": "Karl", "startTime": "37.48", "endTime": "49.184998", "body": "Where's where's it been? Where are marketers in this space, like, spending money? Where are they getting ROI? Where are they, actually investing their their hours and stuff? How are they actually using AI?" }, { "speaker": "Karl", "startTime": "49.184998", "endTime": "76.635", "body": "All those kinds of questions. And, like, we obviously ask people those sorts of things on sales calls all the time and, you know, I probably have, I don't know, a 100 to 200 sales calls a year. So I get a fair number of data points there, but we want to do a more, like, structured, formal kind of questionnaire here this year. This is our first year doing it. So it was partly collected in person, partly collected, remotely depending, you know, we we go to conferences and collect it or we, send it out as a survey to our clients and prospects, people we've talked to." }, { "speaker": "Karl", "startTime": "76.635", "endTime": "98.524994", "body": "So it was great. I mean, we got around 50 responses, not a massive data point, you know, number of data points, but at least like a somewhat representative sample. I mean, realistically, there's only a few 100 companies that are, like, big enough DevTools that it's worth talking about. So I think we did a good enough job with coverage there where I feel good in the results, but maybe not scientific level results, but at least for marketing surveys." }, { "speaker": "Jack", "startTime": "98.524994", "endTime": "105.085", "body": "I think isn't it 30 is usually the minimum? I mean, probably depends, I felt. I think 30 is" }, { "speaker": "Karl", "startTime": "105.564995", "endTime": "130.335", "body": "I'm not a data scientist. I'm not gonna claim to know what statistical significance could be, but I I think either way, you know, you get enough diversity in results where you start to see, okay, there's some alignment on certain things and somewhere it's like, man, maybe nobody knows the answer here. Anyways, there's some big highlights came out of this for me. One is, you know, 2025 budgets were very constrained, in the first half of the year in marketing. There was a lot of fear around what are tariffs gonna do to the general economy." }, { "speaker": "Karl", "startTime": "130.735", "endTime": "159.50499", "body": "What you know, The US has such a outsized, weight on the developer tool economy because a lot of spending is here and a lot of the talent is here and the companies get funded here. So when The US is in a bit of a political economic uncertain time, it sort of ripples through the whole industry. So we saw a lot of clients, you know, anecdotally pulling back on spending, being more cautious, saying we're gonna wait on projects. And that that really, that was the story of 2025. And I talked to a lot of agency owners that work in the DevTools space." }, { "speaker": "Karl", "startTime": "159.50499", "endTime": "193.755", "body": "Almost everybody had that same, you know, sort of pullback, last year. This year, that is not the case. We when we did this survey, about 62% of the teams that we talked to are increasing their budgets, and most of the rest are basically holding steady. And so there's only like about 10% that said they're likely to decrease budgets, which that's a very small number. So my feeling there is that, all the new investment coming into AI and the the, you know, stabilizing of, well, least for the moment, The US economy, has kind of like given people a little more confidence to spend in marketing." }, { "speaker": "Karl", "startTime": "193.755", "endTime": "206.3", "body": "And so that's sort of a good thing overall for those of us who work in the space, but I think it's also good to know that your competitors are probably gonna be spending more this year than they were last year. So if you, you know, you wanna keep an eye on where they're going, probably important to know that." }, { "speaker": "Jack", "startTime": "207.485", "endTime": "215.565", "body": "Yeah. And was there a pattern among the companies that are increasing or holding steady in those that aren't?" }, { "speaker": "Karl", "startTime": "215.565", "endTime": "229.88", "body": "You know, not really. Not that not that it's easy to point out. And we asked this survey anonymously too. So, like, we don't really want to, like, go in and be like, you know, trying to dissect which companies are, you know, spending more and which aren't. That's not really our our goal here to make anybody feel bad." }, { "speaker": "Karl", "startTime": "229.88", "endTime": "246.995", "body": "And part of the reason we can get good results is people don't they can't share their company, you know, because they're not gonna tell you, that. So no no big pattern. I mean, was across the board. Like, the smaller start ups are getting funding right now. That seems to be if you've gotten a good AI story, you know, you should be spending more because you've got funding." }, { "speaker": "Karl", "startTime": "246.995", "endTime": "264.21002", "body": "The bigger companies tend to be if they've started to implement AI in their workflows or starting to leverage it to get, you know, better products out, they're still really in demand and they're still feeling like good about spending and marketing. So it seems across the board, but you know, there's probably some variability in there." }, { "speaker": "Jack", "startTime": "265.35498", "endTime": "278.07498", "body": "Yeah. Yeah. Because I guess like, I guess my question, be curious like around it's like you always think, okay, so what is this what's the impact here on DevTools? Right? Like the AI growth." }, { "speaker": "Jack", "startTime": "278.07498", "endTime": "292.12", "body": "There's oh, obviously, if it if you're selling AI, then that's great. But if you're selling, you know and probably if you're selling compute, that's probably also great. Yep. If you yeah. I mean, that sort of close things." }, { "speaker": "Jack", "startTime": "292.12", "endTime": "302.82498", "body": "But then how does it how does it play out for something that could potentially be in the realms of the sort of thing that you see on Twitter where they're like, I just vibe coded this" }, { "speaker": "Karl", "startTime": "303.625", "endTime": "303.865", "body": "Yeah." }, { "speaker": "Jack", "startTime": "303.94498", "endTime": "320.55", "body": "X alternative. Would it be a pattern that those companies were spending less that are yeah. Basically, it's like, I guess, trying to dig into is is this kind of people buying rather than build sorry, building rather than buying." }, { "speaker": "Karl", "startTime": "320.955", "endTime": "369.34", "body": "There's some fear that there'll be more building because building is cheaper, especially I've heard this in the platform engineering space where in the past few years, there's been a big migration towards things like Backstage for for example, for platform engineering. And then there's a number of tools port, think is another one that's kind of in the similar vein. Because these tools are basically just kind of like a way to track and monitor and, administer Kubernetes essentially, like there's some concern in that space that like more people will just build that stuff on their own because it's a fairly again, I'm I'm simplifying here, but for a lot of companies, it's a fairly straightforward set of tasks. For really big enterprise, I think that probably gets less true, but for small to mid sized companies, do you need a backstage or a port? You know, probably not to be perfectly honest." }, { "speaker": "Karl", "startTime": "369.34", "endTime": "405.875", "body": "So some of those some of those kinds of companies may see some headwinds or may see a shift towards like a specific part of the market that's more open to spending on it. But again, like the overall, the overall pattern or what I've seen is like, because there's money trickling into AI tools, they've still got to buy developer tools behind the scenes. I mean, even these, you know, ChatGPT or Claude or whatever, they're, they have hundreds and thousands of engineers building real stuff and then deploying it and running security checks on it. Like those people are still doing productive engineering. Like the AI agents aren't replacing them yet." }, { "speaker": "Karl", "startTime": "405.875", "endTime": "424.15", "body": "Now, will it happen in one, two, three, five, ten years? I don't know. But at the moment, at least the sort of boom here has basically trickled down and seems to be benefiting all developers. The other bit is that a lot of companies are shifting their products towards more of an AI story. Some of that's a little bit of just marketing, you know, fresh coat of paint on things." }, { "speaker": "Karl", "startTime": "424.15", "endTime": "429.27", "body": "Some of it is like legitimately, they're they're using AI in interesting ways. So just depends." }, { "speaker": "Jack", "startTime": "429.59", "endTime": "431.43", "body": "Okay. Yeah. Yeah. That's that's helpful." }, { "speaker": "Karl", "startTime": "431.88498", "endTime": "457.345", "body": "Yeah. The other thing that I will say about this, you know, looking at these kind of on the AI topic before we get off that, is that there's in marketing teams we talk to, a lot of people are experimenting with it right now, but there is no consensus on how it's best used or no standardization around how companies are using it. So one of the things I would ask is like, okay, have you played with AI? 96% of people yes. Great." }, { "speaker": "Karl", "startTime": "457.345", "endTime": "462.865", "body": "That makes sense. Of course. We're all in tech. We hear about it every day. But only, like, 7% of people are like, oh, yeah." }, { "speaker": "Karl", "startTime": "462.865", "endTime": "488.695", "body": "It's unlocking significant value for us. It's saving us lots of hours very steadily, not requiring a ton of manual overhead and and rework. And this is just November that we finished this survey, December. So this is not, you know, this is three months old data, not that old. So I was a little surprised by that because we, you know, my number would be much higher as far as like how much it's helped us at draft.dev in the last year or two, but I thought that was interesting." }, { "speaker": "Jack", "startTime": "489.175", "endTime": "496.45502", "body": "Yeah. That's actually very surprisingly low. Did they say what things had been helpful to them?" }, { "speaker": "Karl", "startTime": "496.615", "endTime": "510.15002", "body": "Yes. So we've asked kind of about the categories of things that you're using it for. Where is it helpful? The biggest area was con like written content editing and planning. And that seems to be pretty much I won't say across the board, everybody's doing it." }, { "speaker": "Karl", "startTime": "510.15002", "endTime": "540.18", "body": "I think it was somewhere around 60% of people say they're using it for that. Consistently helping. But what the a lot of people mention is that it still requires like an actual knowledgeable subject matter expert to catch any hallucinations or like make sure that things are like subtly correct and sound like a real engineer would would create them or whatever. And that that's always been one of the trickiest bits about using AI in this space is like, we're not selling the best body soap here. We're selling very complex technical products that are changing really quickly." }, { "speaker": "Karl", "startTime": "540.65497", "endTime": "563.13995", "body": "So if you write something based on, you know, the all the LLMs are writing stuff based on source material that's often a couple years out of date or something, they can get off really quickly and in subtle ways that are very hard to catch. So marketers has they struggle to trust these tools because right now, they're not that accurate. And I think that's something we've gotta see improve if we wanna really see them used at scale in in dev marketing at least." }, { "speaker": "Jack", "startTime": "563.13995", "endTime": "572.635", "body": "Yeah. I guess that's the obvious one that I would have guessed, I suppose, the content yeah. Yeah. It's interesting. Surprisingly, most" }, { "speaker": "Karl", "startTime": "573.115", "endTime": "596.78503", "body": "Yeah. Some people are using them for, like, ads and doing dynamic ads and managing ads across different platforms. There's some AI use cases for that creating copy and more AB tests and things like that, I think is interesting. There, you know, are some people who are using it to help with, kind of lead qualification, which also makes sense. Again, like, if you're a marketer and you're getting leads in consistently, how good are these leads and can you score them?" }, { "speaker": "Karl", "startTime": "596.78503", "endTime": "619.47", "body": "And AI does a good job of like building some kind of contextual knowledge around names and emails and other other data that's structured. So there's some use cases there that are interesting. But I think one thing that I found also as I like dug into why isn't this stuff getting used? Yeah. Part of the reason is that companies haven't figured out exactly how best to deploy it and let their teams use it in a productive way." }, { "speaker": "Karl", "startTime": "619.47", "endTime": "636.04", "body": "So for example, I talked to a marketer who's at a, at a big cloud company and they were like, yeah, we we're allowed to use these tools. Certain ones are on our white list or whatever. Certain ones aren't we're not allowed to, but we don't have a company account that lets me share these things with other team members. So I'm over here using Gemini. Guy The next to me uses Chad DevT." }, { "speaker": "Karl", "startTime": "636.04", "endTime": "658.685", "body": "The other guy over there likes Claude and, you know, like so they've all got their own little workflows they're building in silos, and it's not, like, standardized or really sanctioned even. I think that's that's part of the problem as as to why they're limited in their usefulness utility is that people just aren't like they're not all using them together. Everybody is like, each man for themselves trying to, you know, get the most they can out of this." }, { "speaker": "Jack", "startTime": "659.245", "endTime": "677.375", "body": "That's so interesting. I was speaking to one of my friends who's not in tech well, bugs at like a SaaS company, but not not very, like, in the way we would think of it. Yeah. And she's doing she was doing, like, a presentation on how to use Copilot at the company. And she was like, it's so great." }, { "speaker": "Jack", "startTime": "677.375", "endTime": "695.11005", "body": "Copilot's amazing. And my friend and my my other friend was there, and he is like a data scientist more or less. And he he like secretly uses all these tools. Copilot is the one that they have internally. And he's like, it is terrible." }, { "speaker": "Jack", "startTime": "695.67004", "endTime": "697.19", "body": "He's like, I can't" }, { "speaker": "Karl", "startTime": "697.43005", "endTime": "698.15", "body": "That's right. And he" }, { "speaker": "Jack", "startTime": "698.15", "endTime": "709.005", "body": "was going to her like, it is rubbish. Like, there's so much better stuff out there. You just Yeah. And that was kind of I wonder if there's the bigger companies where it's like, you have to use That's right. They're kind of" }, { "speaker": "Karl", "startTime": "709.165", "endTime": "721.485", "body": "my yeah. Absolutely. I think those of us like you and I, Jack, we work at small enough companies where we can just go try something and bring it in. And if it saves us time, our bosses are you know, I'm gonna say yes. Your boss is gonna say yes." }, { "speaker": "Karl", "startTime": "721.5", "endTime": "730.62", "body": "You know? Yeah. I'm the boss in my case. But but either way, like, it's the adoption curve is nothing. There's no, like, no adoption curve other than Carl brought in a tool and it looks cool." }, { "speaker": "Karl", "startTime": "730.62", "endTime": "745.025", "body": "Let's do it. But my wife works in health care and we were talking about this and she's the same way. She has Copilot. That's the only one they're allowed to use. It's a even an older version of Copilot because they're not allowed to upgrade models as frequently because of all sorts of BS behind the scenes." }, { "speaker": "Karl", "startTime": "745.025", "endTime": "782.53", "body": "And so her output is very poor in relative terms, for LLMs. And, you know, this is one of the things that I think we forget in tech sometimes is even though, you know, they say these tool the agent tools are gonna be good enough to displace so many white collar workers in one, three, five years, whatever the timeline is, the adoption cycles still take time. And there's just like like, if there were a way to shortcut that, we'd be in a different world than we are today. And I'm not saying it won't happen, just saying that it may be a little longer than we give it credit for because of these, company internal adoptions and changing human behaviors and procurements that are just painful, and and that stuff doesn't go away just because AI is really good." }, { "speaker": "Jack", "startTime": "782.77", "endTime": "803.015", "body": "Yeah. Do well, I guess that is that's a huge factor of it. Do you think there is also, like is there a fundamental difference between the stuff, you know, it's incredibly useful for programming, obviously, for marketing. Is it just less useful as well? Is that a part of it?" }, { "speaker": "Karl", "startTime": "803.175", "endTime": "826.945", "body": "I I would say no. In our experience at draft. Dev so last year, we heavily reworked our whole production model around AI workflows. And so we built several you know, we've we've got multiple tools working in tandem, and we've got engineers in the loop at key points in the process. And it took us used to take us four to six weeks and several rounds of revisions with real humans and editors to get a single piece of content out." }, { "speaker": "Karl", "startTime": "827.425", "endTime": "866.815", "body": "That was the old way. And now it's two or three hours with one engineer and a couple different LLMs in the loop to get the same quality of content consistently finished and delivered to clients. So I will say that for us at creating content, it has been a huge boost to productivity and our ability to output. And then just our ability to respond quickly to like, you know, what's coming up or what's changed, new releases, things like that that we would have had a we would have taken weeks to get stuff out before, now we can do it in a day. So not every company wants to put out a mass amount of content, but right now, one of the big edges that that smaller startups are getting is around optimizing for LLMs." }, { "speaker": "Karl", "startTime": "866.815", "endTime": "900.05", "body": "So AEO is what you know, maybe maybe it's being called or GEO depending on who you ask. And the only way to optimize for LLMs is basically put out content that shows them and tells them how your product works, how it compares to others, what it does well, etcetera. And, you know, generating more content and putting it on more places is just that seems to be working right now as far as, like, improving discovery through AEO. So I don't think this is gonna go away or be, like, insignificant. I think this is gonna be a huge lever for companies that can lean into AI or AI powered workflows or agencies like ours that use AI to to power this stuff." }, { "speaker": "Karl", "startTime": "900.05", "endTime": "908.93", "body": "We can, you know our prices essentially, like, went in half last year when we implement all this stuff because it was just that much more productive, and there's no reason to keep prices high." }, { "speaker": "Jack", "startTime": "909.17", "endTime": "919.225", "body": "Done. How how does it work in the sense? What so could you even just like at a high level, like how does this Yeah." }, { "speaker": "Karl", "startTime": "920.025", "endTime": "933.89", "body": "Yeah. Yeah. Mean, very high level, like, you know, there's there's a lot of streams of input that you need to create good technical content. You need to know about the products you're talking about. You need to know about the sort of integrations or other products around it." }, { "speaker": "Karl", "startTime": "933.89", "endTime": "970.275", "body": "You need to know about the alternatives and market around it. So we collect a lot of this stuff manually and and through conversations with clients and whatever to get a bunch of context. And then there's a whole other feed of information of, like, stylistic information, whether it's preferences the client has or it's just best practices that we've implemented or it's little things you can do to make AI sound less like AI. Just all that stuff needs to get fed into the the initial thought. We so we we do that and then for each piece of content, there's a a manual process of creating the, like, specific technical expectations for the piece of content that's going be generated." }, { "speaker": "Karl", "startTime": "970.275", "endTime": "995.185", "body": "So a real engineer, in our case, we've got a couple 100, you know, that work around the world and a real engineer comes in there and helps with that initial building of the prompt that gets fed into a model. Another model checks that model, and then the engineer gets something that they can do a final quality control check on that's somewhere around 75 to 80% of the way there. And then they can do the things like make sure it didn't hallucinate any facts or version, you know, numbers are actually correct. It makes sense. Check actual code to make sure it runs." }, { "speaker": "Karl", "startTime": "995.185", "endTime": "1000.865", "body": "And again, I think there's a world where we could automate a lot of that stuff too, but we're just kind of like picking away at the pieces as we can." }, { "speaker": "Jack", "startTime": "1001.665", "endTime": "1008.99", "body": "So can it do what kind of what kind of content does it is it able to do? What kind does it struggle?" }, { "speaker": "Karl", "startTime": "1008.99", "endTime": "1030.955", "body": "So it is let's see. That's a good question. Right now, we have used on every type of content that we've ever produced for for clients. So everything from tutorials where we show hands on how to actually build stuff, all the way up to like higher level thought leadership stuff. What we have is a series, there's different prompts for each of those different types of content." }, { "speaker": "Karl", "startTime": "1031.4349", "endTime": "1049.955", "body": "And there is a little more manual work on certain edge case things. So for example, if a client just released like literally in the last two weeks, something that hasn't been indexed by LMs has very little documentation, no use cases. Yeah. You can ask the LM to write a piece of content around it, but it's probably gonna be pretty wrong. It's gonna take a lot of manual work to fix." }, { "speaker": "Karl", "startTime": "1050.915", "endTime": "1067.86", "body": "That's pretty rare though. I mean, if we're building out, you know, more, you know, standard types of content tutorials, we we tend to, you know, be able to to to get pretty close in the first draft. But, yeah, it it it works. I was, you know, it's funny, Jack. You're you're saying all this, like, the same things I asked last year was like, is this even possible?" }, { "speaker": "Karl", "startTime": "1067.86", "endTime": "1109.105", "body": "Like, seriously, like, because it felt like our processes were just too complex and too many subtle things, but they've the tools have gotten a lot better, and they're now better at checking each other, which has been a really big game changer in, like, getting things to a level where you can reliably count on them more often. So, you know, when you ask one LLM to check another one's output, you often get a lot of, like, potential issues, and then we can flag those for a human editor to actually look at and review instead of making, you know, a human editor go through line by line and measure every single thing. So there's tons of little things you can do like to chip away at the productivity. Same with like client comments. So client comes in with comments about how they talk about their product." }, { "speaker": "Karl", "startTime": "1109.3451", "endTime": "1119.69", "body": "AI processes LLM processes that, and then feeds that back into our inputs for the next piece of content we do for them. So every time a client's giving us revisions, we're basically getting better at it." }, { "speaker": "Jack", "startTime": "1121.13", "endTime": "1121.9299", "body": "Interesting." }, { "speaker": "Karl", "startTime": "1122.01", "endTime": "1131.545", "body": "Yeah. I don't wanna get too far on the, like, just talking about draft.dev. Mean, I could talk about it all day, but I also feel like I'm happy to share more reports and and data and stuff too." }, { "speaker": "Jack", "startTime": "1131.545", "endTime": "1152.06", "body": "But I find it interesting. I guess my initial and and I'll be curious on this. My initial reaction was like, well, I know some tutorials have taken me days and embarrassing number of days Yes. To do well. And it's like constant checking." }, { "speaker": "Jack", "startTime": "1152.06", "endTime": "1162.355", "body": "There's always like things in it. I just go through and build it and I make sure that someone go like, imagine I'm a person coming in, like, I understand this? Would I understand this? Would I do it? All those sorts of stuff." }, { "speaker": "Jack", "startTime": "1162.3551", "endTime": "1173.37", "body": "Go through each process. Like, is it progressive? Is it all this kind of thing? And I guess when you're talking, I'm just running through each of those difficulties and thinking." }, { "speaker": "Karl", "startTime": "1173.37", "endTime": "1173.9299", "body": "Right." }, { "speaker": "Jack", "startTime": "1174.0901", "endTime": "1179.37", "body": "Does it make sense that an Aloha can do that? And I suppose it's" }, { "speaker": "Karl", "startTime": "1179.53", "endTime": "1203.2799", "body": "probably There's gonna be edge cases where it can't. I'm not gonna tell you, like, it's gonna 100% every piece of content we do, and we'll have when that happens, what we do is we take its output and we see where was where did it go wrong? We try to learn from that and, like, feed that back into future prompts and get better. But sometimes, like, to your point, sometimes the information out there is wrong or not out there. And that in those cases, you know, everybody's gonna have a hard time with that tutorial." }, { "speaker": "Karl", "startTime": "1203.2799", "endTime": "1209.5199", "body": "It doesn't matter if you're an engineer with ten years of experience or an LLM that's just pulling data from the Internet." }, { "speaker": "Jack", "startTime": "1210.48", "endTime": "1212.1599", "body": "That's true. Yeah. Yeah." }, { "speaker": "Karl", "startTime": "1212.345", "endTime": "1227.465", "body": "But Yeah. Anyway, say all this because I think what's interesting about this is that, yes, we're doing this right now. I think other companies will do this in the future. I don't think we have any magic that like we've learned that other people we're wrapping LLMs. We're not doing anything that sophisticated." }, { "speaker": "Karl", "startTime": "1227.465", "endTime": "1260.8099", "body": "So my my ultimate question is when content production becomes cheaper and cheaper over time, where's the where do you get your edge in marketing? Where do you get, you know, in front of more people? And I think distribution is gonna be a bigger and bigger challenge and and question mark for people. Right now, putting out a lot of good content, technical content and making sure you show up in LLMs consistently when buyers are looking for what's the best DevTool to do X or whatever. That's a great strategy, but that may not last for another year or two as we get agents actually doing purchasing on behalf of people." }, { "speaker": "Karl", "startTime": "1261.05", "endTime": "1281.7449", "body": "So then again, we may go through this big shift again, like how do you distribute better for agents and agentic workflows and payment cycles and stuff? I don't know what that's going look like Jack, but that's something I think about at night. Like the how do we keep distribution like, keep building an edge in distribution and getting better at getting stuff in front of whoever the buyer is? Do" }, { "speaker": "Jack", "startTime": "1282.225", "endTime": "1283.985", "body": "you have initial thoughts on" }, { "speaker": "Karl", "startTime": "1285.98", "endTime": "1296.62", "body": "I have a couple of theories. So this is and this ties back. I'll tie it back. I'll I'll do a little tie it all together. This ties it back into an area that a lot of people are playing with right now, which is influencers." }, { "speaker": "Karl", "startTime": "1296.94", "endTime": "1323.88", "body": "When we pulled out, like, which channels that you use right now are you experimenting with and which channels drive the most consistent ROI, the number of people like, basically, it's a it's a a one to one. Influencers in video marketing has the lowest ROI that's measurable of any channel right now, at least that from the people we talk to. But it has the highest experimentation rate. So everybody's trying it right now. And what's cool about that, and I think they're on this I think this is not dumb." }, { "speaker": "Karl", "startTime": "1324.04", "endTime": "1366.5651", "body": "I think it's like any new marketing channel. You're trying to push things on the edge and get your little bit of like, you know, delta that you can over your competitors. And these are new channels that don't have sort of known ROI and it's gonna be risky, but it could pay off big. And, you know, people in our space love that sort of trade off. So the reason I mentioned that is that I do think that human distribution, the actual following you can build right now is going to be a very important asset in the next one to five years because if I can, if I start to realize that LLMs are kind of manipulated by people building lots of content and doing a really good job with distribution, where do I go besides just, you know, writing with LLMs?" }, { "speaker": "Karl", "startTime": "1366.5651", "endTime": "1379.9701", "body": "I I gotta go to my friends. I gotta go to the people I follow. Like, I think this has always been one of the other channels that gets kind of like neglected when when when talking about developer marketing. It's just that that word-of-mouth. How do we get more of the word-of-mouth?" }, { "speaker": "Karl", "startTime": "1379.9701", "endTime": "1410.49", "body": "And and I think influencer marketing is essentially a version of that as one answer to that. Others are doing it on Reddit. Others are doing it through communities. The number one channel for ROI, because again, this is kind of like the same idea, is human connections at events and community events and, you know, meetups. And what I think is interesting about that is it just shows you how, like, as the the digital channels of marketing get saturated and flooded, people kind of go the other direction towards in person stuff again." }, { "speaker": "Karl", "startTime": "1410.6649", "endTime": "1416.9049", "body": "And so kinda cool to see it. I mean, again, it's maybe not I don't know if that's good or bad for us at draft.dev, but it's just an interesting data point." }, { "speaker": "Jack", "startTime": "1417.065", "endTime": "1429.61", "body": "Yeah. I guess, like, content a lot of technical content becomes quite similar to you don't necessarily go there for, like, the human experience personality that you want to sometimes you do, maybe." }, { "speaker": "Karl", "startTime": "1429.61", "endTime": "1430.65", "body": "Sometimes you do. Yeah." }, { "speaker": "Jack", "startTime": "1430.9701", "endTime": "1446.705", "body": "There is, like, incredible pieces, but it's usually the stuff that's kind of interest. Like, if, you know, Ben from PlanetScale writes a amazing blog post on databases, like, it's just a unique and it's his personal thing and it's skill and LLMs probably won't be able to do that for long And" }, { "speaker": "Karl", "startTime": "1446.705", "endTime": "1471.325", "body": "I would argue that that's like influencer that's influencer marketing. That's like thought leadership stuff. I mean, it's like, I know he's not an influencer for sale, but, like, he is clearly an influencer who has his own personal cachet to anything he writes. Whether an LLM wrote it for him, which I doubt, or he wrote it himself, he he is going to like, because he has that following and that reputation, he is going to attract an audience. And I think that's gonna become more and more valuable." }, { "speaker": "Karl", "startTime": "1471.325", "endTime": "1486.5099", "body": "So founders doing things like building themselves up into influencers or building their own followings and their own kind of reputations, that's gotta be a a factor if you're starting a business now or you're you're working in DevTools now, probably much more than it was five or ten years ago." }, { "speaker": "Jack", "startTime": "1486.6699", "endTime": "1505.975", "body": "Yeah. I I think it is. Yeah. I think you're right. It's like twit Twister personalities, YouTube, like Primogen, all these kind of I mean, on plus all the smaller ones, but, you know, I think it is it's funny how there is also just this kind of like lifestyle developer lifestyle stuff as well." }, { "speaker": "Jack", "startTime": "1505.975", "endTime": "1513.4", "body": "Just like Primogen. Most of it's not super, super technical. It's more like lifestyle stuff of like Yeah." }, { "speaker": "Karl", "startTime": "1513.4", "endTime": "1532.485", "body": "Well, like all the it's all like all those programming subreddits. Right? Like, yes, there's some actual influential interesting tech in there, but also there's a lot of memes about what it's like to be a programmer Yeah. When your boss interrupts you. I mean, you know, just all this this stuff that we we're, like, in this crowd where we have these in jokes, and, you know, that's kind of a a human thing." }, { "speaker": "Karl", "startTime": "1532.485", "endTime": "1538.885", "body": "It's just to, like, have your in jokes and your your your crowd that you feel part of, and devs wanna feel bad. Like, that that makes sense." }, { "speaker": "Jack", "startTime": "1540.0599", "endTime": "1547.2599", "body": "Yeah. True. Yeah. What Carl, were there any other interesting tidbits in the survey?" }, { "speaker": "Karl", "startTime": "1547.5", "endTime": "1588.185", "body": "Yeah. You know, I'll I'll I'll mention also the the challenge right now in content that came up over and over from this survey, but also anecdotally I hear is this, like, if you've been building SEO for the last five or ten years like a lot of our clients have, you're really you're probably struggling a bit. Last year, there's a huge challenge where what they call zero click results with, you know, basically Google has more and more weighted towards like the AI written results, especially for high level terms like what is a data engineer. Now you could argue that, I'll I'll I'll tell a quick story on that because I think that that topic specifically is really funny. I remember four years ago at Draft." }, { "speaker": "Karl", "startTime": "1588.185", "endTime": "1608.1", "body": "Dev, there was what is a data warehouse was a topic that no less than I I'm not joking with you. No less than 10 clients requested. We write an article called what is a data warehouse. And I, every time, just cringe because, like, it's such a stupid fucking idea for an article. Like, I mean, a, it's like a Wikipedia Wikipedia is gonna be the default source." }, { "speaker": "Karl", "startTime": "1608.1", "endTime": "1628.4349", "body": "Why am I going to, like, x company's blog to find out what is a data warehouse? They just everybody felt like this is a high, it was they all did the same keyword research. This is a a large volume keyword with very low competition because it's brand new and it's coming on the scene. It's a cool thing to talk about. And so let's all write our big, you know, 3,000 word piece on it." }, { "speaker": "Karl", "startTime": "1628.63", "endTime": "1646.915", "body": "We did a million of those and it was fine. I mean, there some of them are good, some of them are great, some of them are meh, but like, whatever. Like that was a strategy that worked for top of funnel traffic. Now my problem with that article is not really that it's a bad article or that it's bad content. It's just that there's no intent to buy your product just because you define data warehouse really well." }, { "speaker": "Karl", "startTime": "1646.915", "endTime": "1667.63", "body": "And just think about like the, you know, if if five years ago, Jack, you're asking what's the data warehouse? I've never heard that term before. The first article is some company and you read the article and, yeah, it was pretty good. Like, sure, you've got their brand name in, but like you're not buying your product right there. Like, it's not like I need to go buy your data lake solution just because I read your really good article about what is a data warehouse." }, { "speaker": "Karl", "startTime": "1667.63", "endTime": "1702.6349", "body": "So it's not a bad piece of content, but it's pretty like low intent as far as like actually getting users. And so people would have write these, you know, low intent, high volume keyword things and get a lot of vanity traffic, but that doesn't really move the needle for, for like actual, I don't know, users. And so Google started to take those results and basically just say, we'll just summarize it ourselves with AI. So very few clicks start going through to, to people who've been optimizing for these high level keywords. A lot of companies we've worked with or, or still work with saw huge dips in traffic as that that took off or kinda happened last year." }, { "speaker": "Karl", "startTime": "1702.7949", "endTime": "1724.4299", "body": "And now what's what's happened is we've gotta focus more of the content towards the middle and bottom of the funnel because that's where buyers start to come to the actual site. So what I mean is if you think about your buying behavior, it's probably shifted a lot in the last year, Jack, like mine has. Like, was searching for a new microphone not too long ago. I didn't Google best microphone. I asked Chad GPT." }, { "speaker": "Karl", "startTime": "1724.59", "endTime": "1737.8649", "body": "Hey. I'm a blah blah blah, and I do a podcast, and I'd like to have a good microphone that meets these criteria and is under this price point. Where am I gonna what are my options? Right? And it gave me half a dozen options, and I went and made a purchase, like, based on that." }, { "speaker": "Karl", "startTime": "1737.865", "endTime": "1765.6349", "body": "I think that's a much more common vector now than searching what is the data warehouse and figuring out, oh, that company wrote a really good data warehouse article. So now I should follow them and two years later maybe buy their product. Like, it's just that's a long cycle. So you could see how AI can really cut down a lot of those steps, but you gotta optimize content that hits that middle to bottom of funnel. And a lot of people in the, you know, that I talked to around the survey were were talking about their shifts in in marketing strategy around that, and I thought it was pretty interesting." }, { "speaker": "Karl", "startTime": "1766.355", "endTime": "1789.745", "body": "Yeah. I mean, tactically, it's it's, like, actually pretty straightforward. You just need to think in terms of what would a potential buyer of our product or our solution actually ask Chad GPT. Like, it sounds kinda silly, but, like, that's pretty much it. And then at the moment, we the tools are pretty limited as far as what kind of, what kind of insight they give you into the prompts, the best prompts to optimize for." }, { "speaker": "Karl", "startTime": "1789.745", "endTime": "1802.5", "body": "But either way, as long as you're directionally thinking about those kinds of questions and then building content that answers those questions and then distributing it across multiple sites, you should see lift, and and that tends to work for for our clients at the moment." }, { "speaker": "Jack", "startTime": "1802.66", "endTime": "1847.555", "body": "Interesting. Yeah. And distributing how are people does it need to have, like, good? Like, I always think if it's just a if you're Ben from Planet's Go and you've just written this incredible article, you can just post it everywhere and other people will start posting attack news and stuff. But if you're if you've written an article, I don't know, the best data warehouse for x y z specific low funnel stuff, where do you distribute a piece of content that is pretty like, it's I mean, it's not it's not gonna make it a hacker news." }, { "speaker": "Jack", "startTime": "1847.555", "endTime": "1848.675", "body": "You know? Yeah. Where does" }, { "speaker": "Karl", "startTime": "1848.675", "endTime": "1856.0399", "body": "the Right. Right. I think I think you're you're I'm gonna paraphrase your question. Where do you distribute a shitty piece of content like that? You don't know." }, { "speaker": "Karl", "startTime": "1856.0399", "endTime": "1870.3451", "body": "Kind of. Yeah. I I I joke I joke a little bit because, like, so I I get what you're saying. I think this is gonna be the, like, multibillion dollar question in dev marketing in the next few years. I I don't think I have, like, a perfect answer for you." }, { "speaker": "Karl", "startTime": "1870.3451", "endTime": "1907.9199", "body": "I can tell you what we do today and what we try today and what kind of seems to work, but I can also tell you that it doesn't feel like we figured it out or anybody's figured it out. So what we do today is we have clients post, you know, content to their own site. We have similar versions of all that content that goes out onto other sites like DevTools or Medium or our writers individual blogs, LinkedIn. We ask clients and we get work with clients to get that content shared across both their company and founder and key employee LinkedIn pages so that you've got more organic, like, you know, reach there. Newsletters, we do, as well, a lot of paid promotion and distribution through newsletters now." }, { "speaker": "Karl", "startTime": "1907.9199", "endTime": "1926.1951", "body": "That's been more interesting channel than it has been in the past. And so I think that's something you can think about for really high value pieces to get more distribution. And honestly, again, like, there's a lot of like, I mean, there's people trying a lot of other things. There's people trying to hack stuff into Reddit. I don't know." }, { "speaker": "Karl", "startTime": "1926.1951", "endTime": "1940.75", "body": "Maybe it works. I'm like fifty fifty on that. I feel like it's it's kind of a a short term thing. Reddit will crack down on it or the the platform will get devalued because it's just full of junk. But but, yeah, there there's a lot of other you can find niche communities and Slack groups." }, { "speaker": "Karl", "startTime": "1941.39", "endTime": "1965.6499", "body": "I think to me, what's what's what's most, compelling is finding a few key channels that work consistently for you and then rinting and repeating on those and just just kinda like trying not to overcomplicate things. Because as long as you do have some distribution, you probably get indexed, but you certainly don't wanna like just post it and hope. You wanna do a little more than that." }, { "speaker": "Jack", "startTime": "1966.29", "endTime": "1997.78", "body": "Yeah. There's this company that I kinda met the founder and incredible guy, and they have they just absolutely just crushed it, grew insanely well, and London based. And then they their whole thing was like SEO, and they were huge into SEO. And then the whole, you know, the whole story that isn't told that was like behind the scenes part was that they had a ton of people in The Philippines, like creating backlinks for them. Yeah." }, { "speaker": "Jack", "startTime": "1997.78", "endTime": "2017.1849", "body": "And that was like kind of the sort of secret source was actually just this army of people just creating backlinks. Yeah. And does is that I guess, is that still, like, the kind of dirty secret of how all this stuff works? Yeah. It can I" }, { "speaker": "Karl", "startTime": "2017.1849", "endTime": "2048.215", "body": "mean, can It's a good question? It can help. So let let's walk through like how an LLM answers a query like, what is the best data warehouse for my very specific use case? Often what they're doing is looking at their existing index of content and then likely also or in certain queries, they're Googling as well, or they're doing some kind of search to get more current information. And so part of the strength and weakness of LMs is they are all building off of traditional search anyway." }, { "speaker": "Karl", "startTime": "2048.375", "endTime": "2095.335", "body": "And so if your if part of your strategy all across the board to get noticed in LLMs is ranking well in search to make sure that they pick us up when they go do a query based on the latest results, then you probably want to do some level of backlink building. Now it doesn't have to be like that kind of, you know, link warehouse type stuff where you've got millions of people hacking in your backlinks into other sites. I think the more effective way to do it is like building links in a more organic and natural way where they actually make sense. Like, so going to your partners and asking if you can write, you know, pieces on their their blog or or update some of their their pages to have references to you where it makes sense. Going to your investors, to your employees and asking them for those sorts of like just kudos mentions." }, { "speaker": "Karl", "startTime": "2095.7349", "endTime": "2114.53", "body": "I you know, we don't right now we don't actively do any backlink building like that, so I can't speak to a ton of personal experience, but I do know that some people are doing that. I I get a lot of outreach for all of our websites to add different you know, brands wanna get added to certain listicles and things like that. That's all it's just part of the game. Yeah." }, { "speaker": "Jack", "startTime": "2114.69", "endTime": "2127.445", "body": "Yeah. It's interesting because I feel like it's one of those ones that's just not spoken about as much as Yeah. Because I was I'd kind of dug into SEO a little bit, and then it was just like, oh, actually, this is the secret. It's just like, probably changed." }, { "speaker": "Karl", "startTime": "2127.445", "endTime": "2156.085", "body": "But I think it's really hard in in DevTools, backlink building super hard because if I went to one of those agencies, they wouldn't have a bunch of other really deep technical content to reference me from. They'd go to like consumer electronics brands and try to like work in backlinks to, you know, sneak or something. It just doesn't really make sense. Like, it it's and so there's a lot of debate in the SEO world about how much is that backlink that's not really relevant even worth. And my answer is it's probably not worth that much." }, { "speaker": "Karl", "startTime": "2156.085", "endTime": "2168.14", "body": "But if you get you get you go with the sneak example. Like, if Okta is backlinking to sneak, that probably means something. Something like that. That's a big deal. These are both well known, you know, brands in the in the space." }, { "speaker": "Karl", "startTime": "2168.14", "endTime": "2190.9949", "body": "And so like there's a lot more weight to that backlink. And so I yeah. I I think again, if you're trying a bunch of backlink stuff and it's working, I'm always curious to hear about it. I've not heard a lot of our clients in DevTools get a ton of lift out of that sort of like backlink, you know, it's not even black hat, but just that kind of like going out and trying to pay for a bunch of links and doing it at mass. I just haven't seen it." }, { "speaker": "Jack", "startTime": "2191.555", "endTime": "2196.57", "body": "Okay. Super interesting. Yeah. Yeah. Okay." }, { "speaker": "Jack", "startTime": "2196.57", "endTime": "2200.09", "body": "Any other exciting insights?" }, { "speaker": "Karl", "startTime": "2201.45", "endTime": "2212.5051", "body": "No, man. I I think, we covered a lot of my my favorites. You know, this this data report, we're we're gonna try to do this every year. We'll have more probably more companies participate next year. I think it was really good first time to do it." }, { "speaker": "Karl", "startTime": "2212.5051", "endTime": "2231.45", "body": "So if you wanna go to draft.dev and download the whole thing, there's little bar at the top, they'll show you how to do it. And then I'm always happy to chat about this kind of stuff. I geek out on on learning where this industry is going. So I think right now we're in like the greatest state of change that we've seen in the last five or ten years. And and it's an industry that's high change no matter what." }, { "speaker": "Karl", "startTime": "2231.45", "endTime": "2232.995", "body": "So that's saying something." }, { "speaker": "Jack", "startTime": "2234.0352", "endTime": "2239.955", "body": "Yeah. True. True. Yeah. It is quite the change." }, { "speaker": "Jack", "startTime": "2241.3152", "endTime": "2248.49", "body": "And how is, on the side thing, so how is life as a multi agency owner?" }, { "speaker": "Karl", "startTime": "2248.49", "endTime": "2258.41", "body": "Yeah. Yeah. If you wanna, like, figure out how to make your life more challenging, you know, buy a second company. That's that's the the end of the end of that story. No." }, { "speaker": "Karl", "startTime": "2258.41", "endTime": "2278.27", "body": "It it's good. So, yeah, for a little context, like, I started Draft. Dev about six years ago, and then a couple years ago, my partner and I bought a podcast production agency. I it's very interesting. This is totally off the DevTools marketing track, but it's so interesting to, to own." }, { "speaker": "Karl", "startTime": "2278.27", "endTime": "2280.1099", "body": "Yeah, we can edit this out if you want. I don't care." }, { "speaker": "Jack", "startTime": "2280.1099", "endTime": "2283.15", "body": "No, I'm sure people won't find it interesting. Yeah. Okay. Well, good. Well, good." }, { "speaker": "Karl", "startTime": "2283.15", "endTime": "2315.8452", "body": "So, so interesting how other industries do not think the same way a lot of developer tools and technical, very deeply technical companies do. So our client base at the podcast consultant, this other company we bought, is almost all financial advisors and investors, very long term thinkers, very slow and steady wins the race type people. They don't worry about the eighteen month funding timeline that every small startup we work with has to deal with. Or even the bigger companies when they go you know, like, we work with a lot of big publicly traded companies. They have a down quarter." }, { "speaker": "Karl", "startTime": "2315.8452", "endTime": "2331.8", "body": "They immediately cut all the marketing budgets. They might fire their CMO and their whole, you know, leadership team. They're very volatile companies, even big ones in tech. There's not much stability in this this industry. Whereas the financial services industry is very stable, very like slow and steady is their kind of mantra." }, { "speaker": "Karl", "startTime": "2331.8", "endTime": "2350.5852", "body": "They see the, the, the value in compounding over time. That's their whole business model. So what the reason I say that is like the podcast consultant has almost no churn, almost no changes in clients. It's very hard to get clients, but it's also very hard to lose them. And so that part of business has been really nice and stable versus draft." }, { "speaker": "Karl", "startTime": "2350.5852", "endTime": "2378.3352", "body": "Dev, which is every year is a whole new set of challenges. And, you know, we like in tech also, we adopt, technology much faster. And so, you know, again, contrasting our financial advisors are just now understanding that chat GBD is out there, but they can't really use it yet because there's a lot of regulations around what financial data they can share and how. And so there's a lot of limit limitations there versus our clients at draft.dev are constantly pushing the limits on what AI can do. And it's cool." }, { "speaker": "Karl", "startTime": "2378.3352", "endTime": "2398.98", "body": "I again, I can geek out on that, but it also makes it super hard to run a stable business. And so I don't know. There's goods and bads everywhere. Like, it's been fun to compare the two. I will say that, like, when you own two small companies, you have to pick your problems that you wanna tackle and which ones you can live with and that you can't fix everything." }, { "speaker": "Karl", "startTime": "2398.98", "endTime": "2411.8352", "body": "So there's times when one company's not doing great and it's just not gonna do great. And there's times when we're just not gonna get everything we want done, but we have to like prioritize relentlessly. And, you know, I I've gotten fairly good at that, I guess." }, { "speaker": "Jack", "startTime": "2412.235", "endTime": "2416.54", "body": "Yeah. Well, what do you think matters running an agency?" }, { "speaker": "Karl", "startTime": "2417.0999", "endTime": "2477.56", "body": "So the biggest the biggest thing is client acquisition. Almost all small agencies operate on the founder's personal reputation as their primary lead driver, and that is fine until the founder wants to step away a little bit and be less of a day to day active participant. We operate on an assumption that we're gonna get, and both companies do this today, about 30% of our clients through referral, about 30% through SEO and GEO or AEO, just basically LMs and search, and the other 30% is gonna be some kind of like mix of social, unattributable events, you know, just kind of random touch points here and there, rebound clients, things like that. That holds true for both of our companies, and that's something that we like actively work to maintain because if you don't have that mix, it's very easy to get to a point where, say, the founder's personal network kind of is tapped out, and then what do you do? You got nothing coming in." }, { "speaker": "Karl", "startTime": "2477.64", "endTime": "2502.8499", "body": "It would be nice if you could just count on your clients to just stick around forever. But again, like, this this industry is pretty volatile. So if gonna start a service business here, you have to be ready to, like, constantly be acquiring new clients. Even we've had a few that have gone on for three or more years with us at draft.dev, but very few who have lasted five or six versus the podcast consultant. Half of our shows are six, seven years in to work with us." }, { "speaker": "Karl", "startTime": "2502.8499", "endTime": "2508.6099", "body": "So very different personnel like profile and very different type of customer. So it's just good to know that." }, { "speaker": "Jack", "startTime": "2509.25", "endTime": "2514.7651", "body": "Yeah. Super, super interesting. Yeah. Wild. It's a very different world." }, { "speaker": "Karl", "startTime": "2515.005", "endTime": "2523.4849", "body": "It is. It is. And it's coming from a developer. Like, this has been you know, for me personally, what's been interesting has been learning the sales side of things. Like, I'm not a salesperson." }, { "speaker": "Karl", "startTime": "2523.4849", "endTime": "2549.145", "body": "I was a developer. I I this is all just like I'm making it up as I go. And now I'm the sales guy at both these agencies. And I like it, but I also recognize that it's a very different skill set. And I you know, a lot of times developers tend to undervalue sales and marketing, and this is probably a common theme for for your show and your your career arc as well, Jack, is like, you know, how much, like, tech tools think that they could just build a really great product and and and the users will come." }, { "speaker": "Karl", "startTime": "2549.145", "endTime": "2558.2202", "body": "It's just hard to do that. And so I do have a lot more appreciation for good sales and marketing people than I had, you know, maybe five, six years ago when I was not in this role." }, { "speaker": "Jack", "startTime": "2558.3801", "endTime": "2562.3", "body": "Yeah. It's hard. Hard hard jobs. Yeah. Yeah." }, { "speaker": "Jack", "startTime": "2562.3", "endTime": "2564.3", "body": "Shout outs to sales and marketing people." }, { "speaker": "Karl", "startTime": "2565.26", "endTime": "2567.58", "body": "Give them some credit. It's got a it's a tough job." }, { "speaker": "Jack", "startTime": "2567.74", "endTime": "2573.595", "body": "Yeah. Amazing. Well, karl, that was awesome. Thank you very much." }, { "speaker": "Karl", "startTime": "2573.915", "endTime": "2575.4348", "body": "Yeah. Always happy to do it." }, { "speaker": "Jack", "startTime": "2575.4348", "endTime": "2581.595", "body": "Yeah. Where can people go to learn more about what you're working on and about draft.dev?" }, { "speaker": "Karl", "startTime": "2582.3699", "endTime": "2589.5698", "body": "Yeah. So draft. Dev website is the name, so you can always check that out. We've got our latest report on there. We've got all sorts of good lead magnets." }, { "speaker": "Karl", "startTime": "2589.5698", "endTime": "2620.0344", "body": "We share a lot about how we, you know, internal processes, best practices, like, we're just constantly putting stuff out. And then karl l hues dot com is my personal website if you wanna follow, my little journey of trying to build some small businesses here. Very slow, not tech businesses for the most part. But, yeah, it's I'm always happy to share on LinkedIn too. I'm I'm pretty active out there and love hearing from developers and DevTools people who are starting their own thing or just trying to figure it all out because it's a there's a lot to navigate." }, { "speaker": "Jack", "startTime": "2621.4744", "endTime": "2623.7944", "body": "Amazing. Thanks, Carl, and thanks," }, { "speaker": "Karl", "startTime": "2623.7944", "endTime": "2623.9543", "body": "everyone," }, { "speaker": "Jack", "startTime": "2623.9543", "endTime": "2624.4343", "body": "for listening." } ] }