Subscribe
Share
Share
Embed
Testing your ideas against reality can be challenging. Not everything will go as planned. It’s about keeping an open mind, having a clear hypothesis and running multiple tests to see if you have enough directional evidence to keep going.
This is the How I Tested That Podcast, where David J Bland connects with entrepreneurs and innovators who had the courage to test their ideas with real people, in the market, with sometimes surprising results.
Join us as we explore the ups and downs of experimentation… together.
David J Bland (0:0.885)
Welcome to the podcast,
Erich Archer | CGA Creative (0:2.914)
David, thanks for having me.
David J Bland (0:4.799)
I'm so excited you're here. You you dabble in all these very interesting, you know, themes of video and AI and GPTs and I myself am trying to learn that and I thought, let's just get one of the experts who test their way through this and explain it to all of us and maybe we'll all learn. So I just really appreciate you making time to join us today.
Erich Archer | CGA Creative (0:27.136)
It's my pleasure. love talking about this stuff. It's really very fun.
David J Bland (0:31.925)
So maybe give our listeners some background about yourself and how you're going to get pulled into this world today.
Erich Archer | CGA Creative (0:37.752)
Sure, thanks. Well, I've been in the TV industry for 20 plus years. I got a film studies degree, went to New York for five years and worked in sports and music and reality television there.
and then moved out to LA to get some bigger studio experience and produced some big reality TV ad agency stuff, things like that. And then moved back to Massachusetts where I'm from to have a family and settle down and all that. And found my way into public television and I've been running a nonprofit public TV station for 12 years up on the North Shore of Massachusetts. huge markets, tiny markets, everything in between, but it's all video production.
And the last couple years have just gotten obsessed with AI. You know, it impacts my industry hugely, very disruptive in the content creation space, as you could imagine, with all the image generation and video generation tools coming out. So...
Uh, that was super interesting to me. And, um, and also just chat GPT and the other LLMs and stuff. I saw those as a real resource gain, you know, as someone in charge of a nonprofit that's always looking for resources. was like, Oh, this AI thing is a pretty helpful, uh, strategy partner at a minimum. And maybe we can figure out some automations to help us go faster and stuff. So it was just, you know, very like, how can we use this? And, and because I'm in this industry, it's always about what are the new tools.
What are the new cameras, the new trends, the new platforms? It's analog to digital and then everyone's getting 24 frame camcorders and then everyone's getting DSLRs and green screens and drones and cell phones and it's like it never stops. So this was just one more thing to learn.
David J Bland (2:31.837)
I myself had to upgrade my setup about the beginning of COVID. And I thought, oh, I mean, can't do all these remote workshops just using this crappy webcam. And I went down that rabbit hole really quickly. And I was like, whoa, when I start unplugging that USB and start plugging in actual cables, it changes the dynamic of everything. So I can relate a little bit. Actually, in a prior episode last year, we had Lex Romanon who
talks about a of testing and growth. And her background was in set design on TV shows. And I often asked her, because she used to be my peer back at Neo, we would talk about what are the parallels between how you would test a set design versus how you do product design and all that. And so maybe a similar question to you is you've been in this industry for so long. Is there anything you've learned about testing and about questioning assumptions that you pull into your work today?
Erich Archer | CGA Creative (3:28.774)
Well, you know, the way that we test in our industry is with pilots and sizzle reels and things like that. You know, we are always creating small little
test pieces to see if there's a there there to see if there's an audience there you know make a little proof of concept piece do some a b testing see if people respond and engage if they do double down make episode one if they like that make the series or half a dozen episodes but it's always sort of like trying to move iteratively otherwise you could spend a lot of time making a video or a tv show that nobody watches and so it's
you know, the parallels are there to products and everything in the same kind of way.
David J Bland (4:14.739)
Interesting. So maybe for our audience, I don't know how well versed they are in all this that you're an expert in. So maybe at a high level, explain what a sizzle reel entails and how that fits into your world as far as maybe finding creative ways to speed that up.
Erich Archer | CGA Creative (4:32.878)
Sure. A sizzle reel is just like a pitch tape. It's a few minutes long. Might be anywhere from a couple minutes to 10 minutes or so. Usually around like four-ish or something where it just is a finished piece of video that demonstrates the concept that you have for this.
episodic series or film or whatever the larger vision is, you might make this little proof of concept tape, often called a sizzle reel, which will just, you know, it's the characters, it's the storylines, it gives you a feel for it. And, you know, in the past, I've shot a lot of these things myself, edited a lot of these things myself, where we're pitching to networks or agencies like...
Oftentimes what will happen is a production company that I'm associated with will call and say we have this idea, we have this group of people we think would be great for a reality show, we want you to go film with them for a day and cut something together under five minutes long that just shows what they have going on so that we can pitch that to networks as a show. So that's what that's all about.
David J Bland (5:38.503)
Okay, okay. So maybe fast forwarding till today or maybe even a couple years ago, you have all this experience and then you're seeing this kind of wave of AI come through the industry is kind of like disrupting all industries right now. And you're working on these sizzle reels. How much of your workflow changed your thought process change now that there are all these different tools available? How are you taking advantage of that? How are you kind of testing your way through that?
Erich Archer | CGA Creative (6:4.570)
It's interesting. That was my inroad into AI was exactly that actually was I was trying to turn around creative briefs quicker using chat GPT. I was getting these creative briefs that were saying these TV networks want this certain type of content and there was insider info in these documents that I was getting and I was like I think chat GPT can process this better than I can.
Help me understand what they're looking for, help me do some research around what these terms might mean on a deeper level so that I can quickly target my idea to what they're asking for. I was like, ChatGBT can help me with that. And then I was like, well, wait a minute. What if I could bottle up that process a little bit more and rinse and repeat so that no matter the information I was given, creative, brief, good one, bad one, I could put it through this process.
and turn around the idea that's targeted faster. And in terms of the generative stuff, you know, when we would go make these sizzle reels in the past, it might cost, you know, five grand to have me go out and do that.
spend a day with a camera sweating my butt off out there chasing people around and come back and spend a couple weeks editing and you know now I'm thinking man these tools are getting pretty good I bet I don't have to do any of that soon if it's a proof of concept I bet I could just generate that video pretty quickly soon
So I'm like, man, I can use Chat-CBT to get the idea there and then soon I'll be able to generate the idea itself in video. That was the big like, whoa, I need to get into this stuff. Cause I could really create like a idea machine, you know, out of this. So that was, that was what I did.
David J Bland (8:1.291)
That's fascinating. So you have almost like this real world need in your day to day work and you're looking at GPTs and all this AI and saying, well, I do think this can help me speed up this process or help me produce a better piece of work as a result versus, know, oh, look at all these GPTs wouldn't be cool if I tried a bunch of stuff. It felt like it was rooted in a real need that you had day to day.
Erich Archer | CGA Creative (8:28.334)
Yeah, it's funny, I go into the GPT store and I look at all these things and I can't, it's not always obvious to me how they're gonna help me, even though they're like very obvious tools, I'm like, meh, it's kind of like impersonal. If you go into the custom GPTs, you make it.
You make it do whatever you want using conversational language. You I don't know how to code. So the fact that things have opened up into conversational interactions has changed everything for me. And I can be like, how would you do this? OK, cool. Do that.
Erich Archer | CGA Creative (9:6.574)
That makes it all pretty easy. It just, are you gonna put the work in? I wanna make a GPT that will allow me to input a terrible creative brief with lots of vague language and I want you to be able to turn around a cleaner version with lots of helpful clear language in it. And I want you to be to do that on demand.
David J Bland (9:29.747)
I like it. So you have this real world need, maybe walk us through some of the ups and downs of that. You know, was your first one a huge success or did you have to go back and iterate on it? Like, how did you kind of like find your way through it?
Erich Archer | CGA Creative (9:44.110)
So it was a complete failure. I tried to make a super GPT that did everything. The first step I wanted to do was just sort of extract the information and give it to me in a cleaner way. Because I was getting these documents and they were like someone had taken notes in a meeting. It was very vague and informal. And it was like, we think ABC wants this kind of stuff. was like similar to Dancing with the Stars. Like, what the hell does that mean?
So, I'm losing my train of thought a little bit.
Oh, the first custom GPT. Yeah, so I tried to make the super GPT that did all the steps, which there were like nine or 10 steps. You know, I wanted to like process the language. I wanted to do some research, some branding around the network, different things along the way, make a thumbnail. And so it failed miserably. And what I learned through that experience was...
If I turn that one GPT into 10 GPTs, where each GPT did that one step well, I could manually take the output and just paste it into the next one. And that's when it unlocked for me. was like, okay, it's not one brain, it's many minds. It's an assembly line of GPTs that do the steps of the normal workflow.
And that was like, oh, okay. So that's when it started to work. When each little GPT that did one step got reliable, it was like, nice, I can now move on. So now I've got the system of GPTs that did that and then that revelation opened up, oh, I could do this in a bigger way. And then I went on to make a whole bunch of GPTs.
David J Bland (11:35.243)
like your thought process, and I like you being open about, it didn't work as well as I thought it would the first time around. I have two custom GPTs on the OpenAI store. And at first, I thought, oh, I'm going to build this all-encompassing, amazing GPT based off my book. And I was like, that's not very lean of me to do this giant, bang release of something that has a lot of uncertainty. So the first one I released was, it was just called Assumptions GPT.
And all it did was sort of like you put an idea into it, right? And then it starts asking you a little bit about the idea and then it'll go through. We use like desirable, viable, feasible a lot, you know, in our work. And so it would come through with a desirability assumptions about like your customer and your value prop and then viability assumptions about like pricing and cost and then feasibility, which is more about like how you, how you execute on it, like your activities, resources and such. And then it would prioritize it and try to pull in evidence to say, is there any evidence?
And I thought, well, let me just break it down into one thing. And see, does anybody use it? Does anybody get value out of it? And I got a lot of positive feedback on it. And I thought, OK, well, maybe I'll create another little GPT that does experiments. And I wasn't thrilled with that at first because I thought, oh, they're disconnected. And this is before you could cross-reference GPTs, chat GPT. And I was like, well, now they have to load up a separate GPT to do the experiment design.
And then when we had the cross-referencing added, was like, oh, okay, that actually is not that bad. so, like you, it's so tempting to just create this giant thing that solves for all these problems that you see it in your head mapped out. But in reality, it's like, I de-risk this a little bit and make just a single version that does one thing really well and then maybe learn from that and use that learning to build out other parts of the workflow.
Erich Archer | CGA Creative (13:24.590)
Yeah, so a couple things. I felt like I needed a visual interface to be able to visually see things at a glance. Being a visual person and having these things, as you said, sort of scattered, was like, there's no way I'm gonna be able to remember this stuff, even if I can connect them in a thread, like using the at sign.
You got to remember the name of the GPT and if you've got 50, 60, 100 GPTs, you know, it's not sustainable to work that way. So I made a visual interface using a free software called Whimsical that is cool. It's like a infinite whiteboard where you can just put text and pictures and links. So it was like, okay, I'll just throw everything up there and text and link.
and make this little mind map thing. So I built this platform inadvertently. And what I have learned by working in this modular way is that it's an advantage, even though it's slower than fully automatic, it keeps human in the loop. You can throttle.
the process and be the human in the loop at the important decision making points, like where you have proprietary knowledge and you want to step in and be the human, you can put yourself at step three. GPT-1, GPT-2, David, GPT-4, and make sure that you're part of the process anywhere you want to be. I think even if it's just like you're taking the output of one and while you walk it to the other one, you're checking it. That's a good thing.
You're checking the output, making sure it's not wonky before you paste it into the next one. That's a good thing. And I think ultimately that's still really fast, faster than manual, and more reliable than automatic.
David J Bland (15:21.097)
Yeah, I think people want just press a button and have it do all the things for me. But then rarely are they satisfied with the output of all the things. And so it's something I don't learn every time with GPT's too is OK, at the end of this prompt, I'll ask, you know, or have it pause and say, is there anything you want to edit or add to what I just presented to you? And I think that pause, maybe it's something similar in the spirit of what you're doing.
It gives you that moment to reflect on it and say, okay, here's where I want to change. Or I think you got that part wrong. Let me get in and kind of like massage this before we get too far down the road in the process. And I'm a big believer in that as well of we do need human interaction here. It's almost like an extra team member, but you're going back and forth with this team member. shouldn't be, I just give the team member all the requirements and they go off and produce this thing. It's a little more back and forth with the GPTs.
Erich Archer | CGA Creative (16:18.306)
Yeah, and think reflection is the exact right term. I think AI is really good at reflecting on its own work and improving it, and people also. That's a really important step. will very often, if not all the time, say, okay, cool, thanks for that great answer. How would you level that up? Just like...
check it against itself and it'll come back with a better answer almost every time, you know. So that reflection is a really useful part multiple times along any workflow.
David J Bland (16:50.495)
So you have all this experience in TV. You see the pain of the day-to-day process, know, the inputs you need and trying to get the outputs that people desire. You see GPTs come on the scene and AI and it's kind of been very disruptive and everything happens so fast now, all these different platforms, all at once it seems. And you're incorporating that. It seems like in a very thoughtful way though of let me take pieces of this and try to make this process better.
Where is this leading to you now? Like, what are your thoughts on, you know, what's working and what's not and how you're testing your way through sort of like present day?
Erich Archer | CGA Creative (17:30.478)
Oh, it's an interesting kind of rapidly moving space. Being in video, I think people are getting really overwhelmed with the pace of change and the amount of new tools and keeping up with all of what does what and everything. But I think it's important to remember that at the end of the day, the deliverable is still the same.
It's still a video and a video is still comprised of smaller videos and audio and animation and the same elements that went into it before. It's just a matter of where you sourcing these assets from. You know, are you shooting them with a camera or are you generating them with mid-journey or some other tool and that tool can change. You know, it's been mid-journey for image generation for me for the past year but Runway is starting to get a little bit
better and so I'm not loyal to Mid Journey. It was working for me in 24, it might not in 25. And that's true with 11 Labs or Topaz or Runway or Mid Journey or any of them when people are like, oh my God, something uprooted this platform, something beat this other platform. It's like, all right, maybe, but like, let's just, the assembly line is still the assembly line. We have pre-production, production, post-production, we're making a video.
David J Bland (19:34.155)
So the pace of change here is amazing. mean, and somewhat terrifying. When you think about back in the day, right? I don't want say back in the day is almost like two years ago back in the day, but we were chatting even before we jumped on this episode. You used to work with just kind of like one or two programs it felt like.
Erich Archer | CGA Creative (19:41.102)
Thank
David J Bland (19:56.341)
you to produce this or you spent most of your time maybe in a few programs. Yeah, there were some other ones you pull in for little tweaks and things, but you spent the majority of your time. So how are you, you know, testing your way through like, what is the flow that works for you now? Because I can imagine people feeling very overwhelmed because they are used, they know how to steer around the curves in this one or two programs. And now you have, it feels like a new AI platform every week.
So how are you sort of navigating that or what helps you manage it and not lose your mind going through this process?
Erich Archer | CGA Creative (20:33.902)
Well, you I think it is going back to that like, okay, at the end of this, I'm trying to make a video. And so let's break it down. if I'm gonna make something fast tempo, 30 seconds long, maybe I need 20 shots. Okay, I need 20 shots because I need two to three seconds per clip and I'm gonna get rid of some along the way, but.
Okay. And those all need to be video. So I could, I have to choose my video generator. And if I'm going to go from image to video, I need to choose my image generator. But that's okay. Like that can, it's no different than choosing your camera back in the day. Like, are we going to shoot this with a, you know, a 5D Mark II or are going to shoot it with a red? Like,
You know, am I going to use mid-journey? Am going to use runway? You know, it's just like, we got to get there. got to, but you know, now you're using a whole lot more tools. You're getting up from your desk a whole lot less because you're, you know, pre-production is all chat GBT like, okay, we got this idea. Let's make the idea better. How do we tighten it with data and research and bop, bop, bop. And then we can go into Dolly and get some real easy, fast starting imagery just to get it on the same page.
Here's some compositions, you know, don't pay attention to the style. We're going to improve that. But like, let's just get a storyboard together. Dolly can do that from the conversation we had about the idea. All right, cool. Now we've got a bunch of pictures. Well, there's a bunch of tools that will turn pictures into video. Maybe we try those, you know.
and try to get those pictures to turn into two seconds of video. And if we can do that successfully 20 times and they all look similarly, we could potentially make a minute long video. And if we need music, are we gonna make it, buy it, generate it? I mean, we need music.
Erich Archer | CGA Creative (22:35.982)
So maybe I'll go try on music generator and see if it works, you know? Maybe I'll go to 11 Labs and try their voiceover and see if it holds up to a person's now. Or maybe I'll get some sound effects generated instead of going into my whoosh pack and listening to 45 freaking whooshes over and over again. You know what mean? And then I'm still gonna go into Premiere and edit the old way, except that's got some new bells and whistles in it too. So like I don't have to cut the music to time anymore. I can just drag it and it'll cut itself and stuff like that.
paying attention to what the tools I already was using are doing with AI also. know, Canva, Premiere, Photoshop, they're all baking AI in rapidly. So it's really just like, all right, I know how to make a video. I already know how to do that. I've made a million of them. So how it happens is very familiar to me. And it's just like, where are we getting stuff from? And that doesn't stress me out because it's what I've been doing anyway.
we need a camera operator and they have this camera and then we get, know.
David J Bland (23:38.973)
It sounds as if there's this great quote from Theodore Levitt, which is somewhat controversial, but it's, people don't want a quarter inch drill. They want a quarter inch hole. know, like the whole point is they need to create a hole and maybe a drills the way to get there or maybe use like something else or maybe you use a command hook or something and you don't even have a Like the idea is like I'm trying to hang something and.
There are many different tools I could choose from to get there, right? And when you were speaking, that's what was almost top of mind for me was, yes, there are all these different tools out there, but you're still looking for like fit for purpose. You're trying to say, hey, I'm trying to do this job. What are the tools available? But also being open to the idea that, hey, some of these AI tools are getting better in quality from week to week. And so maybe it's worth revisiting some of them. So I like that you're sort of open to that and you're testing your way through that. What do you think?
some of the big assumptions are, know, looking to this year and beyond and things that get you excited. What are some of the big assumptions you'd like to test? You know, like you mentioned generative AI, you mentioned some other things. Is there anything out there that just like gets you excited, but you feel, oh man, there's a lot of uncertainty with this. if I, similar to like the one giant GPT, I would need to break that down into smaller pieces. Like what gets you excited and what are some big assumptions you think that come along with that excitement?
Erich Archer | CGA Creative (25:4.187)
I haven't tried to put this into words too often, but I'm starting to feel like we're getting to a place where generating novel insights and ideas might be possible. And it's a hack up, it's like still a thing where you'd have to hack it, but I'm like trying now where it's like Google put out this thing the other day.
About how they're approaching some new technology and a phrase in there stood out to me which was idea tournament They had come up with this concept of an idea tournament and I was like what the what is that? You know, I could use AI to make ideas fight it out in a tournament Is that what they mean? Because I think I could do that and that's awesome and I want to do that because Even if it's just like it gives you the best idea
and it's not a new idea, but they've tested it. I'm thinking like, oh, I could make a council of all my favorite smart people and throw a theme in the mix and they can battle out and come up with the best idea and give that to me every morning. Like, what? And then I can go make content out of that. I don't know, stuff like that to me seems really cool. I'm trying to like, you know.
cure my wife's migraines on chat. I just feel like they're getting so smart, so impactful of data. And if you can really just find side doors and ways to like test it against itself and stuff with different personas and things, you might wind up getting some interesting results that I just kind of enjoy exploring.
David J Bland (26:47.357)
Yeah, if you find a cure for migraines, let me know. They suck. I have them from time to time as well. know, interesting this tournament idea, you know, there is some there is some value in the nature of the idea itself. Right. And I think what's really hard for me is that and my co-author Alex Osterwalder also says this, which is
Erich Archer | CGA Creative (26:50.047)
I'm going to
David J Bland (27:11.007)
Like good ideas and bad ideas often look the same early on. It's really hard to pick winners. And we see that from venture capitalists. We see that from CEOs of companies trying to pick the winner of the project, you know, or maybe even TV shows, right? Like, I know this is a winner because I'm, I can pick this and all my experiences is, is in the background here and I'm using that. It's biasing my, my selection, of course. But I do think there's something also about like, how do you want to test that idea? Right? So if you come up with a good idea,
then it becomes, well, okay, how do I go test that and break it down? And it feels as if some of these things you're building with all this GPT workflow and all that, maybe we get to a point where you have this suite of tools or different GPTs that can help you go test. I mean, I know that's something that's very passionate for me because I'm trying to help people focus on how to test things when maybe they don't have a bunch of experience.
Right? Like you have all this lived experience from from your industry and a lot of people don't have that. And so they're not even aware of what's possible. So I think there is something to be said about the idea, but also how do you go test it in a small way? And I think that might even be the harder part because it's like, how do I go test that idea? And that's not always straightforward.
Erich Archer | CGA Creative (28:33.474)
Yeah, you know, I think that's a large part of why I built the suite of tools I did because, you know, I took over this job 12 years ago as the executive director with no experience prior in management. And I came in green and I didn't know so much about management. I didn't know so many frameworks and mental models and ways to think about stuff.
and analyze stuff and I had to learn those things over the last decade plus. And my board would pay for consultants to come in and do these different frameworks like a SWOT analysis and I would learn what that was and stuff. And it's like, okay, cool. And then when I started working with custom GPTs, it was like, well, wait a minute, maybe I could make a SWOT analysis GPT.
you know, chat GBT knows what a SWOT analysis is. I don't have to explain it to it. I don't need to give it any data. It knows, right? So maybe I just tell it to do that. And how could I have it interact with people? Well, maybe I just do a Q and A where the, the button that you can make says click here. And when you do that, it starts to engage you in a Q and A to extract the SWOT information it needs to do the framework.
So it's like, all right, let's talk strengths, you know? And it takes you on this Q &A and takes the information from you and then gives you the analysis. And it's like, okay, that worked. What other frameworks are out there? Oh, there's a couple hundred. Rinse, repeat. And now they're all on my platform. All these available basic frameworks that will guide you through a Q &A to...
to show you what you didn't know, you know, and be that consultant that would have cost a couple grand to come in and do that for you. So whether you're a startup founder or like just, you a solopreneur or whatever, I think these things can be super helpful.
David J Bland (30:31.979)
Yeah, I agree. It also has a lot of consultants worried that they're about to be replaced.
Erich Archer | CGA Creative (30:35.242)
Yeah, well, you know what's interesting is like a lot of consultants are coming and saying
I have a proprietary way I do things like this marketing guy at higher CMO said to me, I have an 11 step proprietary framework for product market fit. I have it. have my way of doing it. And I said, all right, cool. Well, let's make your GPT for your framework. And we did that and we co-market that. like he's baking his subject matter expertise into a standard framework. And that I think is a super interesting layer of tools where people's proprietary
Expertise is layered on top of these basic tools.
David J Bland (31:13.897)
Yeah, I agree. And I think now I see authors playing with custom GPTs that are partnered with their book. So you launch a book and then you launch your GPT with the book. I think it also makes publishers very stressed out because publishers like to control all the IP in very specific ways. And this idea of, wait, there's a custom GPT with the book. I don't think they're quite there yet. But there's all kinds of crazy opportunities. So I just wanted to.
Thank you so much. mean, your background is so fascinating with all of your TV experience and production experience, but I love how you're breaking things down into smaller pieces and testing them and being willing to adopt new technology and testing your way through that and kind of navigating that. I love your spirit with that. And I just wanted...
to make sure that our listeners have a way to reach out to you. So for those of them that have been listening and they're like, wow, you know, I'd like to learn more about what you have and how to piece stuff together and all these different tools for making videos. What's the best way for them to reach out to you?
Erich Archer | CGA Creative (32:23.054)
Thank you for asking and the kind words there. I appreciate that. I have a portfolio website for my AI work. It's cgacreative.com. And I'm pretty active on LinkedIn also. I spell my first name kind of unusually E-R-I-C-H Archer. But I'm on there every day and always happy to connect with people. I enjoy networking around this stuff and talk and shop with anyone who's into it.
David J Bland (32:50.813)
Awesome, so if you've listened, we'll put these links on the detail page and you can check out Eric's work. Thanks so much Eric for joining us. I really learned a lot in our episode and I wish you the best.
Erich Archer | CGA Creative (33:1.282)
I really appreciate it, David. Thank you.