Subscribe
Share
Share
Embed
Explore your curiosity - Interesting people with fascinating stories.
Life consists of three things:
-The stories we tell others
-The stories others tell us
-The stories we tell ourselves.
Matt, welcome aboard to the show. Thank you so much for joining us today.
Speaker 2:Thanks for having me.
Speaker 1:So, so Matt, you're a really smart guy. So I'm gonna start with a very hard question. Will there be as much demand for engineers in five to ten years.
Speaker 2:Yes. Why?
Speaker 1:Because And present the problem. Why am I asking the question?
Speaker 2:Yeah. I I think I know the answer. I think I know why you're asking the question. Right now, with all of the technologies around, like, vibe coding and, you know, the ability for AI to just jump into your code base and start, you know, building things out. I think there's a lot of people asking that question.
Speaker 2:I've seen a lot of businesses claiming that they're offloading their engineering departments. What I'm predicting with that is actually that they are taking an immediate reduction in cost to actually meet the same productivity level, but that they're going to want to be more productive. So they're going to add engineers back once they've sort of taken that immediate pay cut, you know, loss in in costs. And so you're we're, you know, we're as a as an industry, we're going to lose butts in seats only to then put people back in those seats. But I also think that we need to pay attention to the fact that our AI agents can only do what we ask them to do.
Speaker 2:They're trained on code that's already been written. They're not innovating. So in order for products and projects to to move forward, in order for us to do new things, we still need that human brain to innovate and to think of new stuff. So we're gonna need engineers no matter what we do.
Speaker 1:That's a brilliant I think that's a brilliant take. I would one up you. I think that the demand for engineering is going to plummet, as you said, and I don't think it's going to return to prior levels. I think it's going to be stronger and I'll tell you why. What I think is going to happen is that there's going to be an emergence in the companies that today are consuming SaaS, they're going to build their own.
Speaker 1:So I think there's going be a comeback of the homebrew solutions. So instead of having one IT group that is just managing the small IT stuff, right, users, logins, that IT group that we have today is gonna triple double 10x. They're gonna be building software for the organizations. So I think we're gonna see a flourishing of engineers in companies where they've never worked before and I think that's gonna basically create a movement of engineers from your kind of platform companies, your B2B companies, into your B2C companies, right, your non platform companies. So I'm actually arguing, sure, it's going to crash, but it's going to flourish after that.
Speaker 1:So I think what you're saying is very true and I would even double down on that. Let me ask you this, how can so there's going to be a tough spot where these companies basically take a bath, right, in economic accounting terms. So they take this bath, they're really massaging their numbers. In that short term, there's gonna be suffering. What can individual engineers do to alleviate that suffering or just not be hit by it?
Speaker 2:Yeah. There's a bunch. One is just learn the tools. Right? Like don't be afraid of them.
Speaker 2:And don't necessarily listen to the curvudgeons because there's gonna be tons of people and, and you may be one of them who are like, Oh, these new tools, they're not that good. And your initial response is going to be to go to the tool and ask it to do something really hard. And it's going to fail. And you're going to say, Well, this tool is crap. But you I think the sort of metaphor for this is if somebody hands you a hammer, and it doesn't immediately build you a house.
Speaker 2:Well, yeah, the hammer can't build you a house. But if you know how a house is built, you can build a house with a hammer. And so if you know how software is built, you can build software with ChatGPT or Gemini or Cursor or Claude or any of these tools. You just have to know the nuances of that tool and how to get it to do the software building. And so and I've used a bunch of them.
Speaker 2:Each one has its own little nuances, and you have to know how to massage it and ask it the right way, but you can get it to do a lot of really cool stuff. I've gotten these tools to do things that I could never do as a software developer, but I know how to ask it
Speaker 1:to do it. So we call this prompt engineering. It's not really prompt engineering. It's something much more complicated. Let's break down the So it's worth saying that what Matt and I are gonna talk about now, we've never talked about before.
Speaker 1:The books about this topic haven't been written. So this topic is basically an unresearched science, so to speak. So everybody can take it with a grain of salt. We're probably going have some differs of opinions. But what you're about to hear now is cutting edge technology.
Speaker 1:Okay. Why do why do these agents hallucinate? So when we say a hallucinate, basically, what's happening, you ask it to do a, it does b. Why does that happen?
Speaker 2:So I don't have great answers here. Right? Because I I haven't been in the weeds into all of the, you know, technical nuances of how at the core all of these things work. You know, there is the sliding window of context where, you know, you're in the chat, you can think about like, you know, it can only remember so far back into the chat, and it's trying to, make up and and remember what was going on. Almost like, you know, the kid who fell asleep in class and the teacher calls on them and they're trying to like the the war of eighteen twelve, right, or whatever.
Speaker 2:They're coming up with an answer. Yeah. I from what I've seen just from interacting, it very much feels like the tools are always trying to give an answer that they they almost don't have the ability to say I don't have an answer. They will dig into the muck and pull something out. And so that is part of what feels like a hallucination.
Speaker 2:But they have this ability to give an answer that is close enough to the context of the conversation, even if they make it up. And to me, that's kind of what I think people are talking about with their with the hallucinations.
Speaker 1:So so so what what Matt just described is two different techniques, and he hasn't talked about the technique to to to deal with the mock, but we'll get to that in a second. He's talked about two different problems which both look like hallucinations. One problem is too much context. So basically if the chat gets too long, there's too much information and the agent kind of doesn't know what exactly it's doing. The other one that Matt is talking about is basically the absence of guardrails.
Speaker 1:So if the scope of the problem is undefinable to the level that the agent doesn't have walls to bash up against, then it has too much freedom. So one is the excess of context. The other is excess freedom within the agent's confinement of operation. So these are the two ideas that Matt is talking about. It's one out of five really advanced techniques that people are figuring out nowadays that help agents work correctly.
Speaker 1:And they're both brilliant and they're both correct. So inside of the constraints of these two problems, what are the solutions? How do we architect these agents to basically work better?
Speaker 2:So one thing that's been really interesting that I've been learning a lot about and sort of paying attention to are actually so ChatGPT, Claude, Gemini even, they are large language models. And so some folks have actually been playing with small language models. And so they will train on a very specific subset of knowledge. And then when you ask that language model about that subset, it can only give you answers inside of that subset of knowledge. And there is now a theory that you could effectively take these large language models and create one from thousands or tens of thousands of these small language models, and and like create a tree of knowledge effectively underneath of one of these large language models, so that they reach into the small language models to get the data and then, you know, mash that together.
Speaker 2:That could create less hallucination, but it would create other problems where, like, the context can't connect across the models, which is why you have the large language models. So this is a problem we have in technology and other places as well. And so, you know, it's very much feels like in technology, we get to these places where we solve problems only to create all new fun problems that we have to go and solve.
Speaker 1:So so let's break this down because you went to you went to the silver bullet. You got me down
Speaker 2:the rabbit hole here.
Speaker 1:Yeah. Well, we're this is the show. We go down the rabbit hole, we come back up. So so Matt went to the silver bullet and hit the question that he's that he's basically answering is, could we create a large language model that will work for anything? And what Matt described in more detailed terms is that if the large language model basically had a thousand very specific language models that could do very specific tasks, then all the large language model would need to do is figure out which task it's actually working on.
Speaker 1:And then it would use a smaller model to get that task done, who would come up, bring the data in, aggregate it together. So that's the kind of silver bullet. We don't yet know how effective that approach would be. So that's kind of a futuristic thing. The problem is that today in AI, future is tomorrow.
Speaker 1:It's not ten years out. So this could be around the corner and we just don't know. All our predictions of, oh, this will be an X amount of time, they're all going to shit for all of us. But there's easier things that we can do. So when you're working with Cursor, you know, and you're Unlovable and all those, there's methods to manage the agents in a way where you prevent hallucinations.
Speaker 1:So you talked about the first one about narrowing context. So how are you approaching basically the small window? Are you resetting your chat? Are you having a single source of truth and you're kind of restarting it? How do you kind of work to stay within a smaller narrower context?
Speaker 2:Yeah. So what I try to do is I try to break everything up into effectively projects. And so it's a very, almost agile approach to do it. It's like achieving a task. And so I go back to the basics of you have your end to end project, and then you want to break that up into milestones, and then you're going to break that up into tasks.
Speaker 2:And so I actually typically will start with an idea with ChatGPT. And then I will have it create something like a roadmap and outline a lot of the project itself. Then I will take it over to a cursor to break that up into the actual executable pieces. I might even take some of what I did with ChatGPT and cursor over to Bolt or Lovable for the UI piece. I'll bring that
Speaker 1:back over to cursor.
Speaker 2:Say that again?
Speaker 1:Figma is also pretty Figamamake, I don't know if it's pretty good too.
Speaker 2:I haven't used that one yet.
Speaker 1:Yeah. I don't think it's any better than Mobable. So here's the brilliance in what Matt is saying. Matt is saying basically if GPTs and at the end of the day it's all GPT, even the cursor is GPT, it's just a UX above GPTs with some smart layers in the middle. But any GPT basically will do better by working on smaller tasks.
Speaker 1:So the brilliance in what Matt is saying is he's just break it up to a whole bunch of small tasks that you know come together in a nice way at the end. So he's basically creating an architecture of a work breakdown structure. That work breakdown structure is executed by individual agents and then that basically helps to create smaller contexts. Brilliant approach. I'm doing basically a similar thing.
Speaker 1:It works very well. The other type of hallucination that is a very effective way is to put up guardrails for the agent. What is how does that work? Right? So the kind of read only approach to agents creating content, how does that work?
Speaker 2:So it depends on what you mean by guardrails. So in the like when I'm using cursor, for example, some of the guardrails that I'll use are actually in the cursor rules functionality. So they have this ability for you to write these rules files that tell cursor how you want it to act. So when I'm using TypeScript, right, I will say, you know, everything should be TypeScript strict. Every time we change functionality, we need to update tests.
Speaker 2:Every time we're gonna commit, write docs. And then when we commit, we're gonna run winters and we're gonna do this and this and that. And so it that actually tells it the behavior that we're looking for every single time we take an action.
Speaker 1:Why does that so so you're kind of outlining the how, but you're not outlining the why. Why does adding these I'll get you there.
Speaker 2:I'm with you.
Speaker 1:Why are these you know where I'm going. Yeah. Why do these guardrails force the agent to better results?
Speaker 2:Yeah. So the reason the reason why I have all of these things that I have added to cursor is that cursor will try to get to a result as quickly as possible. And oftentimes, the way that it gets there is not an ideal path for what I'm looking for from the result. So in coding, it will get there as easy as possible, but the quality of the code is not great. It won't have the documentation I need to be able to bring another developer in, or I won't have tests to prove that it's right.
Speaker 2:And so I'm asking it to have very specific behavior so that I have the quality of the product I want, so that it gets committed correctly, all of that, which requires those guardrails, and that behavior. Because yes, I can say go and it will build something, but it may not look and feel the way that I want it to look and feel.
Speaker 1:That's why lovable kinda sucks. There's a third thing you said. The audience might have missed it. You said strict. That's a that's a different type of guardrail.
Speaker 1:You talked about process guardrails. Strict is actually a wall that GPT has to overcome, which creates higher quality. Explain because that type of guardrail can be used in many other different ways. Explain that guardrail to us.
Speaker 2:Yeah. So in in TypeScript specifically, there's a there's typing of all of the variables. So this is the thing that goes back to the the core way that computers work. You know, you have your ones and zeros that go down to like how the electrons move through the computer and all of that. And then as you start to abstract layers from there, you start to get to numbers and strings and arrays and all of that.
Speaker 2:Well, once we got abstracted far enough away, we started to actually get to where languages didn't care what the variables were. And then that got to where languages were very loosey goosey, and you could start to like, take a paragraph of text and pass it into a variable that was a number before. And then when you start to display things, they get all goofed up. And so JavaScript is one of those languages that was very loose. When you go to write code, like we were talking about, and a lot of the, Bolt, TypeScript, replet or sorry, Bolt, lovable replet, they'll rip out code in Next.
Speaker 2:Js and JavaScript, and they'll do it loosely typed. They won't do it in in strict, because it's very simple, but then you end up where you have text showing up where integers should be and all of that and things don't start quite work right. And if you're not very technical, you end up having to debug some very technical problems. So if you can tell it to be strict, it will always make an integer an integer, an array will be an array, text will be text from the database all the way through to the UI and back. And so that forces the AI to spend a little bit more time on every single step.
Speaker 2:It takes a lot longer, but the quality will be there and you'll spend a lot less time debugging and with a lot fewer pervasive issues that end up, you know, having you rewrite all of your code.
Speaker 1:Brilliant. This is brilliant. And what's really interesting about this example that Matt gave, and if anybody's wondering, Matt is absolutely brilliant. What's brilliant about this example is that it extrapolates to many other different types of guardrails that can be used. For example, if you put a test against an API and you put an API against a database, a read only database acts as a guardrail for an API.
Speaker 1:The UX working against the API, the API works as a guardrail for the UI and the test acts as each one respectively, one, three or four or five test agents, each one acts as a guardrail to test against the creator. So, so what's happening in the industry is we're understanding that there's actually AI agent architecture that's becoming a science. This is this is as I said, this is cutting edge stuff. There's no book about this yet. And if there is, I I haven't heard about it.
Speaker 1:We're not going to give away all Matt's secrets today because we want you to come to him and learn from him. Matt, you started a company not so long ago. You started on this very difficult journey of becoming an entrepreneur. As a serial entrepreneur with multiple exits and manage a consulting business, a very successful $110,000,000 ARR, I feel for you deeply, and I wish you the utmost success. But let me we'll go into that a second.
Speaker 1:But before that, tell us the name of your business so people can find you.
Speaker 2:Yeah. It's a, you know, very creative name here. It's my last name LLC, so Bernier LLC. And it's the the website is mbernier.com. So trying to make it easy to find me.
Speaker 2:You know, we're in Denver and, you know, working with, you know, small to medium sized companies that may be deep in tech but may not be, you know, trying to help folks that are just getting into AI things or that, you know, need to level up their tech, and then also consulting companies that have, product management, situations that are going on. Maybe they've got the market fit, but they can't quite figure out how to get the product side moving forward. Like people aren't onboarding well and they're dropping off, or they've got a team and they need to grow it very rapidly. How do we get those processes fixed internally? You know?
Speaker 2:So I'm working across the whole product spectrum and the tech spectrum translating, you know, helping folks out. I like working with with good people doing good things and, you know, having fun throughout the day. And it's been it's been a heck of an adventure, you know, meeting a couple 100 people a month sometimes and, you know, just learning about their business and, you know, get to play and, you know, play at with other people in their sandboxes, which is fun.
Speaker 1:Beautiful. So, so you're coming into these organizations which are asking themselves, should we vibe code and how do we do it right? And there's a lot of fear, right? So the senior executives, the C suite that's not the CTO, they kind of have a lot of questions. They kind of know that it's happening and it's going to be a thing, but they're afraid.
Speaker 1:The chief technology officer is a whole different beast that can go either way. You're coming into this environment of extreme uncertainty, but you've kind of been invited to talk to a certain degree. What do you do? How do you even approach this?
Speaker 2:You know, I think it depends on the persona of the person that you're talking to. You know? And, you know, if they're an old, you can always just be like, hey, you know, at some point you went from pen and paper to spreadsheets. Right? Like, this is this this could be something like that.
Speaker 2:But I I think that one of the things is just getting people to try it out and to see that it's not this big scary thing. Right? You know, I I led a meetup where I sat down with a bunch of marketers who were not super technical. And we in forty minutes, we built a landing page with a form that put the contact information into a spreadsheet. Right?
Speaker 2:Not a big deal. But these folks didn't have to deal with Salesforce, they didn't have to deal with a designer, they could put whatever copy they wanted on the page, They could change the colors very quickly, change the buttons. They didn't have to have a developer. And I showed them how to go and use GitHub and Vercel. And they were like, wait, I can like, in thirty minutes, I can go and just make this thing show up on the Internet.
Speaker 2:Right? And then I can show somebody. And, It's crazy because I mean, you're talking to somebody who twenty years ago would be sitting here cutting designs from Adobe Photoshop. And I'd have a designer walk by my desk and lean over and go, that's two pixels off. Right?
Speaker 2:And I'd be just like, you know, like, you know, or that color's not right. I didn't pick the color correctly or whatever. And I can go to Bolt and, you know, after talking to ChatGPT for five minutes and rip out a yoga site or whatever, you know, as an example, It's it's insane. It's
Speaker 1:And so much even fun to the young designers today, don't think they know what cutting a design means. Like, that's not a thing people, the young even do nowadays. I did that. Like, I had the the big images that we cut like, shit. That's that's that's hardcore.
Speaker 1:I remember having to manually do rounded corners. Right? And line that up with with three to do that in Photoshop.
Speaker 2:Yeah. Three divs in in HTML. And now you can do a CSS. And I'm like, y'all don't know what this is God
Speaker 1:bless HTML5. Mean really, know, the USUI guys today, they just don't know they're very lucky to have missed out on what we lived through, but yeah, no, that's fair enough. Matt, this has been so delightful. There's only one question that we ask every single guest. It is a difficult question.
Speaker 1:If you had to go back to the hardest points of your life, be it yesterday or ten years ago, what advice would you give to yourself?
Speaker 2:I would give myself the advice that I learned in some very intense therapy that I did last summer. And it would be advice that I would not take, but I would hear and that would rattle around in my brain until it finally clicked, which is you are not your thoughts or your feelings. You are only the reactions that other people see to your thoughts and feelings.
Speaker 1:Matt, that is absolutely brilliant. I think every single body in the audience needs to hear that, myself included.
Speaker 2:It's something that I think about every single day. It's something I work on every single day. It's something that is hard to say as it is here. And I mean, I've I've got a tween daughter and just like a I've told her, but, like, even just handling, like, not wanting to help her through every single thing in her life, I have to sit there and remind myself that I just have to show her how I'm reacting to her reacting to the world. Right?
Speaker 2:And so, like, it's it's the impossible task, but it is the thing that has kept me from just, like, bouncing off the planet, if that makes sense.
Speaker 1:To me, makes perfect sense. There's two sentences that an old boss of mine said. One is perception trumps reality. That's half of what you said. The other half of what you said is leadership by example.
Speaker 1:Matt, thank you so much for coming on the show today. This was absolutely brilliant. I feel very fortunate to have you with us.
Speaker 2:Yeah, anytime. Ari, it was great talking to you. This was fun.