{ "version": "1.0.0", "segments": [ { "speaker": "Joff Thyer", "startTime": "0.79999995", "endTime": "23.045", "body": "Hello, and welcome to another episode of AI Security Ops with Black Hills Information Security. We have our illustrious panel of guests as usual. We have Brian Furman, Derek Banks, and Bronwyn Aker with us. And we have a very special guest with us today. Introducing Daniel Meisler." }, { "speaker": "Joff Thyer", "startTime": "23.205", "endTime": "24.325", "body": "Welcome, Daniel." }, { "speaker": "Daniel Miessler", "startTime": "24.645", "endTime": "26.005001", "body": "Yeah. Thanks for having me." }, { "speaker": "Joff Thyer", "startTime": "26.725", "endTime": "92.74", "body": "As usual, this podcast is brought to you by Black Hills Information Security. If you are interested in getting an AI security assessment for any of your locally deployed models or implementations, please feel free to reach out to us. Today, however, we are going to be speaking with Daniel and asking him about his interest in AI and all things AI that he has been doing in recent history and many of you out there, you listeners and followers of, artificial intelligence and particularly artificial intelligence and the cyber security field have probably encountered Daniel because he's been very active in the field and is pretty widely published and I know Daniel does a lot of presentations as well. So without further ado, there's a couple things we'd like to ask you Daniel, but I think first of all, you know, maybe a little bit of foundational story and little bit of background and, you know, how how did you, Daniel, become interested in artificial intelligence and make the pivot into this this area?" }, { "speaker": "Daniel Miessler", "startTime": "94.034996", "endTime": "113.83", "body": "Yeah. I I would say I've been interested in, like, public tech for I don't know. I I guess, a long time, for forever. I can't really remember when that started. But in, like, 2014 or something, I started thinking about, like, where was the future of tech." }, { "speaker": "Daniel Miessler", "startTime": "113.83", "endTime": "129.405", "body": "And at the time, I was thinking, in terms of IoT and infer artificial intelligence or Mhmm. I think I was calling it synthetic intelligence at the time because it's not really artificial. It's it's just different. Yeah. Yeah." }, { "speaker": "Daniel Miessler", "startTime": "129.405", "endTime": "162.265", "body": "You kind of like, I don't wanna be, like, specious against things that aren't human, you know, if if it's gonna actually have intelligence, which I think it does, although it doesn't have consciousness. And then in 2016, I wrote, a short little kind of crappy book. It had good ideas, but the book itself was kind of bad. But you could read it in, like, forty five minutes, and I made a whole bunch of predictions there. And then in 2018, I ended up going to, Apple, to work in a security department that was focused on ML." }, { "speaker": "Daniel Miessler", "startTime": "162.265", "endTime": "182.905", "body": "So it was machine learning AI stuff for applying to security. And in 2022, I went independent with unsupervised learning, and I've been going crazy with it ever since then. Because a few months right after that is when JaiGPT came out." }, { "speaker": "Joff Thyer", "startTime": "182.905", "endTime": "184.345", "body": "Yeah. Right. Right." }, { "speaker": "Daniel Miessler", "startTime": "184.345", "endTime": "185.865", "body": "So everything went crazy." }, { "speaker": "Joff Thyer", "startTime": "186.10501", "endTime": "212.985", "body": "Yeah. I mean, so we've, you know, from our perspective, been been following the field of AI and LLMs. And in fact, just a little bit of background here. You know, Brian here has actually got a PhD in data science, and Derek above me has a nice master's degree in data science. And we're we've all been interested very interested in artificial intelligence as well as cybersecurity and the the intersection thereof." }, { "speaker": "Joff Thyer", "startTime": "213.385", "endTime": "241.115", "body": "And I think we're living in really, really interesting times and reading kind of your perspective. I I I'll tell you what. I'll give the others a time to chime in here in just a minute. But what really interested me in what you started publishing is that you started very very early on pushing the human element and making sure that it was human centric. Yeah." }, { "speaker": "Joff Thyer", "startTime": "241.115", "endTime": "304.84003", "body": "And I love the way that you approached that because I think a lot of us as we saw, you know, the LLMs really start to take hold 2122 that period and and start to grow on an encroach in knowledge workers field, that human element was like the big fear. Think that that's been persistent theme and and going on. So so and and that's when I personally and I think every the everybody in the panel here also encountered fabric, which I I love the way that oh, for me, fabric became just this really beautiful resource for prompt engineering, and I I just I just love the the way that you put that out there with all the templates and and let that sort of organically grow, so I will segue into that, but I'm I'm gonna say, if you could talk a little bit about about fabric and the inspiration and that human centric aspect that would be really great." }, { "speaker": "Daniel Miessler", "startTime": "305.56", "endTime": "341.32498", "body": "Yeah, that's cool that you got Brian there, I'm curious what Brian thinks about the transition from like, the the original ML work to whatever this is that we have now and, like, what your position is on that. But, yeah, I would say fabric is very much I appreciate your comments about the human thing. To me, like, we're a planet of humans and we have human problems. Mhmm. I've always thought that, like, the liberal arts are kind of the point, and science and tech is, like, the backdrop that, like, accelerates that." }, { "speaker": "Daniel Miessler", "startTime": "341.32498", "endTime": "372.975", "body": "Or I I feel like that's how it should work, is that science and tech should lift what we're trying to do on the liberal arts side, which is, you know, being creative, interacting, having conversations, you know, being human. So I view all tech as and and I try to not even think of AI as special. I just it is special as an instance, but to me, it's just tech. And so, therefore, it's a tool. Therefore, it should be used to magnify something that matters." }, { "speaker": "Daniel Miessler", "startTime": "372.975", "endTime": "384.51", "body": "What matters? Human things. Right? So fabric, I put that together basically exactly oriented in that direction. The patterns are named according to problems." }, { "speaker": "Daniel Miessler", "startTime": "384.75", "endTime": "390.11", "body": "Mhmm. What problems? Problems we have all the time. I need a summary of this. Give me some ideas." }, { "speaker": "Daniel Miessler", "startTime": "391.15", "endTime": "402.015", "body": "You know, help me write this report or whatever. So, again, it's it's kind of sending the tech to the background and bringing the problems forward, the everyday things that humans need to do." }, { "speaker": "Joff Thyer", "startTime": "403.215", "endTime": "455.605", "body": "Yeah. I I really, really resonated with that approach personally, and it it I think for some of us, fabric became kind of our introduction into the idea that the patterns were very centric and very important to get LLMs to do reasonable things for us, I mean, intelligible things for us. But for me, they were also kind of a bit of an accelerant into understanding that it takes a lot more detail on the front end of of prompting these LLMs to get the simulated digital reasoning to do something much more productive for you. And I found that to be, really useful. Anybody else got a question" }, { "speaker": "Daniel Miessler", "startTime": "455.605", "endTime": "472.63998", "body": "around that? I a quick comment on that real quick. Yeah. The further back you go towards, like, 2023, and I would say, you know, before that, obviously, but the more the the format of the prompt mattered. Right?" }, { "speaker": "Daniel Miessler", "startTime": "472.63998", "endTime": "501.1", "body": "There there was a whole bunch of stuff where, like, oh, XML is way better or, you should do it in markdown or whatever. I did it in markdown because it's human readable. So, I continue to believe, and and this is the structure around the whole fabric thing, the most most important skill is actually a clear explanation of intent. It is, like, the most important thing. So if you think of, like the the way I break everything down is, like, thought and ideas are core." }, { "speaker": "Daniel Miessler", "startTime": "501.5", "endTime": "516.365", "body": "The central to human experience is number one. For to me, kind of, like, coming up from that is, like, maybe text. I love text. This is why I'm, like, you know, religious about neovim. I feel like the manipulation of text is like magic." }, { "speaker": "Daniel Miessler", "startTime": "516.365", "endTime": "530.62", "body": "And I I've always felt that way. And then I'm like, AI happens and I'm like, wait a minute. Ideas and text, what happens if you combine them? It turns into a prompt. Yeah." }, { "speaker": "Daniel Miessler", "startTime": "530.78", "endTime": "549.555", "body": "So everyone's like, prompt engineering? Nope. Idea engineering, text engineering, intent engineering. So as things move through 23, 24, 25, the format of the prompt actually doesn't matter as much because the models are better at just, like, taking whatever. They're just extracting the intent out of it." }, { "speaker": "Daniel Miessler", "startTime": "549.555", "endTime": "566.77496", "body": "It doesn't really matter what the format is as much. I I I think there are still some nuances there. But what I try to tell people is have your thought super clear. This is why I love writing so much, because writing is thinking. Yeah." }, { "speaker": "Daniel Miessler", "startTime": "566.77496", "endTime": "586.27997", "body": "Right? You you know, the bet the clearer your thoughts are, the easier it is to write them down. So if you practice writing, you're practicing thinking. So I see that and AI and prompting as all being linked together. So if you're gonna build some giant application, I believe the future of doing that is like Minority Report." }, { "speaker": "Daniel Miessler", "startTime": "586.27997", "endTime": "612.08997", "body": "You have this giant wall screen. What are you basically doing? You're basically being interviewed by an AI, which is getting you to describe in excruciating detail how you want this thing to work and why and what problem you're trying to solve. And the better you can articulate that, the better the app's gonna be. That's why the vibe coding is such a problem, is because people don't know exactly what they're trying to do." }, { "speaker": "Daniel Miessler", "startTime": "612.25", "endTime": "615.61", "body": "And they're just like, nah, do something. And like, that's when they They're fall of" }, { "speaker": "Joff Thyer", "startTime": "615.61", "endTime": "617.85", "body": "throwing spaghetti at the wall and seeing what" }, { "speaker": "Daniel Miessler", "startTime": "617.85", "endTime": "618.00995", "body": "Yeah." }, { "speaker": "Derek Banks", "startTime": "618.00995", "endTime": "635.59", "body": "That's exactly what I just did. So I I usually start each project now with fabric and an improved prompt that I kind of made from the improved prompt that exists there. Like, I modified it a bit. And then I usually take an iteration or two to refine that prompt even further. And then here here recently, I did exactly what you're describing." }, { "speaker": "Derek Banks", "startTime": "635.91003", "endTime": "659.325", "body": "I played software engineer, like senior software engineer and made Claude my my coder and was very specific in what I wanted. I want you to store this in SQLite. I want the tables to like this. I want like I and so I ended up with a a database driven cyber threat intelligence tracker that I ended up putting on GitHub because I was so impressed with it. I was like, man, this worked out pretty well." }, { "speaker": "Derek Banks", "startTime": "659.325", "endTime": "660.52496", "body": "Right? I mean, it's a proof of concept." }, { "speaker": "Bronwen Aker", "startTime": "660.52496", "endTime": "661.245", "body": "That's very cool." }, { "speaker": "Derek Banks", "startTime": "661.49", "endTime": "668.69", "body": "And so that's think we're like, I've been following what you've been doing, and I kind of, like, refined that process from, like, what you just described." }, { "speaker": "Joff Thyer", "startTime": "669.00995", "endTime": "669.49", "body": "Yeah. I I" }, { "speaker": "Daniel Miessler", "startTime": "669.49", "endTime": "676.05", "body": "did And then there's a I'll just insert real quick. There's people are calling it spec coding instead of vibe coding." }, { "speaker": "Derek Banks", "startTime": "676.325", "endTime": "687.365", "body": "Spec coding? Okay. Yeah. Yeah. I mean, like I said, I mean, it was like if you read my prompt, in fact, we did a workshop and I put the prompt we put the prompt in there just to just to, like, you know, to to describe to people this is how I did it." }, { "speaker": "Derek Banks", "startTime": "687.525", "endTime": "694.49", "body": "And the prompt is very specific and very this is exactly what I want. And Claude, I mean, nailed it out of the gate." }, { "speaker": "Daniel Miessler", "startTime": "694.88995", "endTime": "712.70496", "body": "Yeah. Imagine like a super senior engineer who already knows how to do every single step themselves, which I would say I'm like a I'm like a 60% engineer. I'm like not like a true dev engineer. I never have been. My ideas have always been strong, but I've always been a little bit crap at the coding of them." }, { "speaker": "Derek Banks", "startTime": "712.785", "endTime": "714.065", "body": "I feel your pain." }, { "speaker": "Daniel Miessler", "startTime": "714.065", "endTime": "723.06", "body": "But but but I can at least pick my tech. I could pick pick my spec. I could pick my I could write a PRD that says, I want it done exactly like this." }, { "speaker": "Derek Banks", "startTime": "723.06", "endTime": "723.54", "body": "Yeah." }, { "speaker": "Daniel Miessler", "startTime": "723.69995", "endTime": "735.375", "body": "And when you see really advanced developers use this, that's exactly what they do. They're they literally Yeah. Do what they would do if they spent the time. And then now the the, Cloud Code is just writing the implementation." }, { "speaker": "Joff Thyer", "startTime": "735.615", "endTime": "761.395", "body": "Yeah. Yes. I I think it's a really, really good topic because it comes down to the quality that you're putting up front to the the development of that spec. And I have found very similar experiences. The better I do at describing the way I want that thing architected, the tool just does a much much higher quality result on on on the back end." }, { "speaker": "Joff Thyer", "startTime": "761.395", "endTime": "771.395", "body": "I just wanna make one comment before I hand it over because I know Bramman has a question. I'm pretty sure Brian does too. But the term prompt engineering, I really don't like it. Would you comment on that, Daniel?" }, { "speaker": "Daniel Miessler", "startTime": "772.27", "endTime": "804.43", "body": "Yeah. I I don't like the reason I don't like it either is because it's focusing too much on kind of prompt engineering from 2023, which is, hey, make sure you have the little trick in there, which is, like, if you get this wrong, we'll lose $2. Like, there was a whole bunch of tricks that actually worked, but those are artifacts of, like, a problem with the model. Right? And as those start coming out, those tricks start to actually be pollution." }, { "speaker": "Daniel Miessler", "startTime": "804.67", "endTime": "819.215", "body": "But even more so, the bigger problem is that they're getting away from the core thing. It's teaching people that there are tricks when actually you shouldn't be thinking tricks. You should be thinking clarity of thought and clarity of communication of intent." }, { "speaker": "Joff Thyer", "startTime": "820.495", "endTime": "824.175", "body": "Perfect. I love that. Brahmin, I believe you have a question." }, { "speaker": "Bronwen Aker", "startTime": "824.54004", "endTime": "872.885", "body": "Well, actually, I was gonna chime in basically building on what Daniel just said. My experience has been, and my belief is that a lot of people right now are using AI badly, because I think that it has the opportunity to free us from the burden of having to memorize everything, all of this information, all of this data. And now we have the ability to focus on the critical thinking on figuring out what are the right questions that I need to ask. And I'm wonderfully blessed. I have a roommate who does not come from a technology background, and so he's had the opportunity to ride along this whole AI thing as I've been diving deeper and deeper." }, { "speaker": "Bronwen Aker", "startTime": "873.205", "endTime": "920.605", "body": "And right now, his tool of of preference is perplexity because he's able to use it. He's a retired veteran, and he's wrangling with the VA. And so figuring out the morass of regulations to get benefits, AI has been so valuable, and he's using it right. I mean, I remember years ago reading a science fiction novel where new technology came in and people were able to get these these cybernetic implants. So now all of a sudden matrix like everyone had the ability to access the data, access the information." }, { "speaker": "Bronwen Aker", "startTime": "921.00494", "endTime": "939.04", "body": "And now on this large scale, educators were able to shift their emphasis. And I think that's where we need to put more emphasis, is on the critical thinking on how to use it, and I think that's where we're gonna see a lot of positive returns." }, { "speaker": "Derek Banks", "startTime": "939.04", "endTime": "941.84", "body": "That emphasis on the other syllable. Right?" }, { "speaker": "Joff Thyer", "startTime": "941.92", "endTime": "1008.985", "body": "Yeah. So so let's so Derek and I so I wanna roll back a couple of years. Derek and I developed a class on artificial intelligence fundamentals, and and we're teaching it for anti siphon training, is a a training company that Black Hills started up a few years back. And Derek and I came at it from a very much fundamentals data science machine learning perspective, and we did a whole lot of work in this space of, you know, basic classification models, basic neural networks, the whole a whole lot of work that was short of natural language processing and short of the the frontier models that we're seeing out here today because we felt very strongly at the time that those in the cybersecurity industry needed to understand more about the fundamentals that went into the technology before they could really start running with with natural language processing. And now we've got Brian on board as well, who's working with us on course development material." }, { "speaker": "Joff Thyer", "startTime": "1010.58496", "endTime": "1069.915", "body": "So, you know, we we're we're real eggheads on this and we come at it from that from that perspective, but I think as we went through our maturity cycle with the the larger and larger models and realize just how capable the simulated digital reasoning, which is kind of the term I like to use for it, has become we became converts over to this sort of idea that you can nearly make these large language models do any task you want is just all in the the text construction that you put in on the front end. So I wonder what you think about that idea first of all about teaching the fundamentals and then moving forward from there And I think, Daniel, you also might have a question or two for Brian and Derek who who have have more formal education here on the machine learning side. So make a comment in that area if you would." }, { "speaker": "Daniel Miessler", "startTime": "1069.915", "endTime": "1086.8201", "body": "Yeah. Yeah. So I've I've got a couple of things that I wish, I had learned more when I was a kid. One is I wish I spoke math. I saw I saw, somebody a couple years ago, they were streaming." }, { "speaker": "Daniel Miessler", "startTime": "1088.005", "endTime": "1093.445", "body": "What's he's actually famous in security as well. Damn, what's that guy's name?" }, { "speaker": "Derek Banks", "startTime": "1094.005", "endTime": "1095.605", "body": "Schnire? Schnire?" }, { "speaker": "Daniel Miessler", "startTime": "1095.605", "endTime": "1103.205", "body": "No, no, no. He's a younger guy. He's like, kinda crazy but super smart. And he like hacked something. I I forget." }, { "speaker": "Daniel Miessler", "startTime": "1103.205", "endTime": "1115.82", "body": "But anyway, he's like, hey, I'm gonna write an LLM from scratch. It's not it was not Goparthy. It's another another guy. But I'm gonna write a, LLM from scratch. And he starts speaking to the LLM." }, { "speaker": "Daniel Miessler", "startTime": "1116.695", "endTime": "1136.77", "body": "Well, he wasn't, prompting. I forget what he was doing. But in the course of conversation where he was saying English sentences, he was referring to math ideas. He was just referring casually to really advanced math. And I realized it was just part of his vocabulary, and I was, like, horribly jealous." }, { "speaker": "Daniel Miessler", "startTime": "1137.65", "endTime": "1167.47", "body": "So I've been trying to learn a lot more math. So, I forgot a piece of my sort of, kind of origin here was, before I went to Apple, because I was joining an ML team full of PhDs, I'm like and and I'm bad at math. So so I went and did Andrew Ng's course, start to finish. It was like twenty twenty little pieces, and it was heavy, heavy math. It turns out the math wasn't as intimidating because the matrix multiplication is not that bad if" }, { "speaker": "Brian Fehrman", "startTime": "1167.47", "endTime": "1167.7899", "body": "you Right." }, { "speaker": "Daniel Miessler", "startTime": "1167.7899", "endTime": "1169.07", "body": "It's if you break it open." }, { "speaker": "Joff Thyer", "startTime": "1169.07", "endTime": "1169.6299", "body": "Yeah. Right." }, { "speaker": "Daniel Miessler", "startTime": "1169.6299", "endTime": "1190.74", "body": "Yeah. I saw a thing saying it was actually addition instead of multiplication, but I don't think that really matters. But but anyway, I I I think it is important to go all the way back to math. Max Tegmark had a book called The Mathematical Universe, where he has a premise that just everything is math underneath. Yeah." }, { "speaker": "Daniel Miessler", "startTime": "1190.74", "endTime": "1215.085", "body": "And I found that really attractive as well. But currently reading a book called The Mathematica or something like that. But I just I want to understand the world at this deep deep fundamental level. I feel like it's math under there. And then, to me, machine learning and AI, I find it interesting because it's figuring out the nature of the world." }, { "speaker": "Daniel Miessler", "startTime": "1215.085", "endTime": "1231.435", "body": "How is it doing that? Like, separate question of how are humans doing that, we're approximating that in some sort of way. But the whole idea of climbing the hill, like reducing the cost function, I am enamored with that. And I do think people should understand their core concept." }, { "speaker": "Derek Banks", "startTime": "1231.435", "endTime": "1234.555", "body": "So now you're speaking math, because you just said cost function." }, { "speaker": "Joff Thyer", "startTime": "1234.555", "endTime": "1235.995", "body": "Well, it's funny. It's" }, { "speaker": "Daniel Miessler", "startTime": "1235.995", "endTime": "1236.395", "body": "a But" }, { "speaker": "Derek Banks", "startTime": "1236.395", "endTime": "1241.275", "body": "I I feel like you said math enough times where, doctor Ferryman needs to, like, weigh in." }, { "speaker": "Joff Thyer", "startTime": "1241.275", "endTime": "1243.115", "body": "Well, I yeah. And I want I want Brian" }, { "speaker": "Daniel Miessler", "startTime": "1243.115", "endTime": "1244.155", "body": "to jump on this." }, { "speaker": "Joff Thyer", "startTime": "1244.28", "endTime": "1275.09", "body": "But but Brian and I Brian and I are very academic on this side of the the field of data science. I didn't go out and get the degree because I'm a lot older than Brian. And I I or not doesn't mean I couldn't go out and get another degree. But, you know, my interest comes from the fact that I've got a bachelor's degree in mathematics. So and then it was a long time ago, and I've got a post post graduate degree in in computer science as well." }, { "speaker": "Joff Thyer", "startTime": "1275.09", "endTime": "1323.335", "body": "But so I I came at it from I came at it from the other side, if you like, that I did have that deep mathematical background, and I find myself wanting to refresh that background, especially the deep linear algebra side of of mathematics because I think it is kind of fundamentally important. I spent a lot of time understanding, refamiliarizing myself and understanding tokenization and and the vectors involved in in the neural networks, so I could see what's going on there. So but Brian's much got a much fresher perspective because he's just actually finished the what is invariably a political process of navigating a PhD. So, Brian, questions and comments and thoughts from you, sir?" }, { "speaker": "Brian Fehrman", "startTime": "1324.02", "endTime": "1358.23", "body": "One of the things that you had mentioned a couple times that that really, I think, resonates with me and kind of what really sparked my interest in artificial intelligence originally is this idea of trying to kind of trying to get computers and and other systems to emulate, the way that we view the world, interact with the world, the way that we reason. And, and not just us, as humans, because that's another thing you mentioned that I really love too. It's not limited to just people. Right? And that's that's one of the problems with, like, with some of the definitions I have for AI is they usually talk about just human reasoning." }, { "speaker": "Brian Fehrman", "startTime": "1358.23", "endTime": "1382.715", "body": "It's like, well, it doesn't have to do be just people. I mean, obviously, I don't know how a dog or a cat actually thinks because I'm not a dog or cat, but we can look at things like swarm theory, for instance, with, you know, birds and and other these Yes. Large groups where, you know, where none of them know what every other one is doing, but they know what the ones around them are doing. And based upon that behavior, they can make their decision. So trying to emulate these things we see in nature." }, { "speaker": "Brian Fehrman", "startTime": "1383.26", "endTime": "1422.84", "body": "But going back to the human aspect of it, you know, we have these things that are so seem so easy for us to do. Right? It's so easy for us to to think through things, to look around, to, you know, see things and and, kinda pull out conclusions from that very quickly. But getting computers to do that in a effective and repeatable way is certainly, it's much more challenging, and it's something that people have been working on for a very long time now. I mean, neural networks have been around since the sixties, and now we're, you know, within the last couple years, especially, really leveraging these, to try to, kind of emulate some of this, some of this reasoning process." }, { "speaker": "Brian Fehrman", "startTime": "1422.84", "endTime": "1434.7051", "body": "And so that's just some of the things that I think are just really fascinating about the the the field in in general is just trying to get that, you know, just trying to make it do what we can do, essentially." }, { "speaker": "Daniel Miessler", "startTime": "1434.7051", "endTime": "1447.16", "body": "Yeah. Yeah. Love what you're saying there. The way I see that is like, there are universal things. And I'm curious how humans, humans are just an instance of a thing dealing with a universal thing." }, { "speaker": "Daniel Miessler", "startTime": "1447.16", "endTime": "1459.645", "body": "So the the thing is, challenge. Okay? Capabilities, applying capabilities to challenge. And like, it's an algorithm. Right?" }, { "speaker": "Daniel Miessler", "startTime": "1459.645", "endTime": "1483.26", "body": "It's like and a dog does that. I guess the other thing would be like goals. So it's like you're trying to produce some, pursue something and you're moving through this environment. And each individual small thing could be like a challenge or a problem or whatever. But, yeah, I love thinking about in some distant universe, there are certain things that are universal." }, { "speaker": "Daniel Miessler", "startTime": "1483.5", "endTime": "1503.7999", "body": "Right? It's like there is a substrate that they are living inside of. If they're at some level of consciousness, maybe they have goals. And you have to pursue goals with the capabilities that you have. These things these things seem fairly universal, and I love the fact that they are universal." }, { "speaker": "Daniel Miessler", "startTime": "1503.96", "endTime": "1517.1249", "body": "And we could try to get to the kernel of it and then just look at instances of, okay. This is how this particular this is how transformers seem to handle it. This is how humans seem to handle it. Dogs, cats, dolphins, whatever." }, { "speaker": "Bronwen Aker", "startTime": "1517.845", "endTime": "1526.8049", "body": "One of the things I don't know if the other guys are aware of is that, I participated in that first class that you taught on fabric." }, { "speaker": "Daniel Miessler", "startTime": "1526.8049", "endTime": "1527.205", "body": "Oh, cool." }, { "speaker": "Bronwen Aker", "startTime": "1527.42", "endTime": "1531.18", "body": "Remember? I mean, that seems like ancient history now." }, { "speaker": "Daniel Miessler", "startTime": "1531.18", "endTime": "1532.86", "body": "Yeah. The augmented class. Yeah." }, { "speaker": "Bronwen Aker", "startTime": "1532.86", "endTime": "1569.49", "body": "Yes. And, and I was just floored by you were able to bring order out of chaos. Was that was one of the things that I appreciated very much is the methodology and especially a lot of people that are not going to know to normal people can't break things down and turn them into processes and it's something that I have done more than once. And so I really appreciated that and I was so glad when you open sourced it. That was that was a brilliant move on your part." }, { "speaker": "Bronwen Aker", "startTime": "1570.435", "endTime": "1575.0751", "body": "And I'm eternally eternally grateful, I just wanted to tell you since you're here." }, { "speaker": "Daniel Miessler", "startTime": "1575.0751", "endTime": "1590.6901", "body": "Yeah. Thank you. I feel like the the concept has held up fairly well, because wasn't it mostly a mapping system of, like, how to get your your human tasks that you want to do and use AI to help you with them. Like, that was the fundamental idea. Yeah." }, { "speaker": "Bronwen Aker", "startTime": "1590.6901", "endTime": "1613.085", "body": "Well, you've said over and over again, the human focus is very much where your headspace is in terms of being able to utilize the tools to the best advantage for humans. And Yeah. That was consistent from the augmented course. What was that 2023 or was I mean, it was a while ago." }, { "speaker": "Daniel Miessler", "startTime": "1613.76", "endTime": "1620.24", "body": "Yeah. It was yeah. Definitely not '22. '23. Must have been the end of '23." }, { "speaker": "Bronwen Aker", "startTime": "1620.3201", "endTime": "1643.5599", "body": "Yeah. And and again, that that focus on what is a problem that you're trying to solve, And here's a framework of how to get the most bang for your buck from these tools to help you solve the problem. So that was and and fabric is still doing that, which is why it's so wonderful in my opinion." }, { "speaker": "Daniel Miessler", "startTime": "1643.56", "endTime": "1644.28", "body": "Thank you." }, { "speaker": "Joff Thyer", "startTime": "1644.6", "endTime": "1701.5399", "body": "Let's turn the focus over to cybersecurity for minute because we've talked a lot about you know, we've talked about Fabric. We've talked about tooling. We talked about some AI fundamentals and some some concerns in around natural language processing. But cybersecurity is is the industry we're all in even though we've taken a very large detour, especially this group down the AI path and and trying to understand it. So I guess the first question is how how concerned the security professionals need to be about the risks around particularly large language models and the deployment of large language models in the in individual organizations and or the usage of some of these frontier mod models via various software API interfaces." }, { "speaker": "Joff Thyer", "startTime": "1701.86", "endTime": "1723.32", "body": "And then I guess a follow on question, which is I think probably the bigger question is, should us about cybersecurity professionals be worried about our future jobs, you know, from that from that knowledge worker perspective with with the advances of AI, or should we be perhaps embracing it in a different way? So let's let's go with risks first." }, { "speaker": "Daniel Miessler", "startTime": "1723.9601", "endTime": "1762.23", "body": "Yeah. Risks. I I would say there's significant risk. I I would say, generally, the reason there's so much risk is because so much more is being created. If if you suddenly have millions of people, potentially tens or hundreds of millions of people making things, and they are not engineers in the civil engineer type, you know, nomenclature where where if you have people designing bridges who didn't go to school to design bridges, you gotta be careful when you drive over them." }, { "speaker": "Daniel Miessler", "startTime": "1762.23", "endTime": "1785.41", "body": "So and then that's the same thing with, like, building all these applications because it's pretty easy to vibe code, not not like spec code, but purely vibe code, a fully working web application. And guess what? You could tell an AI to, write a login form and make it so people can log in. Yeah. It'll probably use TLS, but how is it gonna store the credentials?" }, { "speaker": "Daniel Miessler", "startTime": "1785.41", "endTime": "1803.515", "body": "Like, how are they gonna make passwords? Like, it it could really get it wrong. And if you have hundreds of millions of people building that kind of stuff and you have API keys, and just like a whole lot of garbage is being created right now as we speak. Mhmm. And, I don't think that's a long term problem." }, { "speaker": "Daniel Miessler", "startTime": "1803.515", "endTime": "1820.1951", "body": "I think it's more of a short term problem. But it is building a lot of cruft that that's gonna have to be cleaned up on the Internet. So I I would say that's kind of the number one risk. I would say there's lots of independent tactical risks that are gonna come from this. Like, we're gonna have outages." }, { "speaker": "Daniel Miessler", "startTime": "1820.1951", "endTime": "1848.0549", "body": "We're gonna have breaches. We're gonna have whatever. But I don't think they really matter that much. The the amount of banking fraud that exists day to day, how many millions of dollars are being stolen day to day in banking, across the infrastructure that we use every day that's just kind of common is extraordinary. It's massive, you know, millions of dollars being lost every day." }, { "speaker": "Daniel Miessler", "startTime": "1848.455", "endTime": "1875.74", "body": "Guess what? Banks work fine. Like, the whole structure still works. Like, I I don't think that's too much of an issue. As far as jobs, though, I am much more worried about jobs just because I think for most people, the task that people do to get paid on a daily basis are starting to move below the bar of what AI can do." }, { "speaker": "Daniel Miessler", "startTime": "1876.275", "endTime": "1888.515", "body": "Yeah. And this is not a problem of in my mind, it's not a problem of people being bad at their jobs or AI being good. It's just a matter that those thresholds are switching." }, { "speaker": "Derek Banks", "startTime": "1888.515", "endTime": "1890.035", "body": "It's a cost benefit analysis." }, { "speaker": "Daniel Miessler", "startTime": "1890.6", "endTime": "1914.475", "body": "Yeah. And so here here's the best way to think about this, in my opinion, is just imagine that all creators of any company suddenly became aliens, and they have 10,000 heads and brains. So they're very wide. They have 10,000 heads and brains and that's how they walk around the world. And they have 40,000 arms and hands." }, { "speaker": "Daniel Miessler", "startTime": "1915.515", "endTime": "1933.1101", "body": "Except for the 40,000 arms and hands can reach into computers and also it could pick up things and move things or whatever. Well, guess what? They wouldn't hire an employees. Yeah. If you go back to the very beginning, a founder would just do the work." }, { "speaker": "Daniel Miessler", "startTime": "1934.1051", "endTime": "1956.3899", "body": "Yeah. Founders literally hire employees because they can't physically do the work themselves. So if they could just give five of their heads, you know, the, the admin job and five of their heads the engineer job And and keep extending that for all the other heads that they have, they would be one person shops." }, { "speaker": "Joff Thyer", "startTime": "1957.03", "endTime": "1957.6699", "body": "Right." }, { "speaker": "Daniel Miessler", "startTime": "1958.39", "endTime": "1983.85", "body": "And and and that is a natural state for that to happen. That is that is natural. So it's almost like what we have now with our current economy that's based on work is kind of an unnatural state. It's it's like, it's just what happened because people can't do all the work. So AI and tech in general and automation in general is simply enabling those founders to not have employees anymore." }, { "speaker": "Daniel Miessler", "startTime": "1983.85", "endTime": "2005.5751", "body": "So this is why I'm so obsessed with the human focus and human three point o because all that disappears. Now the question is, what are we gonna talk about on this call when all the work can be done by these other agents? We can still talk about the books we wanna write. We could talk about the worlds we wanna create for each other. We could talk about the communities we wanna build." }, { "speaker": "Daniel Miessler", "startTime": "2006.35", "endTime": "2012.3499", "body": "Friendships, coffee, love, like, all these things don't require that we have a nine to five day job." }, { "speaker": "Derek Banks", "startTime": "2012.59", "endTime": "2027.875", "body": "Yeah. I think you had said in a previous blog post or maybe, like, a pie guess, the ideal ideal number of employees for a company is zero. Like, that Yeah. That's really if you could get by with zero employees, then that's what the company would do. Right?" }, { "speaker": "Joff Thyer", "startTime": "2027.955", "endTime": "2035.715", "body": "Well, so so what we're saying is, you know, we are seeing a process of of the economic system being upended. Right?" }, { "speaker": "Daniel Miessler", "startTime": "2035.715", "endTime": "2036.115", "body": "Yes." }, { "speaker": "Joff Thyer", "startTime": "2036.115", "endTime": "2046.33", "body": "So clearly, that's a concern that we're we're we're going to have to wrestle with because I don't think we can put the genie back in the bottle at this point." }, { "speaker": "Derek Banks", "startTime": "2046.33", "endTime": "2059.945", "body": "Well, no. I think that the current state of models is already there where a lot of, like, you know, what people do, like you're saying, day to day. Like, I think that if you glue it together in the right way, the Yeah. The models themselves are already capable enough to replace." }, { "speaker": "Joff Thyer", "startTime": "2059.945", "endTime": "2062.585", "body": "That's that's the aspect I wanna zoom in on. Alright?" }, { "speaker": "Derek Banks", "startTime": "2062.585", "endTime": "2079.66", "body": "Yes. It's just the trust factor because right now, especially in information security, but I'm sure other way places, people are still like, oh, yeah. But what about hallucinations? Which in my opinion, anecdotally, have really diminished in the frontier models Yeah. Over the last, like, year or so." }, { "speaker": "Derek Banks", "startTime": "2079.66", "endTime": "2087.865", "body": "Right? Yes. But but then, you know, my my retort to that is what about hallucinations? Like, well, have you ever had a coworker who's wrong? I mean, of course." }, { "speaker": "Derek Banks", "startTime": "2087.865", "endTime": "2098.59", "body": "Right? And so just because it's occasionally wrong doesn't, in my opinion, discount what it can do. Right? Yeah. I still I still do believe human in the loop, but I'm with you." }, { "speaker": "Derek Banks", "startTime": "2098.59", "endTime": "2115.995", "body": "If I'm a web app penetration tester and I can get an AI to do 50% of my job, I'm like I'm twice as good as the next person. Right? And so Yep. That's where I think we'll see job losses, folks who aren't able to compete with people who are using AI in in a very efficient manner." }, { "speaker": "Joff Thyer", "startTime": "2116.155", "endTime": "2160.14", "body": "So that that's where I that's where I wanna step in. Right? Because right now, we're at this juncture where, yes, we are seeing some autonomy begin to development develop. We are seeing the the fledgling agentic space start to take off where where the output of AI's actually can have actions, they can even go as far as kinetic actions and so forth, but on the input side of the equation where we are seeing competitive advantage from my viewpoint is people that can very very creatively create the specs. Right?" }, { "speaker": "Joff Thyer", "startTime": "2160.14", "endTime": "2177.275", "body": "Create the the appropriate context. Are we in a space where we are on the edge of pivoting into autonomy on the input side where the human is not needed is the real question I think that we need to get at." }, { "speaker": "Daniel Miessler", "startTime": "2179.1501", "endTime": "2209.9702", "body": "It depends on the task. I I would say, if you're trying to replace a really, really creative, movie or script or comic or or something that's like pinnacle human creativity, we're not there yet. We're also not at the point yet of going from Newtonian physics to, Einsteinian physics. Right? So for Einstein to say, hey." }, { "speaker": "Daniel Miessler", "startTime": "2209.9702", "endTime": "2229.855", "body": "What if it's just like you have a rubber sheet and you drop a bowling ball on it, and the bending of the sheep sheet is actually the way gravity works? That's not a thing that AI can do yet. Mm-mm. And there's smaller versions of that that we all do occasionally where we're really creative at work. Right?" }, { "speaker": "Daniel Miessler", "startTime": "2229.855", "endTime": "2251.03", "body": "And I think those are the harder things to get to. The problem is the AI will just keep moving up through there. And the really difficult thing is that, let's say that bar is here. When AI moves up and hits it, there's no reason it's gonna stop there. Because being five levels above that is kind of arbitrary, I think." }, { "speaker": "Daniel Miessler", "startTime": "2251.615", "endTime": "2261.2952", "body": "It's likely to be arbitrary, and who knows where it'll stop at what point, but there's not a magical human level that it's gonna be hard to pass." }, { "speaker": "Joff Thyer", "startTime": "2262.175", "endTime": "2292.075", "body": "Yeah. So I I I struggle with this too. This very quickly delves into the the space of deep philosophical debate where we start to start we start to explore the areas of, you know, our our large language frontier models verging on the the space of being sentient? Is it is it really simulated digital reasoning anymore? Or has it become its own form of reasoning?" }, { "speaker": "Derek Banks", "startTime": "2292.075", "endTime": "2296.075", "body": "Well, sentience is the same thing as intelligence or consciousness." }, { "speaker": "Joff Thyer", "startTime": "2296.075", "endTime": "2296.2349", "body": "Right." }, { "speaker": "Derek Banks", "startTime": "2296.4001", "endTime": "2298.24", "body": "Right? Like, I I I think Yeah." }, { "speaker": "Daniel Miessler", "startTime": "2298.24", "endTime": "2305.36", "body": "I I consider them very different axes. I definitely think they are reasoning, and I don't think they're anywhere close to sentience as far as I could tell." }, { "speaker": "Derek Banks", "startTime": "2305.4402", "endTime": "2306.6401", "body": "Yeah. And that's that's" }, { "speaker": "Joff Thyer", "startTime": "2306.6401", "endTime": "2333.4302", "body": "my viewpoint as well, Daniel. I think you have to work with them for quite a period of time to come to that place because I do think there's a lot of FUD, frankly, out there. Right? There's this sort of fear, uncertainty, doubt. People don't know what is possible even because they haven't explored the area enough to to and they they they just know the media sound bites." }, { "speaker": "Daniel Miessler", "startTime": "2333.4302", "endTime": "2334.2302", "body": "Well, yeah." }, { "speaker": "Derek Banks", "startTime": "2334.3901", "endTime": "2349.895", "body": "One of the reasons that Joffe and I wrote that foundations class was we were trying to like, could tell when I would talk to folks, especially in information security, they would make comments. And I could tell they didn't really understand like what the thing even really was. Like, what is a large language model?" }, { "speaker": "Joff Thyer", "startTime": "2350.7", "endTime": "2357.3398", "body": "So let's let's let's hit a couple more questions. We've actually taken a pretty good amount of Daniel's time, so we I" }, { "speaker": "Derek Banks", "startTime": "2357.3398", "endTime": "2357.66", "body": "wanna be" }, { "speaker": "Joff Thyer", "startTime": "2357.66", "endTime": "2359.0999", "body": "respectful of your And thank" }, { "speaker": "Bronwen Aker", "startTime": "2359.0999", "endTime": "2360.6199", "body": "you so much again." }, { "speaker": "Daniel Miessler", "startTime": "2361.3398", "endTime": "2362.22", "body": "Yeah. No problem." }, { "speaker": "Joff Thyer", "startTime": "2362.22", "endTime": "2362.8599", "body": "And if I haven't" }, { "speaker": "Derek Banks", "startTime": "2362.8599", "endTime": "2366.295", "body": "said this already, thank you for seclists too. That's really came in." }, { "speaker": "Daniel Miessler", "startTime": "2366.295", "endTime": "2367.255", "body": "Oh, yeah. Won that one." }, { "speaker": "Joff Thyer", "startTime": "2367.255", "endTime": "2378.215", "body": "That was that was great. And I have I have one at some point, I hope to meet you in person and and tell you some of the more interesting things we've done with Sicklists, but that's that's" }, { "speaker": "Daniel Miessler", "startTime": "2378.215", "endTime": "2378.375", "body": "a" }, { "speaker": "Joff Thyer", "startTime": "2378.375", "endTime": "2432.035", "body": "conversation not for a podcast. There's a a couple of last questions I think I have here, and that is we've seen recent press from Sam Altman on the impending release of GPT five, and we've seen some very concerning statements from him about its extraordinary capability. And related to that question is, you know, where do you, you know, given that we're in, you know, 2025, I I kinda feel like we're at this inflection point where things are gonna start accelerating even more. That's my personal view. But what do you see what do you see coming for us late this year and into '26 as we see these models take that very next step, in aka GPT five and and Companions?" }, { "speaker": "Daniel Miessler", "startTime": "2432.595", "endTime": "2447.8", "body": "Yeah. Yeah. Great question. So I I think I think a lot of the benefit that's that's gonna come or a lot of the power that we're about to see is not actually from model improvement. I think model improvement helps a lot." }, { "speaker": "Daniel Miessler", "startTime": "2447.8", "endTime": "2466.6401", "body": "Like, the smarter the model it is, the more it can mess with you know, it can handle bad prompting and bad scaffolding. But the benefit's actually going to come from the scaffolding. Or I think what you said, Derek, glue or plumbing. I I call it scaffolding. I call it glue, plumbing, wiring." }, { "speaker": "Daniel Miessler", "startTime": "2467.04", "endTime": "2492.785", "body": "It's actually just the stitching together of these models into a working system. Because the way humans work, right, we have to have a working memory. We have to be able to recall old tasks we learned back in college with arithmetic. We have to be also remember what our manager said yesterday about how the project is different. And this is a problem we currently have is that that whole structure, like, is kind of still kludgy right now." }, { "speaker": "Daniel Miessler", "startTime": "2493.2002", "endTime": "2529.25", "body": "So you couldn't actually hire an AI to do any one of our jobs because it would just kind of lose the plot after a couple of weeks or even a couple of hours. It would lose the plot, and it would turn into a very bad employee. So that progress there is actually, I think, what most AI companies are gonna be working on. Progress on the stitching and the and the plumbing is actually what's gonna get us, in my opinion, to AGI, which is the ability to replace a knowledge worker. That's that's the definition I use." }, { "speaker": "Daniel Miessler", "startTime": "2529.25", "endTime": "2554.91", "body": "Yeah. Because it'll be general enough to do the work of most knowledge workers. So I think, model advancements is not where it's at, in my opinion. I think it's it's scaffolding advancements. And I think companies are just now starting to deploy scaffolding, and companies are just now starting to work on scaffolding based AI worker replacements." }, { "speaker": "Daniel Miessler", "startTime": "2555.71", "endTime": "2573.615", "body": "And that's what's gonna be your developer and your project manager and your planner and your marketer. You know, you're you're literally just gonna hire one of these things, and you're gonna send it to onboarding on Monday just like other employees. Yeah. And it's gonna go read all the docs and come back and say, okay. I understand." }, { "speaker": "Daniel Miessler", "startTime": "2573.615", "endTime": "2584.0598", "body": "What do you want me to do? It's gonna show up in Slack. It's gonna show up to team meetings. Like, that that's that's what I see happening, not because of models, but because of scaffolding." }, { "speaker": "Joff Thyer", "startTime": "2584.0598", "endTime": "2599.0452", "body": "Right. And it'll and it'll be up here. And I think I think you're absolutely right. And it kind of addresses my earlier question of of, you know, discrete models and discrete natural language prompting is one thing. Putting it all together into a cohesive system is still occurring." }, { "speaker": "Joff Thyer", "startTime": "2599.445", "endTime": "2606.3252", "body": "And I think I think you're right on right on target. Any final short questions from anybody before we let Daniel go?" }, { "speaker": "Bronwen Aker", "startTime": "2607.74", "endTime": "2611.02", "body": "What will it take for us to get you to WellWiz Hackenfest?" }, { "speaker": "Daniel Miessler", "startTime": "2615.74", "endTime": "2619.595", "body": "Not too much, actually. Too much." }, { "speaker": "Bronwen Aker", "startTime": "2619.595", "endTime": "2630.1553", "body": "We would love to have you there. I this year, I believe, is awash because of scheduling conflicts. But what we've got, Denver in February and, of course" }, { "speaker": "Derek Banks", "startTime": "2630.155", "endTime": "2637.7102", "body": "Let's everybody wants to go to Denver in February. So it wasn't that bad this past year. It was nice." }, { "speaker": "Daniel Miessler", "startTime": "2637.79", "endTime": "2642.03", "body": "Actually, I Denver in February, let's talk about that one." }, { "speaker": "Joff Thyer", "startTime": "2642.19", "endTime": "2646.35", "body": "Okay. Okay. Yeah. Yeah. I think it's the second week." }, { "speaker": "Joff Thyer", "startTime": "2647.1948", "endTime": "2647.9949", "body": "Well, we can always" }, { "speaker": "Derek Banks", "startTime": "2647.9949", "endTime": "2648.635", "body": "look at that date." }, { "speaker": "Bronwen Aker", "startTime": "2648.635", "endTime": "2664.27", "body": "It's early in February. I can I'll confirm the dates and ship them to you, and I'll also I'll probably send an e an email that will connect you with the people organizing it, so we can make sure that you get get signed up because that would be so awesome." }, { "speaker": "Daniel Miessler", "startTime": "2664.67", "endTime": "2664.91", "body": "Okay." }, { "speaker": "Joff Thyer", "startTime": "2664.91", "endTime": "2687.255", "body": "I know that I'm doing an AI teach with Derek probably at that event. And I'm also doing an AI teach in Vegas like the week before, so February is gonna be busy next year for me. But anyway, that all aside, thank you so much, Daniel, for being here today. It was a it was a pleasure having you, and I hope we will we can see you again on the show sometime." }, { "speaker": "Daniel Miessler", "startTime": "2687.4148", "endTime": "2689.4148", "body": "Awesome. Thanks for having me. Appreciate it." }, { "speaker": "Brian Fehrman", "startTime": "2689.4148", "endTime": "2691.575", "body": "Thank you so Appreciate it. Yep." }, { "speaker": "Joff Thyer", "startTime": "2691.575", "endTime": "2693.1748", "body": "Alright. Hit the button, Brian." } ] }