Podcast audio-only versions of weekly webcasts from Black Hills Information Security
Hello, everybody. Welcome to today's Black Hills Information Security webcast. Today joining me is Patterson Cake, and I've known Patterson for a long, long, long time. I got a chance to meet him when I worked at another training organization. I I met him there, and was immediately impressed.
Jason Blanchard:Immediately impressed with the way that it conveyed information, immediately compressed with his heart towards helping other people understand, and so it is a joy whenever Patterson agrees to do another webcast for us. So he's got a training class coming up soon, but for the most part, we're here today to talk about this one topic, and it is on incident response, things that you can do, and do it now. Alright, so you can start doing these things now to get ready for the possible incident in the future. Alright, and so Patterson's gonna give the webcast, we'll do some q and a at the end. If you haven't yet checked in for Hackett, you can go ahead and do that now.
Jason Blanchard:If you don't know what I'm talking about, I'll talk about it at the end. And thank you so much for joining us, you could have done anything with this hour, you chose to come here and learn and better yourself, and better your organization, and we appreciate that. And with that, I'm turning it over to Patterson. I'll be back when you're ready.
Patterson Cake:Thank you.
Jason Blanchard:Okay. Good luck.
Patterson Cake:Thank you for being here today. Thank you for the opportunity. It's it's been a minute and I always enjoy these conversations. It's a fantastic crowd audience, so again, really grateful for the opportunity to talk about incident preparedness. In anticipation of this conversation I cannot help but reflect on an impactful moment in my career.
Patterson Cake:I was sitting in the IT Dungeon. Ever been in the IT Dungeon? The data center of course is in the basement and there are no windows and it's dark and dank and noisy and I'm sitting in the dungeon. It's day two or day three of a serious significant ransomware outbreak case and I am at a desk, head down, hands on keyboard, hunting down the threat actor and we're in that containment eradication phase. It's pretty intense, pretty involved.
Patterson Cake:It's been a couple really long days. Pretty bustling place, noisy, lots of other IT staff, security staff, and again, I'm working away and I I notice there's a quiet that comes over the dungeon and I I All I can do is hear myself clacking away at the keyboard and so I stop, I look around and I pivot in my chair and standing behind me is the CEO and his entourage. And it's pretty clear by the look on their faces and everyone else faces that maybe this is the first time the CEO's ever been to the IT dungeon. The CEO stops and he says, know you're busy, but I need to ask you a question. Shoot.
Patterson Cake:The CEO says to me, if you were me, what three things would you want to know about this incident right now? And as mentioned, it's been a couple three twenty, twenty two, twenty three hour days. I'm a little raw, maybe not as poised and polished as I would like to be under the circumstances. What three things would you want to know if you were the CEO? And I said, you're understaffed, you're under budget, and people listening to this conversation right now knew what needed to be done to avoid this incident.
Patterson Cake:After they escorted me out of the building, just kidding, the IT crowd, the security crowd, man, I made friends for life in that moment. It was just honest, it was just from the hip, and to the CEO's credit he said, thank you so much, can we please convene a meeting to talk more about this later, and much to my relief, I'm here to tell you honestly that is one of the best part, it may be the best part of doing active incident response for a living is that moment, that moment of clarity that comes out the other side of crisis where the organization is ready to listen to the things that you've been telling them for months, maybe years. Unfortunately in that specific scenario, in that story, the direct costs of that incident were high 6 figures, direct costs. The indirect costs pushed that into several million dollars of response. The lessons learned that came out of that came at a incredibly high price, literally millions of dollars.
Patterson Cake:My hope today, my hope in almost every conversation I have related ints and responses to channel that moment, to channel that perspective and say stop, stop. We don't have to enter into a multi million dollar crisis to make some improvements in our security posture to be better prepared should we have an active incident. So welcome to an incident preparedness checklist, notice recurrent themes. I've been doing this a lot for quite a long time and I've lost count of the major incidents that I have led response to, seventy, eighty, 90, I really don't know, but a lot, and there are recurrent themes. Recurrent themes that I will suggest today can be fairly easily addressed, fairly inexpensively significant return on investment.
Patterson Cake:Hi, my name is Patterson Cake. I am the director of incident response for Black Hills Information Security and I've come to the very recent conclusion that I am an incident response contrarian and what I've discovered is that as practical technicians, as IT and security professionals, we we are trying to behave and operate a certain way and and this is not this is not a criticism of that, but incident response is different. Incident response is different. Incident response should be different. Incident response is not a time for subtlety, it is not a time for strategy, it is the time for decisive action.
Patterson Cake:I was thinking to myself earlier if my job were a video game, anybody like video games? My job were an FPS, I would be a breacher. I'm not the sneaky type, I'm not the subtle type. I am the kick down the door, eliminate the threat, and get back to business as usual, and frankly when you kick down the door, you broke it, right? You broke it.
Patterson Cake:And in that moment, nobody cares. I set the stage because that is my perspective, that is where I live, that is not necessarily where you live, so keep that in mind, context is king as we move forward, and that is that is my day job, so to speak, so keep that perspective in mind as we go through the remainder of this conversation, because I know I'm a little different. Oh my god, about an hour, brief introduction, a little bit of presupposition, then we're talking about 10 things. Your Hopefully, to do list. Maybe, honestly, hopefully, you've done all of them already, which would be fantastic, and then we'll end as per usual with some additional external resources so you can come back and and investigate this a little more later.
Patterson Cake:Cybersecurity is the art of the possible, the attainable, the art of the next best. Yes, I butchered that quote on purpose, but it is a mantra for me and my point is that we need to do something actionable. We need to come out of this conversation whether you do one or two or 10 things, please oh please, find something that can be directly applicable to you, your role, your environment. Necessarily. I'm going to use a little bit of hyperbole.
Patterson Cake:I'm gonna do my best to shake up your paradigm just a little bit, and so I'm gonna draw on extreme examples. The next piece is oversimplified. I'm gonna work every step of the way to keep this as simple as possible. Crisis, crisis intervention is complicated enough all by itself, and then these are absolutely generalities. Your environment, your business is a snowflake.
Patterson Cake:Every single one is of course. Having said that I've noticed that snowflakes have a lot of commonalities. So my challenge to you today is to take at least some of what we discuss and then modify it, customize it so it is applicable to your particular snowflake, again with the idea of the art of the possible. What can you accomplish? Don't let the perfect be the enemy of the good.
Patterson Cake:What can you actually accomplish today, this coming week, this coming month as opposed to your five year strategic plan. Without further ado, I'm gonna ask you to use your imagination just a little bit today, is a little bit out of the ordinary for my typical technical conversations. I want you to imagine for just a moment that you started a new job in a sizable enterprise. It's your first day or two on the job and you find yourself in the typical sprawling cube farm. Have you ever worked in a cube farm where they put distinct pictures in different areas of the office space, a tractor, a tree, a mountain, specifically so you'll have some idea where you are because otherwise you would have no clue.
Patterson Cake:You're new on the job. It's late at night. You're trying to impress your team, your boss. You find yourself working alone in the cube farm, and as you are sitting there working away, the fire alarm goes off. You panic ever so slightly and you realize you have no idea how best to evacuate the building, you're not even sure exactly where you are, but you remember that there's an emergency sign, a placard on the wall nearby, and so you rush through that sign hoping for clarity about how to escape from the potential fire.
Patterson Cake:As you reach the sign, smelling smoke ever so slightly, you find a little booklet, you open the booklet, and the booklet begins with what is a fire? Why do we need a fire escape plan? Let's talk about fire related terms. How about different types of fire extinguishers? Here's a runbook for how to escape from an electrical fire, and by this point I'm sorry to report you've died of smoke inhalation.
Patterson Cake:This is exactly how we write our incident response plans. Number one, number one, I have literally quantified seventy, eighty, 90 major incident response technical leadership, project leadership, and I can tell you exactly how many times, exactly how many times an organization has said, let's reference our incident response plan. Absolutely never. It has literally never come up and that is a scathing review. Your IR plan is likely useless, it does not have to be.
Patterson Cake:It does not have to be. I have an outline of things I think can and should be included in your incident response plan. It should be about yay long. 10 pages is probably far too long. It doesn't need to be, it needs to be simple.
Patterson Cake:I'm a huge fan of modularity, meaning a body and potential adjuncts or appendices etcetera, and a handful of critical components. Roles and responsibilities, how do you get ahold of critical personnel, vendors, etcetera, a little bit of severity priority, a little bit of information about who has the authority to do what, the end, the end. I have an incident response plan template that I have a link to there, I'll reshare that link towards the end just as a potential option to come out of what I call a tactical incident response plan. You may still need a fifty, sixty page document, need, from a strategic compliance and overarching integration perspective, but I'm talking about actual set of tactical useful information that might be of value in the face of active crisis intervention. That's number one.
Patterson Cake:Ironically, at the outset of the vast majority of incident response engagements, have a question. I have lots of questions, but one of those is almost always do you have cybersecurity insurance? And there is a singular answer to that question nine times out of 10. The vast majority of the time I say to the cybersecurity lead, whether it's the CISO or the VP of InfoSec or something along those lines, do you have cybersecurity insurance? And the almost invariable response is, I think so.
Patterson Cake:You think so? And I put this diagram in because, well, it made me smile. But this is exactly how most organizations relate to their cybersecurity insurance policy. Okay, we've got on a little fender bender, no worries, I'm pretty sure I can fix it. I really don't have any experience in this area, but I got some Vaseline and a plunger, and I'm pretty sure I can make it right.
Patterson Cake:Don't, no, don't call insurance. Don't call them. Number two, your cyber insurance provider should not be your adversary, they should be your business partner. And trust me on this one, if you are unclear, unsure about any of these components, now is the time to deal with it. Now is the time to engage with your insurance provider and find out if they are there to support you.
Patterson Cake:Find out critical components like when do you want us to call you? Do you want us to wait till we've done an initial investigation? Do you want us to alert you right out of the gate? The vast majority of orders are like, don't call them, don't call them. In fact, sometimes they'll call me first and say, we really need to find out how terrible this is before we call our insurance provider.
Patterson Cake:I'm like, ugh. Figure this out in advance. The third bullet point is super critical, and often this is a mystery to the average organization. If you call your cyber insurance provider, are you allowed to engage with Black Hills? Are you required to use Mandiant or CrowdStrike or some other vendor?
Patterson Cake:Get that figured out in advance. And then the final many bullet points you can read on your own, and these are things that honestly can and should be in the aforementioned tactical IR plan. I got a call from an organization a couple months back, and they called and said we're gonna need some help performing an investigation. We've been targeted by an advanced persistent threat. I'm always fascinated to find out how did you become aware of this?
Patterson Cake:And the organization said, well, the FBI called us, and the FBI let us know that they had found evidence that we were targeted by an APT group, advanced persistent threat nation state, you know, big deal, heavy hitters, and okay, fascinating. What information did the FBI give you? And in typical fashion they gave very little information. I said, well, do you have any idea when this began, when this potential incident began? And they said, have we asked the FBI that?
Patterson Cake:And they said April.
Patterson Cake:I said,
Patterson Cake:okay, April? Okay, it's March now, so I'm a tiny confused, but that's it. April? They said, yeah, April 2023. Excuse me?
Patterson Cake:Yeah, April 2023. Thanks a lot, FBI. Really no offense to the FBI, but legitimately they notified this organization almost three years after the fact, and it is, talk about hyperbole, it is a quintessential example of the inadequacy of our log retention. I'm here to tell you that there are many, many, many painful conversations in response to an active incident. How much telemetry do you have?
Patterson Cake:How far can we look back for investigation? And sometimes the answer is seven days. Seven days is woefully inadequate. Fourteen days is pretty tight. Thirty days is minimal from my perspective.
Patterson Cake:Ninety days, okay, now we're talking, now we're talking. Number three, chances are your log detail and retention are inadequate and I hate, I hate these conversations. I hate having to go to the organization and say, I'm sorry I can't help you. I can't help you answer the question of what happened three weeks ago because you have no more telemetry. Super painful and something that can and should be reviewed today, right away.
Patterson Cake:You have internal requirements, you have regulatory requirements, the cloud facing piece is tough. This is probably the most critical gotcha of lack of understanding of what exactly is being logged, what exactly is being audited, and how long do I get to keep it, and I can't tell you how many times a customer said, can I call Microsoft and increase our retention? And I say, yeah. Yeah, you can. But that doesn't help you for what happened three weeks ago.
Patterson Cake:Review these pieces, review your business critical applications. Did I mention cloud, m three sixty five, AWS, Azure, GCP? Make sure that you are collecting the right data, the right telemetry, and that you're keeping it for an adequate period of time. Most importantly, and I mean this, these are business decisions. You can't just turn on all the things and keep them for all eternity, it costs money.
Patterson Cake:These are business decisions. Most important takeaway from this one is review this, document it, and make sure that the organization is aware that we only have thirty days or we only have fourteen days or or or. Save yourself the pain and anguish and then of course it is a business decision, you can make recommendations accordingly and once again I would push for a minimum of thirty day retention on any of these components. Ninety days is better, obviously six months to a year is better, but recognizing there's a cost, review, logging adequacy, and retention. My job is affectionately write a boom after bad things happen, And one of the most challenging conversations we have right of boom, we roll in and containment and eradication are often our first priorities, stop the bleeding if you will, evict the threat actor, and then start moving back into where do we go from here?
Patterson Cake:How do we get back to business as usual? And one of the most painful conversations in the recovery and remediation phase is, let's talk about what is your most important information technology asset? What is most important to you as we think about restoration and recovery? And I kid you not, I have been in conversations that took hours and hours and hours, occasionally more than a day to determine what do we care most about? What should we prioritize in restoration services?
Patterson Cake:Fascinatingly, often the outcome of that conversation is a singular response, what's most important in your environment? What's most important in your environment? The answer, payroll. Almost every time payroll, which makes pretty good sense, you don't want everybody to walk off. Number four, strongly suggest that you identify tier zero and tier one priorities.
Patterson Cake:What are your most important assets? And remember, remember, please, the art of the possible, chances are you have 500, 700, 1,000, 1,500 different categorical assets. You don't have to finish the whole thing. You really don't, at least for the purposes of this conversation, but you do absolutely need to find tier zero and tier one infrastructure. And by tier zero things like o active directory and active directory and o active directory and maybe intra.
Patterson Cake:The core underlying components of identity and access management are absolutely top of the list and then we work our way down to payroll and payroll and business critical apps and ERPs and health management systems and etcetera, etcetera. Define this list even if it's 10 things or 20 things, you don't have to finish. You don't have to get to tier two and three and four at least in advance of crisis, but please oh please identify your most critical assets for prioritize restoration and recovery in advance of crisis. I don't know, I really don't think I can overemphasize this particular line item, and that is the criticality of collaboration and communication in response to crisis, and sadly by far the most prevalent type of case that we work is, you guessed it, business email compromise. And in that moment where your M365 or your GWS tenant is compromised, that's not the time to be thinking about how do we communicate?
Patterson Cake:How do we communicate with staff? How do we let everybody know what's going on? And truth be told, this one can be pretty much free. Implement out of band communications like now, please, before crisis, and trust me on this one, don't just turn it up and then forget about it. Can you remember URL, username, password, access mechanism for something you used six months ago?
Patterson Cake:I can't, unless I was fortunate enough to record it somewhere. This is huge. Implement an out of van emergency communication solution immediately, socialize that mechanism, and I mean engage with the business leadership team, the board, whoever needs to be involved, and make sure they test it every now and again, make sure they know how to get to it. This can be literally free. Free may not be the best solution, but there are lots of different options.
Patterson Cake:Make sure that you have the accessibility information somehow offline and accessible to the teams that are involved. Make sure you have critical primary, secondary, tertiary roles and assignments defined. This is just, it's just stupid simple and there's really no excuse for having this in place and maybe you'll never need it, hope that you never need it. But if you do, this is a tremendous efficiency improvement in your active IR response every time. Many many times in active IR day one, day two as per my introductory story is a really really long day.
Patterson Cake:And it is not uncommon for us to spend twenty plus hours on day one in active response, especially if it's something that's a malware outbreak or a ransomware outbreak, and that's tough. You know, I'm not as young as I used to be, and by the time I get to twenty plus hours in a day, fact of the matter is sometimes I forget to eat, sometimes I forget to drink. It's just my own stupidity, but I'm, you know, I'm up to my eyeballs in alligators. I'm focused and intense, and a lot of times it seems like it went by quickly, and I remember one specific fairly recent where I worked twenty three hours straight, and at the end of that we were actually working on restoration of tier zero functionality, Active Directory, and if you've ever restored Active Directory, you know, talk about eyeballs up the alligators. You gotta get it right.
Patterson Cake:You gotta do the timing and the sequence and the restoration, and we went through many hours of preparation to restore the identity infrastructure, and we chose the wrong snapshots and blew up AD completely at hour twenty three, and in that moment at least somebody had a clear enough head to say, why you go home? Why don't you go home and get a little bit of rest? And it's counterintuitive, like I can't go home right now. And the truth of the matter is that I'm not worth a whole heck of a lot after twenty hours, and neither are most of you, and this is really, really important. I have this conversation with customers all the time.
Patterson Cake:We do a lot of tabletop exercises, and one of the things I love to bring up in the middle of a cybersecurity incident response scenario is your team has been working for three days straight. They need to go home. They need some rest. They probably haven't had a good meal. We need to think about that, health and safety, not only for the employee's benefit, but for the efficiency of the response process.
Patterson Cake:You need a staff rotation policy. I have literally heard executive teams just say, Well, they're just gonna keep working. I got three people, and we're just gonna keep working until we get it done. I say, I'm sorry, that's just foolishness. That is foolishness.
Patterson Cake:Think this thing through, define maximum shift durations, look for third party support, come up with a plan, because you can't work twenty four hours straight for more than a day or two max. And again, it's not good for anybody. One of the things I love, and actually in the scenario, in the story that I told at the beginning in the IT dungeon, there was a fantastic lady that was working there. She was former clinical support and she'd moved into an IT supervisory role, and that woman saved our lives and made the entire response process at least 30% more efficient because she walked around and said, would you like some water? Can I get you a hamburger?
Patterson Cake:And I'm telling you, that kind of attention to your staff health and safety is monumentally beneficial. For some unknown reason, I don't know, I guess, you know, small businesses, we don't have a dedicated security team, we don't have a dedicated incident response team, that's fine, I get it, but that's not an excuse. There are places, oh, like Black Hills and Antisyphon where you can get free or almost free high quality training to prepare your staff for complex interactions and engagements just like this. IR is not something generally you're doing all day every day, and your teams can benefit directly and specifically from training and practice in incident response. IR is a perishable skill.
Patterson Cake:I love this quote that no man fears to do that which he knows he does well. Who's afraid of active IR? Well truthfully, most IT and security people live in fear and trepidation of that day when they have a true positive critical, and they have to engage in this process to like, I'm not ready. I'm not ready. And I'm here to tell you, setting those folk up for success so if and when that day comes, and their internal response is, I know what to do.
Patterson Cake:I know what to do. Hugely beneficial. Hugely beneficial. Be purposeful about this. Get some training for your staff.
Patterson Cake:It does not have to be three weeks of training, it just needs to be here's a plan, here's an approach, here's a little practice, and then do this on a recurrent basis so that your team is set up for success, and this will pay monumental dividends. Who loves insurance? Who loves insurance? I mean the whole idea of insurance really, I mean insurance companies wouldn't exist, right, if the odds weren't in their favor. So this is painful.
Patterson Cake:Nobody likes an insurance policy and honestly in this particular case I'm not talking about cyber security insurance, I'm talking more generally about things that you can do to protect yourself and I worked a it's been a couple years back, we're getting better at this, but I worked a case. Case a couple years back with a ransomware outbreak and the After initial access, one of the first things that the threat actors do after sort of some general enumeration, where am I, what kind of access did I just gain, what type of organization is this, the next things of course in their mind are actions on objectives, and almost always one of those things is to find your backups and to destroy them. That is often sort of priority one or two, because they know if you got great backups then you're not too worried about ransomware outbreaks, so to speak. You need the insurance of immutable backups. This is no longer a, you know, wouldn't be a bad idea, we probably should have that.
Patterson Cake:This is your due diligence absolutely and completely, and if you're unfamiliar with the idea of immutable backups, it just means effectively backups that cannot be changed. We need our backups to be impervious to the threat actors getting at them, destroying them, encrypting them, etcetera. I mentioned, I started talking a minute ago about a case that I'd worked, and the first thing that they did was find the on disk backups and nuke them, and that organization, long standing publishing company, newspapers, magazines all over the Western United States, and they had to rebuild all of their information, all of their network data network data through email. Literally, they had to go all their staff and their customers and say, do you have that image template or that proof or draft in your email inbox rebuilding an org, a three generation org from email. Nut.
Patterson Cake:Don't let that happen. You know backups are important, I'm preaching to the choir, I know I am, but this is just again, your due diligence. Immutable backups, backups that are impervious as much as possible to threat actor engagement activity and or other disasters, and of course, for crying out loud, if you implement it, you need to test it, you need to make sure. You need to prove out these theories. How well does restoration actually occur?
Patterson Cake:I am so excited. I am so thrilled when I roll into an org and one of the questions I have after do you have cybersecurity insurance is let's talk about your Doctor solutions. What are you doing to back up your critical system server data? And it is so refreshing when someone says, no worry, we have this in place, and this in place, and this in place, and we are prepared to restore from bare metal in a reasonable meantime to recovery, and oh, by the way, we have two or three weeks worth of retention so we can go back far enough to known good. Monumental.
Patterson Cake:This is the insurance I'm talking about in the previous slide. Imagine the opportunity, imagine the opportunity in the face of crisis to push a button effectively and go back to known good configuration on your most critical systems. That's what I'm talking about. That's what we need desperately yet again. I've worked I've worked three cases that I'm aware of specific Oh boy, that's not true.
Patterson Cake:I think it's six if I include a different category. I've worked three to six cases in the last year or so that were all the result of firewall misconfigurations. Firewall misconfigurations. Two of them were we changed brands. We went from Cisco to Palo Alto or whatever, it doesn't really matter, and we migrated our rule set.
Patterson Cake:We migrated our rule set to our perimeter device, our edge security, and then footnote, we didn't really test it, we just assumed it would be fine. Excuse me? Migrating firewall rules, upgrading firewalls, making changes, I'm here to tell you that in three different cases we were able to walk in the door before ever doing any investigation and tell them what happened, and they're like floored for just a minute, and we used these crazy things like open source intelligence. We did a crazy thing like a Shodan search and discovered they had RMP opened the universe. We roll in and say, let me tell you what happened, and they're like, what?
Patterson Cake:For crying out loud, all of your internet facing infrastructure, you have zero margin for error. Zero margin for error. In my humble opinion, this is your change management priority number one. Change management priority number one. Any internet facing services you need to be paying careful attention.
Patterson Cake:You need to be doing constant monitoring and testing. Attack surface management's kind of this kind of a relatively new service offering, and whether you pay someone to do this or you engage with an ASM service or you do this on your own internally, well at a very very basic minimum, the baseline, any time you make firewall configuration changes, have somebody test it, right? How hard is that? And in many instances if that goes undone, unnoticed, if you publish a three thousand three and eighty nine to the internet or a 22 to the internet, and then if you have any, any vulnerabilities or any poorly managed accounts or passwords, you're just you're just owned. And we're literally talking about a few hours often from unintentional exposure, internet facing, to a really, really, really bad day for the organization.
Patterson Cake:I think it's important, although this is not specifically addressing it, it is an internet facing component obviously, that's your cloud infrastructure. Firewalls are just a given, That's an absolute no brainer, CMD priority one. Change management database, have a process in place, even if you don't have a CMDB, make one. Make one that includes nothing other than your internet facing systems and services and what I mean by that is document a change, schedule a change, schedule validation of the change and no additional internet exposure, roll into production, happy day, major major win. Same thing on Cloud Impreh.
Patterson Cake:This is a tough one and we see this all the time. Cloud Impreh is still newish, right? It's still newer than oh, on prem active directory for example. And so one more time we have zero margin for error. You deploy a new system service feature function in the cloud, it is incumbent upon you to do some examination, some attack surface testing monitoring to make sure that you didn't just expose the organization, data, services, identity, etcetera.
Patterson Cake:CMDB priority number one. I said earlier that one of the best parts about active incident response is that lessons learned moment, and it is honestly, but I'll tell you in some ways the most gratifying component of active incident response is crushing the adversary, is rolling into a system, a situation, a circumstance and saying I know what to do and if we do this and this and this we will win the day. We can contain, we can eradicate, we can expediently recover systems, services, and away we go. That is truthfully the greatest joy in my life as an active incident responder. What do we do?
Patterson Cake:How do we do this? And the fact is, it's not overly complicated. It is not overly complicated. Remember what I said earlier, that I come from a bit of a unique perspective, right? I am an incident responder.
Patterson Cake:I tend towards being a breacher, meaning I'm gonna kick down the door, there's gonna be some collateral damage, don't really care, nobody else cares. You don't roll into the office every single day and kick down the door, or you'll definitely need to be revisiting Jason Blanchard's video series on how to get a different job. Two things, your two highest priorities from my perspective are egress and off, egress and off, egress and off. Now frankly it should probably be ingress, egress, authentication. Consider for just a moment, consider for just a moment how often does a typical attack How often does a typical breach involve someone physically coming to your facility?
Patterson Cake:Okay, that does happen, right, but it is very very rare. Anymore, the attack vector is 99.24% of the time the internet. And point in fact, we can evict the threat actor, completely evict the threat actor from the battlefield with one playbook, and that is kill ingress egress. Kill the internet. Deny all the things.
Patterson Cake:And it blows me away that in many instances, the organization rebels against that. They're like, well that'll disrupt that'll disrupt normal business. I'm like, so will ransomware. Fact, we're way beyond that. Don't you dare kick down the door.
Patterson Cake:Are you kidding me? Disconnect the internet, deny everything. This is brute force, this is blunt, right? We don't have to do that. We can actually plan ahead.
Patterson Cake:If we did not plan ahead then it's time to kick down the freaking door. Fortunately you're listening to this conversation and if you don't have this in place now you can at least consider it in just a few minutes. The second piece is everything we care about, and I mean this almost unequivocally, everything we care about should be protected behind identity. There are a few things in your environment that maybe don't require authenticated access, that are just public or open and accessible, almost everything we care about is hidden behind an identity, So this gives us a second tremendous capability to defang the threat actor by the rotation of all passwords, and really I'm talking about two pieces there. Revoke all authenticated sessions, rotate all credentials, and we do these two things, we just bought ourselves some time.
Patterson Cake:We're not done, right? We're not done, but point in fact, we just set ourselves up for success. Now we could take as long as we need to investigate the scope of engagement, scope of impact, take other remedial actions, eradicate the threat, we wanna do that quickly, but if we do these two things then we go okay, it's been twenty two hours, you can go home and take a nap, we're okay for a little bit, we just stopped the bleeding, we revoked all authenticated sessions, we've killed the internet and away we go. This doesn't have to be entirely blunt force. We don't have to kick down the door so to speak.
Patterson Cake:In advance of crisis you can and you should, please oh please, consider at least the granularity of capabilities you have for containment through internet connectivity, through management of egress. For some of us, we have multiple points of presence. For some of us, we have segregated business environments. For some of us, we have distinct physical locations or business units or whatever. So have a plan, have a granular plan.
Patterson Cake:Do we need to disconnect all internet for HQ, for data center, for our San Francisco location, for our Chicago location, whatever? In conjunction with that, consider defining an allow list in advance. How many how many actual business critical websites do you need, do you use on a daily? I'm not talking about Facebook, I'm not talking about any number of other potential websites, I'm talking about to actually accomplish your core features and functions, and chances are it's 10 or 20 or something pretty finite. Business critical internet accessibility, and now, right now, define that allow list in advance, and then you can kill ingress, egress, you can kill command and control to the threat actor, and still allow business critical functions should you need to.
Patterson Cake:Huge win, hugely beneficial, and I see very few organizations do this, very few. So please consider that, consider developing that, consider documenting that, test that, define please, define the authority to engage this particular playbook in advance of need, you'll be poised again to literally evict the threat actor from your environment with one move. The second piece is a fun one. Change all passwords. Change all passwords, and not long ago we were called to engage with an organization, they'd already engaged with one incident response team.
Patterson Cake:It's always pretty concerning when you get a call and say, well, we called such and such a team, and they really couldn't help us, and you're like, okay, I've heard of them and they're pretty good, and why are you calling me now? And asked some very careful questions, and in this particular instance, it was a globally distributed organization, construction related, and they had distributed independent infrastructure all over the planet, and the original threat actor, the original containment and eradication had just failed. They'd found the threat actor, they'd found his behaviors, they'd implemented some containment strategies, and they could not get rid of him. And so one of my first questions is have you rotated passwords? Have you rotated passwords?
Patterson Cake:And they said yes, and I said well which ones? And they said well all of them. I said all of them? They said, well yeah, absolutely everything that is AD integrated. Okay okay, let's talk about that.
Patterson Cake:Absolutely everything that was active directory integrated, well what else is there? And like, we have lots of other stuff and specifically we have these Linux systems all over the environment. We have network attached storage all over the environment, and so what do you think that threat actor did? The threat actor pivoted to all of their Linux underlying operating systems for their network attached storage, and they had no way to centrally manage those credentials. So effectively, we kicked the threat actor out of active directory integrated authentication, and then they moved elsewhere, whether it was printers, or network attached storage or network devices.
Patterson Cake:This is what I'm talking about when I say change all passwords. You should start, from my perspective and humble opinion, start with your most critical components, and by that I mean the obvious ones, if you're using after directory like everybody else, DomainAdmin, Enterprise admin, etcetera, etcetera, but then consider the other components, the non AD integrated. Are you using different credentials to access your hypervisor or or or your business critical applications, etcetera, etcetera? So think, please, think this through. Be prepared with these two huge hammers, huge responsive hammers.
Patterson Cake:These are in your playbook, these belong in your playbook, you should have a plan, and by a plan I mean tactical components, whether it's scripted or something along those lines, disrupting ingress egress, pill threat actor authentication, and we are poised again for success. Not the end of the conversation, but a successful beginning to the conversation. I feel like I've been preaching at you for forty five minutes, so forgive me, I'm passionate about this stuff, and I think the vast majority of these things are things you could do, I mean, legit, like next week, or or maybe the week after. And almost all of them are effectively free, other than some internal conversation, time, energy and effort and to recap, to recap and then I'll stop preaching at you, the 10 things we touched on today, and these are not really in prioritized order, so to speak, because I think they're all honestly important. Please, oh, please remember the frame of reference of the art of the possible.
Patterson Cake:If some of these things were hard for your org, if you're like number three just isn't gonna happen, or number eight is just gonna require a committee and meeting and Then go for the other ones. Cherry pick the easiest things to accomplish and work your way through this. Maybe you've done several already, I hope so. Number one was create a succinct tactical IR plan. I will share another shortened link in just a minute on the next slide so you can reference at least a potential document.
Patterson Cake:As mentioned earlier in the chat, this is a pretty slick resource. We spent a lot of time and effort into making this sort of an IR simplistic tactical approach, so huge, huge call out there. We'll reshare that link. Engage with your cybersecurity provider, cybersecurity insurance provider now. Find out whether that's a workable relationship or not.
Patterson Cake:Make sure you understand when you're supposed to call them, how you're supposed to contact them, if they are actively going to partner with you in your best interest as an organization, which service providers you're allowed to use, what does your policy cover? A lot of policies Imagine you wrecked your car, right, and it was white before and you've really always wanted it to be red. And in the middle of fixing your car, could we just go ahead and paint it red? And your insurance company will say, no, your policy covers restoration to pre accident state. Pre accident state.
Patterson Cake:What that means is that technically, in many instances, your cyber security policy will not help you improve your security posture, that will not be covered necessarily, so answer those questions now, please. Log retention, just work it out, walk it through. It ties in nicely to some of the other pieces, the asset inventory that's coming up as the next bullet point. Now start with priorities. Identity should be your priority.
Patterson Cake:Yep, it absolutely should. You need active directory visibility, you need infra ID visibility, and you need them like now. Prioritize those things, obviously you could be using other identity solutions, but that's the most, those are the most common. Make sure you understand what those are, make sure you document, make sure you have corporate approval and acceptance of those retention components, and if you don't then you're gonna probably have to spend some money. That one is one that may cost you a little.
Patterson Cake:Number four, again, asset restoration and recovery priorities. You you talk about asset management and you know, you know, it used to be the CIS top one and two were your hardware and application inventory, and we've known this forever. It's not a mystery. It's hard sometimes. It is a huge lift in the overarching sprawling enterprise.
Patterson Cake:I'm not talking about that. I'm talking about tier zero, tier one. And I'm literally thinking, are your top 10? What are your top 10? Maybe your top five, part of the possible, start with your top three and then start working your way outward so that you have a recovery and restoration prioritization list.
Patterson Cake:Emergency comms, out of band comms, again, could be free, could be free. Very low cost. Shouldn't be a huge lift, shouldn't be a huge effort, but please don't implement this and then forget about it, or guess what, everybody else will forget about it too. Staff rotation policies, again, this doesn't have to be complicated. It just needs to be realistic.
Patterson Cake:Just need to be realistic. Maybe you don't even need to augment your staff. If you implement playbook one and playbook two and you buy yourself some time, you're confident the threat actor is no longer actively engaged in your environment, then maybe two or three security people can manage this incident response process because you're smart enough to go, well, you know, you can go home and sleep for four hours, it's gonna be okay, or you rotate those individuals or whatever. Think that one through, get corporate buy in. Oh, by the way, the incident responders will be the least compliant.
Patterson Cake:You're probably gonna have to pry them out of their chair and force them to go home. It's just the way our hearts and minds work. The IT people, the security people, they're gonna be like, I'm not leaving until we're done here, and that's when somebody needs to step in and say, yes, yes you are. You need to go home. Give those folks some training, just even some basic training about how to respond.
Patterson Cake:No joke again, not to beat the proverbial dead horse, but just something that's like this. This has basic tactical response components literally start to finish. I created the overarching outline with lots of help and support and input, but I started this process with, okay, what would I want to know if I were a small to medium business incident response team? This is a great place to start and it's free. Thank you, Black Hills.
Patterson Cake:Backups, backups, backups, not just backups though, immutable backups or backups that are impervious to attack, so they're there when you're sure you need them. Change management for all internet facing systems and make sure you're paying attention to your internet facing attack surface like always, all the time. You do have a tiny delta, but it's a very tiny delta if you make a mistake there. Last and definitely not least again was playbook one and playbook two which was create a granular response plan for disruption of command and control otherwise put as kill the internet, please. Oh by the way, when you kill the internet, test that it actually worked.
Patterson Cake:More than once in active IR we've killed the internet and left DNS fully functional for some unknown reason. Well DNS can still be an attack vector. Command and control over DNS is a thing, so when you're doing this process, test, validate, etcetera, and then the auth playbooks again, kill all authenticated sessions, rotate all passwords, and if you do that, you do those two things, you literally just kicked the threat actor out of your environment and bought yourself some time. One through 10, part of the possible. Thank you so much for your time and attention today.
Patterson Cake:Couple Boy, this is just no fun to look at yourself on a slide, is it? Just ignore the picture. I am doing a class next month where we actually step through our tactical response process from start to finish in an eight hour incident response simplified approach. Couple shortened links for you. The first one is related to this conversation.
Patterson Cake:I have an incident response crisis communications template that I would love for you to leverage, and then a basic just word doc tactical incident response plan, and then the second one is my overarching tactical workflow that honestly we use in many of our production investigations. You know me, I love to talk about this stuff. I'd love to help you in any way I can, let's be friends. Obviously, professionally we provide all sorts of incident response services and capabilities, but on a sort of more personal level, I love to share the information. I love that I work for an organization that allows me to stand up and talk to you about all this stuff, to actually share Rapid Triage workflow, which is our technical process.
Patterson Cake:So again, please hit me up if there's anything I can do for you, if you have questions, etcetera. And I think I think I'm done. We got a few extra minutes for questions and stuff.
Jason Blanchard:Well done, Patterson.
Patterson Cake:Thank you.
Jason Blanchard:Patterson, I have a quick first, I'm gonna like, thank you for sharing your knowledge today. The for SOC Summit's coming up next week, you're speaking at it. If you haven't signed up yet for SOC Summit, there's a bunch of links in the Zoom chat right now for you to get free resources to take Patterson's class, to just sign up for the SOC Summit. And for those of you that have taken us up on the offer for the free InfoSec Survival Guide, there's a 100 of you so far. Good.
Jason Blanchard:So a 100 of you have trusted us enough so far to place an order at the Spearfish General Store that we'll mail it to you. So thank you so much for trusting us to do that, and we are gonna mail it to you. For those of you that have already received it, just you know, let everyone else know that it's legit, it's real, we'll send it to you. There's no strings attached, we're not like gonna hound you or anything. Okay.
Jason Blanchard:So Patterson, my first question for you is how how does running so many tabletop exercises, because you do, you run a ton of tabletop exercises, and if anyone wants to hire us for tabletops, you totally can. But running so many tabletops, how does it prepare you to do the work that you do as an incident response person?
Patterson Cake:Wow, that's a great question. I think a couple different things come to mind immediately. I mentioned it a couple times in today's presentation. The ability to communicate and collaborate as a team is one of the most critical, maybe the most critical feature function of successful incident response. So learning how to work with different audiences, with technical folks, with business leadership, learning how to coordinate and extract knowledge and get engagement, that I would think for me, probably one of the best most significant takeaways from that common interaction, you know, in fictitious scenarios.
Jason Blanchard:Sure. And once again, if you do ever want us to help facilitate Run Your Table Top exercise, because of the amount of opportunities we get to see so many different organizations, and see how they do what they do, and find practices across all these different companies, and then bring them to you, you can always reach out to us for tabletops. I used to do them. I'm not great at being billable. I'm terrible at being billable, but Patterson is not.
Jason Blanchard:Patterson is fantastic. So definitely Patterson can do that for you. Alright. So I had a question here from from the from an attendee. It was, which is more important to track or block, egress or ingress?
Jason Blanchard:And could you explain egress and ingress to me who does not quite understand?
Patterson Cake:I That's a fantastic question, and legit as I was using those words I thought, I wonder if anybody knows what I'm talking about, and then I moved on. So full disclosure, my first career was building houses for a living, and so egress was hugely critical when you're building a house, have to have adequate way to exit a room, and that is egress, outbound. Ingress is inbound. And so I pinpoint egress because we're already usually paying attention to inbound. We're blocking most things inbound.
Patterson Cake:That's sort of our We learned that a long time ago, you should have a firewall, you should stop bad things coming in from the internet and so we block all the things generally. Many enterprises still allow all the outbound stuff. And so truthfully in active IR I'm more concerned with the outbound piece usually than I am the inbound. Full disclosure again, those are An ingress egress is often just a session, they're really distinct. It's just really one thing.
Patterson Cake:It's like having a conversation, know, Jason and I are having ingress and egress. So in some ways it's just kill the internet, please. But technically speaking, egress is more important to me in that moment.
Jason Blanchard:Yeah. Speaking of talking to each other, after the webcast is over, if you wanna stick around, I know one's liked to do business with Black Hills, feel free to stick around. We're gonna talk about incident response retainers, we'll talk about what it's like to do business with us, and that's after the webcast is over. Yeah. Hey, last question, Patterson, then we'll get your final thoughts.
Jason Blanchard:Might do some Q and A if we still have some more.
Deb Wigley:There's a couple.
Jason Blanchard:Yeah. And then don't forget to check-in for Hackett. Alright. For your pre incident checklist, would you add prepping your pre built Velociraptor image capture to get forensic images of key machines? That's a great question.
Patterson Cake:And of course, yeah. There's a whole another discussion about pre staging of visibility capabilities, to be honest. And it would probably be like number eleven and twelve or 13 or something like that. Those are more typical preparedness and responsive components. I'm a huge fan of pre staging your visibility capabilities, and rapid triage is one way to do that, so that would definitely go on the list, so to speak, but probably a little further down in terms of active IR.
Patterson Cake:Okay.
Jason Blanchard:Alright everybody, I'm gonna ask Patterson for his final thoughts, then we're gonna finish the official webcast, and then we'll stick around for some additional Q and A, and then what I'd like to do business with Black Hills. Thank you so much for joining us. Patterson, if you could sum up everything today in one final thought, and you've done that multiple times, but here's the last chance to do it, what would it be?
Patterson Cake:Maybe I'll get it right this time. No. The the again, my my thought is I hope that you can take one or more of these lessons learned and apply them to your org and improve your security posture, the end. Choose one, choose the easiest one, work on the second one, and make tangible progress, and then you will make me very, very happy inside.
Jason Blanchard:Awesome. Thank you everyone for joining us today, and don't forget Backdoors and Breaches is a game that we developed to help you with tabletop exercises, and it is useful to do it over and over and over and over and over again for like thirty to forty five minutes with your team once a week, every two weeks, once a month. It will help you with that iterative process of improvement. Alright, everybody. That is it for the webcast.
Jason Blanchard:Okay. Alright. So the webcast is officially over. Well done, Patterson.
Deb Wigley:Good job, Patterson.
Jason Blanchard:Thank you. But we do have a couple more questions, so Questions. Yeah. Yeah. Deb, go ahead now.
Deb Wigley:Mom, mom, Cake. Mom, didn't forget you. We there's two lengthy questions, which I think are great. The first question is, what would you recommend is the threshold of education knowledge professionals need in order to help companies with disaster planning slash tabletop exercises? Well, what?
Bryan Strand:It'll be his training coming up here.
Deb Wigley:Yeah. He's training.
Patterson Cake:Fantastic. That's a fantastic loaded question. Gosh. Honestly, I think we talked about the building blocks for that answer today. I think bare minimum, and it is pretty bare minimum frankly, but the things we talked about today in terms of your asset prioritization and and really understanding the nature of what's important to the business from an IT perspective and then managing the interactions with the people who know the technical pieces and parts to ensure those things are safe, ensure adequate visibility into those things, ensure mean time to recovery for those things and then stop.
Patterson Cake:Like just do that and you're leaps and bounds ahead of most organizations. You know, you can keep going from there, obviously, and and honestly, but great place to start from my perspective.
Deb Wigley:And then the second one is great great answer, by the way. It's a two parter. Does it make sense for IR and offensive security services such as pen testing red teaming to integrate work together to give clients great service? If so, what is the best way for these two services to work together?
Patterson Cake:Yo. What a great question. That's fantastic. I'll tell you, the most fun that I have ever had in providing IR consulting services is active red team engagement with the incident responders. Tremendous synergy, it's like magical, where you have a trusted service provider or even internal resources and partner with a service provider and you act out an active threat while the incident responders are observing and learning and this cooperation between the two where you're actually testing detection and response and there is no better way, in my humble opinion, to accomplish that comfort level and sort of to test the alerting detection response in that safe sort of environment.
Patterson Cake:So I think that's a fantastic way to approach that. You know, we talk about purple teaming, and I think to me that's what that that is, or or should be at least.
Deb Wigley:Yeah. Thank you, sir.
Jason Blanchard:So one the things I'm noticing is there's a ton of you that are registering and ordering the Incident Response Survival Guide. And most likely that means this might be your first time here. So if this is your first time here, hello and welcome. Thank you so much for joining us today. We're Black Hills Information Security.
Jason Blanchard:We do this every single week, pretty much every single week, where we have free educational content that we share as much as we. So the the philosophy of our company is give away everything you possibly can, because we have enough. Right? Like there's enough to go around to give away everything we possibly can. So you're here as a part of that.
Jason Blanchard:We don't really have a marketing department. We have a, hey. Let's go share our knowledge department, and then that way you get to know who we are. Deb, is there other questions?
Deb Wigley:If we have missed any of your questions, go ahead and put them in the Discord or in Zoom, and we will ask them. I don't see anything.
Jason Blanchard:A new one just popped up. So Patterson, when it comes to incident response tabletops, should the ITIS side should it be the ITIS side only, or should should that involve other areas?
Patterson Cake:I'm a huge fan of bifurcated tabletops. And I actually There there's a side note in my presentation which I failed to hammer on, and that is that you absolutely, in my humble opinion, should have technical teams separated from business leadership teams in active IR, and so I push for that in tabletops. I want everybody involved, but I want them involved in their right sort of purview, and then I'm a huge fan of a communications liaison for a specific role that coordinates and communicates between the technical teams and the business leadership teams because that is a monumentally important component of IR. Tabletop bifurcation for sure, involve anybody and everybody, but let's talk about it. Let's talk about who they should be engaged with and interacting with, and and maybe do more than one separate exercise.
Jason Blanchard:Yeah, I got a chance to do a tabletop for C Suite, and like their senior technical staff, and it took about twenty minutes for the C Suite to get into it. And then once they did, there was a lot of like, oh, but I made the scenario very, it was a spear phishing attack from a C Suite person to a junior member, and the attack was essentially the junior member used the email account, because it was left unlocked, of the senior member to send it to the junior member to get invoices paid. And so it was an insider threat, and so once they were like, oh, an insider threat? I remember the quote was, I never thought the attack would come from inside. And I was like, well, sorry.
Deb Wigley:Now you will. Perfect. Alright. I have one one question. This is gonna be fun.
Deb Wigley:Patterson, what would you say your top 10 list from the slides is in order of importance?
Patterson Cake:Oh, boy. Now that I've closed my slide deck.
Deb Wigley:Yeah. Mhmm.
Patterson Cake:From memory, unless you
Deb Wigley:show me.
Patterson Cake:If I was gonna do one thing tomorrow, it would be the last one, the playbooks. The ingress, egress, and auth playbooks, because those are the most emergent need, truthfully. You want those in place today. The the second one would be probably the change management for internet facing systems. One more time, that's that's crisis, that's if we fix that one, if we avoid that problem, then we got a little breathing room, a little more time.
Patterson Cake:Yeah, the remainder I think as mentioned at the beginning of this conversation, you're gonna need to prioritize on your own based on what you can do for your environment. I could come up with a list, but ultimately, which which of those can you get done in a very finite time range? And I'm like, literally, let's say less than thirty days, and then begin to prioritize those based on your org, your structure, your capabilities.
Deb Wigley:So I misunderstood the question. He actually asked if it was already in order of importance. Oh, that would have been easier. Right.
Patterson Cake:And the answer is yeah. The answer is is no. I think it's actually almost reverse order. But as I just labored on about, truthfully my thought is I'm gonna give these to you. I don't know how to define those priorities for your org, and I think hopefully as a result of this conversation, you have an idea how to approach that conversation and sort of prioritize them yourself.
Patterson Cake:I stand by the my original response, however, that the last two I think are probably the highest priorities.
Jason Blanchard:Mhmm. So I'm gonna ask this next question about tabletop exercises, and I want that to lead into Bryan talking about what it's like to do business with us. Alright. So before I move forward, if you wanna hire us, and Patterson specifically, to run your tabletop exercise, you can do that. You can reach out to blackhillsinfosec.com, go to our contact us form page, and say we'd like a tabletop exercise, and Patterson will help run that whole thing and support it, and it's fantastic.
Jason Blanchard:We also have Hal, so if Patterson's busy on an actual IR engagement, you might get Hal, but you're gonna get one of us. And so this question is, how do you get buy in to perform the cross functional exercise, right? Like, so you have a tabletop exercise, you have different teams, and so I'm dealing with that right now where I can't get buy in outside of IT, and they currently work in higher ed. And so there's a part of me that's like, well, if you work in higher ed, it's about educational, and like people learning stuff, like, why wouldn't people not be bought into learning something that they don't know? Anyway, Patterson, what are your thoughts?
Patterson Cake:I think it's unusually common, so don't don't feel like this is a unique situation to you and your environment, honestly. What what I often do is I, again, I intentionally bifurcate. And one of the ways I do that sort of subtly is I say, okay, we're gonna have a we're gonna have a two to three hour technical exercise, and immediately the executive team is like, we're out. And that's good. That's what I want.
Patterson Cake:And then we have this opportunity as a tactical team to spend a couple hours, and then our goal is a refined output so that we can spend thirty, forty five minutes, an hour with the executive team and make it extremely useful, extremely beneficial. You know, just jam packed with things they care about. And sometimes I'll trick them honestly. We'll do the tactical table pop, and then as mentioned in my deck, have a crisis communications template, and I will use that as a hey, you know if we ever have an emergency, I would like to review with you how we communicate that information, and how you want to receive that information, and make sure we're getting you all the things you need to make business decisions, and sort of gently pivot to It's not really a tabletop even. It's just a communications template.
Patterson Cake:It's just something bad happened, here's what you need to know, and we need your review and approval on that. That has worked pretty well, and in many instances it will then actually mutate into, well let's do an actual exercise, and let's spend a little bit more time. So that's at least a possibility.
Jason Blanchard:Alright. So Brian, if somebody did wanna do business with us, what's step one?
Bryan Strand:Question your sanity. No, just kidding. Step one, I actually have it saved here. I was actually kind of ready for it this time, you asked me this a lot.
Deb Wigley:So
Bryan Strand:I actually think the best way is to contact us form on our website. I think I think that's probably the best way you get you get because you're gonna get an insanely fast response from either Melissa or Nora if you fill that out within, like, an hour at at and that's me being very generous.
Deb Wigley:Yeah. They're super
Bryan Strand:They're they're insanely fast. It's almost like they have nothing else to do for the day. But I know they do. I know that's not true. My sister works very hard.
Bryan Strand:But yet, I honestly I feel like the contact us form is the best. You can email consulting at Black Hills Info Sec. You can reach out to me on LinkedIn, you can reach out to me on Discord, but probably the best way is that contact us form, honestly.
Jason Blanchard:Alright. So they fill out the contact us form, they get a response back, they attend the meeting, and then after that, how much do you follow-up and like hound them to sign the contracts that they didn't even agree to?
Bryan Strand:Right. So we get the initial quote, which is another thing that's stupid fast, because we don't do commissions, and like I've heard this from other companies, like, I'll get it to you by well, Nora get it to you by end of day, they're like, woah. That's really fast. You guys must not be that busy. It's like, no, we just we don't work off commissions, we don't need 15 people touching it so they can get their little slice of the pie.
Bryan Strand:And so once she sends that out, we it's typically like two weeks, and then we just follow-up and just say, hey, do you have any questions? And if you're like, no, we're good, we're we're just blah blah blah blah, we're trying to figure this out, great. If it's a net I think we wait another two weeks if we don't hear anything, and then it's a month, and then I think it's like two months, and then we kill it, or something like Melissa knows, like there's a specific formula that we've that we've worked on that's not pushy at all, and and so, yeah, so we literally have, in our CRM, it's just like, hey, email this person. So we email them, hey, just curious if you have any questions. But no, we're not pushing for signatures or anything like that.
Bryan Strand:We're always we're always trying to be thankful for the fact that the people are willing to reach out to us to begin with, then also willing to get on another call to go over anything that they need.
Jason Blanchard:Okay. So let's say they contact us about doing a tabletop exercise, or incident response retainer. Alright? Because we do have incident response retainers. Mhmm.
Jason Blanchard:And one of the reasons that we do this is because people are like, okay, so you do webcasts, do comic books, you do guides, you do these things, what else do you do? And you're like, well we we do do pen testing. Yeah. We do do incident response, we have a training organization. So sometimes people don't know.
Jason Blanchard:And and we've heard from people like, why didn't you tell us you have a sock? We would've hired you for our sock. Yeah. Oh, okay, well we have a sock. It's like, why did you sign a three year contract?
Jason Blanchard:And you're like, well, see you in three years.
Patterson Cake:So
Jason Blanchard:Brian, we have incident response services. So if someone contacts us and says, we wanna do a tabletop incident response, do they go to you? Do they go to Patterson? Who do they go to?
Bryan Strand:Well first off, I want everybody to know I am not gonna make any jokes about the doo doos that he threw out I'm above those kind of things. But second of all, honestly, yeah, you probably talked to Tom, Logan, or myself, and we more than likely will probably bring on Patterson to talk with you as well. Yeah. Yeah. So we we also are not afraid to have the technical people on our sales calls, so to speak.
Bryan Strand:I was actually at an ISC squared conference like a decade ago, and I had some young kid who was a sales guy for a very large company, was competitive of ours, He's right next to us in a booth, and he's like, oh, Black Hills, I've heard so much about you guys, blah blah blah blah blah, I wanna talk, you know. He's like, hey, I got a question, how do you scope web apps? And so I started talking about how we do that, and was like, and then at the end of the day, if it's really complicated, we'll get we'll get a pen tester on there. And he's like, you let your testers talk to customers? Like that was the look on his face, he was like, ew, that's gross.
Bryan Strand:Why would you let them talk to customers? And so we're actually really big on that. So yeah, Patterson might be involved in those conversations, and just because we we wanna make sure because what I've learned in a lot of times, and Patterson, back me up on this, when we have people that call us up, and sometimes it's like hair on fire IR, sometimes they just need someone to talk to for like a half hour to forty five minutes, to just let them know they're either, hey, you're doing the right thing. You're doing a good job, buddy. Keep it up.
Bryan Strand:Or try these couple of extra things. And I would say a a good number of customers that reach out to us that wanna do IR stuff that are in the middle of things, they end up actually really not even they end up not even like going with it, just because of like the half hour, forty five minutes, an hour long call that Patterson has got on with these guys. Just because sometimes they don't need it.
Jason Blanchard:Okay. Yeah. So a question from the audience is, Patterson and Bryan, I'm gonna take this. Let's say it's a tabletop exercise, how far out would we need to schedule this to get your tabletop?
Patterson Cake:Great question. Like everything else, depends a little bit on the nature of the tabletop and the complexity thereof, but usually it's harder to schedule your team than it is to schedule us. So we usually say about four weeks out just as a general timeline. If it needs to be done faster than that, sometimes we can. But four, you know, four to six weeks is plenty of roadmap.
Patterson Cake:Now again, fair warning, if you're engaging with the executive team, that's not nearly long enough to get on their their calendar. It's not it's not me. That's probably not you, but in those instances, probably better to have sixty days, maybe even ninety days.
Jason Blanchard:Okay. And then Patterson, like growing up you'd always hear about like the lawyer retainer. I don't know, but like rich people had a lawyer on retainer. It was always on TV shows stuff like that, it's like, well I have a lawyer on retainer. And and I didn't quite understand that until, you know, you get a little older and you're like, oh, so you just have like a lawyer that you can contact, and and you've given them money ahead of time?
Jason Blanchard:Is that how that works? So sometimes people reach out to us and they want an incident response retainer. What is an incident response retainer and how does someone get it?
Patterson Cake:Well again, that's kind of a loaded question. An incident response retainer is is generally speaking a pre engagement with an incident response service provider like an insurance policy. So you're anticipating you might have a fire someday, so you engage with the fire department. If there's an emergency you can call them and they'll roll in with fire hoses. That's a typical incident response retainer.
Patterson Cake:I hate those kind of retainers, incidentally, and so once again, being who we are at Black Hills, we've we've tried to turn it around a little bit and we provide a retainer, forgive me, it's lengthy because I can't help myself, not very good at being succinct. We call it an incident preparedness and event handling services retainer. And we call it that very intentionally because we we don't wanna be your fire department, we want to be engaged with you on an ongoing basis, well frankly to talk about things like we just talked about today, like things that we can do, we can partner with you to be better prepared for an incident, we spend time getting to know you throughout the entirety of your contractual engagement. If you need a fire department, we can also do that. But you don't wanna have a fire, we don't want you to have a fire, and so anything we can do in advance, we will do to help you avoid that.
Patterson Cake:And then again, if you do have a fire, we're there, we're ready, we know you, we're actively engaged, you don't have to go engage in that contractual process, etcetera, etcetera, etcetera. Happy day. If you do need to draw down on your hours, we let you do it in very very small increments. You can engage with us for fifteen minutes if you need to. A lot of retainers don't let you do that, a lot of retainers will require you to engage for a twenty to forty hour engagement just to ask a question, that's insane from our perspective.
Patterson Cake:And then especially if we achieve our mutual goal of avoiding any kind of active incident throughout the year, then we partner with you to reuse some of the retainer hours to do something else proactive. We love to then invest time in helping you develop your tactical incident response plan or help you train your staff or or or Boy, this sounds like a lengthy sales pitch, but the fact is we want to be your business partner, we wanna be an extension of your internal teams if you don't have an incident response capability. That's our goal, that's my hope.
Jason Blanchard:Okay, so I got two questions from the audience. Also to the 200 of you sticking around right now, we're nineteen minutes into, you know, post show, and you guys have great questions, and so thanks for sticking around, thanks for being here, thanks for wanting to listen to this.
Patterson Cake:It
Jason Blanchard:says, do do the law firms engage with us, like, because right, because there's like, Yeah. You know, during an incident, some things could be discovered that could be used in a lawsuit for that company. Yep. So how does the law firm, and incident, and retainers, and privilege, and all those things work together?
Patterson Cake:That's a fantastic question, and it is something that you should be thinking about, so bravo, because there are considerations, and and if you have cyber security insurance, then they have requirements for you. They have requirements about the service providers that you use, they have probably breach coaches or specific legal resources, so you're asking the right questions. For our part, we will absolutely partner with your insurance company, with your legal counsel. At the end of the day, we really, really, really want to work for you, however. We don't really want to work for the cybersecurity companies, insurance companies.
Patterson Cake:We want to work directly for you as your representative and often that means being a liaison with those providers, which we're absolutely comfortable with. We can engage in tri party agreements, which is a fancy way of saying that we can transition to actually working for your legal advisor in order to implement that privileged and confidential sort of communication thread. So sorry, that's a little complicated there, but we would be happy to talk with you about that, happy to work with your existing resources, and sort of map that out.
Jason Blanchard:Okay. Do we work directly with clients, or can we work with an MSP for, on behalf of their clients?
Patterson Cake:Yes.
Jason Blanchard:Okay. So if you're an MSP and you want us to work on behalf of your client
Patterson Cake:We work with MSPs all the time. We work directly with clients all the time. The yeah. Okay. We're pretty flexible.
Jason Blanchard:Another one is, what about CMMC when it comes to defense contractors? Are we only for commercial, or can we work with defense?
Patterson Cake:That would be sort of a longer overarching conversation. Generally we don't engage frequently with defense. Could we put that together? We should talk.
Jason Blanchard:Okay. Alright, and then Bryan, is a question for you. If if someone already uses Black Hills for current service, can they get a twofer discount if they go for the retainer?
Bryan Strand:Well actually if you're a SOC customer you get 40 hours of incident response retainer with with that. So that's that's a good two for deal. It's it's buy one, get one free type thing. But Patterson, what is our what is our what do we typically do if somebody doesn't, like, use their entire retainer for the year? We allow them to transfer that over to other BHIS services or a good chunk of it, right?
Patterson Cake:We absolutely do. We generally allow a 50% reuse of unused hours for the course of the year for anything else in our environment. Now we're selfish, so we'd rather you use it for IR related services, but it's not uncommon again to pivot to training or offset testing or something like that. Yep.
Jason Blanchard:And then I think that's There's a couple more
Bryan Strand:banter we've ever had, it feels like.
Jason Blanchard:Yeah. There's a couple more about, like I think here, let's do it. Alright. It's not a how to to purchase
Bryan Strand:Make sure I'm not missing something else with that.
Jason Blanchard:Service. So this is more of a question about like actual DFIR and some other things. Get your, put your webcast hat back on. For those who enjoy the DFIR parts, what are your thoughts on the obvious preference for fire preventing fires while also being excited for crisis management analysis and attribution?
Patterson Cake:Wow. That's a great question. I'm the
Bryan Strand:same person asking all of these questions.
Jason Blanchard:No. Are different people these are different people, Bryan. We're not alone here.
Patterson Cake:Bunch of That's one person with lots of different pseudonyms. Yeah. Wow. That is a great question. I was literally thinking about this morning how weird it is, Like my passion is of course preventing active incidents.
Patterson Cake:Right? I mean, I hope that we all exist left of boom, and then strangely, I profit from right of boom, and it's a conundrum. To this day, I still feel like this huge empathy for anybody who's encountering an any kind of active incident. So my strong preference, and I believe this, part and parcel of the retainer discussion, my strong preference is to just prepare like mad. Do all the things that you should be doing and be absolutely prepared should crisis ensue, and as a result, I think legit we avoid crisis by doing that.
Patterson Cake:And so you know, it's kind of a weird catch 22 to Obviously, we're huge proponents of the preparation piece so that if this does happen, it's like we got it, we know what to do, we know what comes next. Then you don't have to lay awake at night stressed and strained, and when you're prepared, Active IR is a lot more fun. It really is. Like, okay, I got this. I got this.
Jason Blanchard:Alright. Last question. This is not not necessarily about the content, but what is the pistol underneath the chopper picture?
Patterson Cake:Oh, can I show it? Say.
Jason Blanchard:You can do whatever you want, Patterson. It's your time.
Patterson Cake:So this is can you see
Deb Wigley:it? Mhmm.
Patterson Cake:Anybody recognize it?
Jason Blanchard:Indiana Joneses? Oh, come on.
Patterson Cake:We're nerds in the room. So this is this is the Mal Reynolds from Serenity or Okay. Firefly as it should be. This my son three d printed this, which is pretty cool. Nice.
Patterson Cake:Brain texture Oh, that is cool. So yeah. Everything cool in my life came from my children and my wife. But so that's yeah. That's a Serenity replica that won't actually shoot anything, unfortunately.
Deb Wigley:Unfortunately, because Kennedy can't three d print something.
Jason Blanchard:Alright. Bryan Strand, if if people here wanna do business with us, why why Black Hills and not someone else? And this is the last question that we're gonna wrap up. Why Black Hills?
Bryan Strand:I think it's because of what we do here on these webcasts every week. We only hire and bring in people that wanna do this stuff, you know, that love it, and really love sharing their knowledge. And I've always, I say this on pen testing calls and anything, where how we hire people is not necessarily based off like the best of the best of the best. Like, we've done that. We've hired the best of the best of the best, and it what didn't work didn't turn out so great, because there wasn't a huge desire for knowledge sharing, and that equates to a higher quality report.
Bryan Strand:At the end of the day, you can have the best pen tester in the entire world, and they come in and they do your pen test, and they give you a crappy report. It's not a good pen test. You can have an okay pen tester that's amazing at knowledge transfer that that wants to convey what they what what they've done, and you're gonna get a dang good pen test. And we tend to hire and look for people and bring them on all across the board that want to share their knowledge on what they've done, which is why we started a training company, and like half more than half the instructors are pen testers, or IR people from BHIS side of the house. And I'd say that's probably why.
Bryan Strand:I think you wanna have people who are excited to share what they've done, and actually wanna help you, and that's who we hire, and that's why we're kinda doing it the way we do it.
Jason Blanchard:Awesome. Well, thanks, Brian. Deb, let's go ahead and finish this out. What are your final thoughts?
Deb Wigley:Final thoughts with Jerry. Well, I guess I'm Deb Wigley and Bob today. Just again, thank you so much for sharing your time with us and spending your time with us, and I always say there's so much noise out there. Thanks for letting us be some of that noise for you. And just be kind to each other.
Deb Wigley:There's a lot going on. Keep sharing your knowledge with the community. Just be a good human. Please. Just be a good human.
Jason Blanchard:Alright. That's it. If you joined us for the Unhinged pre show banters, thank you so much for spending the last two hours with us. Awesome. We appreciate you.
Jason Blanchard:We can't do this without you, so we'll see you next time. Yeah. Thank you. Alright, Bryan. Kill it with fire, Ryan.
Deb Wigley:Is that new?
Jason Blanchard:Thank Yeah. That's new. That was weird. It came out, and I was like, I don't know what that is. Ryan might be on a meeting right now.
Jason Blanchard:We went so long after. Like, Ryan's
Deb Wigley:like, broke another meeting.