The Risky Science Podcast features conversations with scientists, insurers, investors, portfolio managers, and others about the evolving science of predicting and modeling risk across both natural and man-made perils.
Hi, this is Christopher Westfall and this is the Risky Science Podcast. We took August off, which in hindsight may not have been the best month to step away from a podcast that talks about catastrophe risk. But we're back and we're back with a guest that has a lot to say about risk models and markets. Karen Clark essentially helped develop catastrophe modeling as a commercial discipline. And she's been on the podcast before.
Chris Wesfall:She's founder of Karen Clark and Company, and KCC has just put out a new white paper on where artificial intelligence actually fits into catastrophe models. And just as importantly, where it doesn't. We get into that. We also gone to why she thinks a $100,000,000,000 loss year isn't as big as it sounds. Why reinsurers stop trusting the models on Frequency Perils?
Chris Wesfall:And what happens when an e f five tornado goes through a hyper scale data center. She's also on her way to Monte Carlo next week, which is one of the biggest and most important meetings where capital and reinsurers get together. And we have a little bit of discussion about that. Hope you enjoy the conversation. Karen, thanks so much for joining me today.
Chris Wesfall:I know you're probably busy getting ready to head to France in Monte Carlo and Yeah. All that that entails. And I really appreciate you taking the time. And especially since, you know, the firm came out with a new report regarding AI and and models. And it's something I've been sort of trying to follow more closely.
Chris Wesfall:You know, obviously not with the technical acumen that you have. So it's great to have that sort of perspective on it. So maybe I wanna start off the white paper and the research. What was the goal of that? What were you trying to communicate to the model users and other insurers, reinsurers?
Karen Clark:Sure. Well, first of all, you for having me. And yeah, the goal of the white paper was really to actually, can I start that over again? That was a different question. Let me think of that.
Karen Clark:Okay. So ask me that again. Start over.
Chris Wesfall:Sure, yeah, absolutely. So Karen, what was the goal of the paper? What were you trying to communicate to the users of models?
Karen Clark:Well, as you can probably imagine, everyone is interested in AI and how AI is touching every aspect of their lives, including the catastrophe models. So there's a lot of interest in how AI can be incorporated to enhance the models, where AI is relevant, where it isn't, and what it can really achieve for the industry. So the goal of the white paper was to give a perspective on that.
Chris Wesfall:And, you know, reading through it, obviously, I'm not a complete expert on this, but I mean it seems that the paper or well, let me put it this way. You know, in conversations of the past, you know, if I understood your argument, there's a great deal of time talking about how statistical models were sort of the wrong tool for like frequency perils and and they require like higher resolution, you know, physical models. Now, if I understand it, the paper, it's shorter throws in artificial intelligence and AI informed models into this. So maybe you could walk me through what AI actually does, specifically for the KCC model and that a dynamic model couldn't. What's the change that is sort of moving the models in?
Karen Clark:Sure. Well, I think actually the biggest scientific step change in catastrophe modeling was the introduction of the physical dynamical models that KCC introduced several years ago. That was really a radical departure from the statistical approach. And again, the statistical approach first of all, let me step back. The catastrophe models were invented so many years ago to deal with low frequency, high severity events, specifically hurricanes and earthquakes.
Karen Clark:And because these events are infrequent, there's
Chris Wesfall:not
Karen Clark:a lot of historical data. And hence, kind of the actuarial community, which relies on a lot of historical data, was not really addressing the area of extreme events. So the statistical approach was the right approach for hurricanes and earthquakes, because it enabled you to take a very sparse data set and fit distributions, and then generate a very large sample of potential future events. So again, it was the right approach, and it still is the right approach, I will say, for hurricanes and earthquakes. But that approach doesn't work with the Frequency Perils, particularly severe convective storms, which produce the lion's share of property claims every year and losses.
Karen Clark:And the reason it doesn't the paper explains that it's because the hazard component is much more complex with severe convective storms versus hurricanes. Hurricanes you can model with a limited static set of parameters, storm trap, maximum wind speed, a few other variables. But if you look at the weather channel in the morning, you see a severe convective storm, you see how dynamic they are, how amorphous, and you can note that you can't really model that well with a statistical approach. So, and there were, you know, there was even, you know, several years ago, there has been this lack of confidence in the other models that were based on statistical techniques. So when KCC scientists, or when KCC embarked upon building a severe convective storm model, we knew that we had to really invent a new approach.
Karen Clark:And not that we invented physical models, but we actually are the first and I believe still the only who have applied this physical modeling approach to the Frequency Perils. So that includes winter storms as well as severe convective storms. Now how AI comes in is it's not a radical departure from the physical models, in the same way that physical models were a radical departure from the statistical models, but they can enhance the dynamical models. So the dynamical models are, they consist of hundreds of complex scientific algorithms that can take high resolution, you know, vast amounts of high resolution atmospheric data and, you know, project hail, winds, tornadoes. So, they can do that pretty well, but in some instances, different, and I don't want to get too technical, but mesoscale convective systems are difficult, even for the most complex scientific algorithms to forecast in advance.
Karen Clark:And so that is where KCC scientists are implementing AI techniques to go back to this vast amount of high resolution atmospheric data and see if using machine learning and other techniques, we can better forecast things like dereshios or other of these embedded mesoscale convective systems within a larger severe convective storm. So AI is really enabling KCC scientists to enhance the physical models further. And, you know, and the paper gives some examples of how we're, KCC scientists have already developed, you know, these components and that, you know, some areas that are currently being worked on.
Chris Wesfall:Was it, yeah, I have to ask because, you know, everybody talks about the rapid expansion of AI and how quickly it is developing. Were these techniques even possible a year ago? Or is this something you've been looking at further than that?
Karen Clark:Oh yes, so KCC scientists have been implementing machine learning techniques for several years. So, you know, this is not something just in the past year, but it takes time. So, you know, you actually, and this is again why the paper shows that the most potential for AI is for the Frequency Perils, where we have a lot of high resolution data, because to use these techniques, you have to train. You have to spend a lot of time training the algorithms. I mean, they don't learn on their own.
Karen Clark:They can do a lot, but scientists do have to spend a lot of time creating the training data and then using the training data on these techniques. So it takes time.
Chris Wesfall:So the paper says that loss estimates will move predictably and asymptomatically to 100% accuracy. Since CAT models produce sort of like a distribution of events that haven't really happened yet, what does 100% accuracy mean for an EP curve? Like, what's the measurement and how can a user verify that?
Karen Clark:Sure. And the word is asymptotically, which everyone loves because they never use it. Agree. But, you know, But I love that word. So it means you're approaching something, you'll never quite get there, but you're getting very close.
Karen Clark:So of course the models will never be perfect. But you bring up the EP curve, the exceedance probability curve, and if you think of the EP curve, there are two major sources of uncertainty around the curve. One is the probability of One is the loss given an event. And second is the probability of a certain type of event occurring. So these are the two main sources of uncertainty.
Karen Clark:So if we start with the uncertainty around the loss, the only way really to increase the accuracy of that is to actually test the model with real events, with actual events. So, and again, we have an advantage with the Frequency Perils because we have so many more events. So, severe weather happens just about every day across The US. So events are all happening a lot more frequently again than hurricanes and earthquakes. So we have a lot of testing data.
Karen Clark:So, something very unique that KCC scientists developed, and this was in 2018, is what we call our daily live events for SES. So, every day, and what's super cool is this is completely automatic. 30 gigabytes of data are ingested into our SES model every day, and then the model automatically produces a hail and a tornado wind footprint for that day that insurers actually use, combined with the vulnerability functions in the model, to estimate their claims and losses that day, every day. So what they can do is six months, several months after an event, they can compare their actual losses to the KCC model estimates. And again, that has been in place since 2018.
Karen Clark:So that is why we can say with confidence that the KCC model is accurate because we've had all of those years of tests. And early on, where it wasn't accurate, obviously, that's where scientists spent their time to enhance the model further, advance it to continuing to enhance the accuracy. So, again, KCC is the only modeling company that actually puts our model to the test with every actual event. And that's why our models have become so accurate. Now, with respect to the uncertainty, the other aspect of uncertainty, the probability of an event, and we have an advantage with the Frequency Perils because we have so many.
Karen Clark:So the way companies, insurers would test the lower return periods, the five, the ten, the twenty year return periods, is you can compare, again, what the model says to your actual historical loss experience. So, what the complaints have been about the other models is that it may say your one in 50 year loss is $100,000,000 but that insurance company may have had $300,000,000 losses in the past ten years. So clearly that model is incredible. So you can benchmark again the lower return periods with actual historical experience. Now, what about the tail of the distribution, the one in 100?
Karen Clark:How do you test that? Well, again, it's hard to test the accuracy of that, but you can certainly test to see if a model is wrong. And for this, I'll bring up the example of the twenty twenty one Arctic air outbreak, the winter storm called Yuri, February 2021. That event caused the industry $20,000,000,000 loss from a winter storm. And the KCC model, or winter storm model, already had that as a one in seventy five year return period loss, which we believe is exactly where it should be.
Karen Clark:The other modeling companies didn't even have that in a ten thousand year return period. So, it's a benchmark that insurers use to gain high confidence in the KCC model. So, we don't have model misses, where the industry has been seeing a lot of model misses from the other modelers. So you really have to put the model to rigorous tests. And that's what KCC scientists do because our goal, as you mentioned earlier, is to have the models be asymptotically moved to 100 accuracy.
Chris Wesfall:So, one of the things I think about, and this is just in catastrophe modeling, but, I mean, you've talked about how much effort you've put into training the models, and your scientists have been training the models. And even to the point where you have these like very specific, if I understand it, like bespoke inputs you're putting into the models. So I guess the question, you know, becomes like how do you keep from sort of encoding KCC's house view as like the ground risk in the model? Or is that what you're trying to do? You know, how do you manage that?
Karen Clark:Yeah, no, it's a pretty objective process because if anyone reads the paper, we're focused a lot on derechos, for example. That's one area. And any trained meteorologist can look at a radar image and tell that that's very likely a dereshio. But the scientific formulas underneath the physical models don't necessarily detect they detect that it will be some activity in that area, but maybe not as extreme as a derecho. So, 10, let's say, trained forecasters looked at radar images, chances are they would all say, Oh yeah, that's a derecho, or That's not a derecho.
Karen Clark:In fact, and it's the same thing, we're also working on tornadoes, because today, weather forecasters for tornadoes, they don't tell you, they can't tell you it's gonna touch down here, but they draw a box around an area and they say, we're likely to have a tornado in that area. And that's because a human can look at these images and with experience know that that's probably where it's going to happen. And so those are the kind of images that we're using to train the model. And so and we're not working in a vacuum, by the way. So we are collaborating with universities to work on developing training data sets.
Karen Clark:So it's fairly objective. It wouldn't be like putting in any KCC subjective scientific judgment into the model.
Chris Wesfall:Yeah, that makes sense. I wanted to ask, because there's always this thought about like, you you can give the best model to an insurer or anyone, but if you know, the data they're putting into it, claims data or vulnerability data, you know, that doesn't really affect much, or doesn't make best use of the model. And the paper, you know, if I understand it correctly, says, you know, AI advances are in the intensity footprints on the hazard side, and that the vulnerability in the financial model is largely untouched. So isn't that like a larger error function now? How do you get around sort of the replacement costs on the hazard?
Chris Wesfall:How do you navigate that?
Karen Clark:Sure. Well, first of all, costs for properties are inputs into the model, as you know, and insurance companies provide that. And, know, KCC software, we can go through tests to see how credible that is, and we can do quality control on that. But I will say insurers, at least on the residential side, are getting very good at estimating replacement costs. I will say there are still issues on the commercial side, which maybe we can come back to.
Karen Clark:But let me first address the vulnerability component, because I would say that KCC engineers are very knowledgeable about wind, hail, even earthquake impacts on properties. There's a lot of experimental data on that. There are wind tunnel tests. The IBHS does a lot of experiments on their wind, you know, wall of wind and hail. And you know, two inch hail falls on an asphalt shingle roof.
Karen Clark:You know, we're pretty knowledgeable about what kind of damage or a tile roof or what have you. You know, EF5 tornado hits a single family home, we pretty much know what's going to happen. And for hurricanes, we've and for severe convective storm, there's a wealth of claims data that we can use to verify the vulnerability function. So, would say that we're a lot more confident in the vulnerability functions across perils, I would say, than the hazard. The hazard component, which most people don't think of, I think KCC is bringing that to light to the industry, is that it's really the hazard component, and particularly the intensity footprints.
Karen Clark:I mean, another one we're not talking about it now, but earthquake ground motion footprints. You know, there are lots of equations that are used for that, but there's still a lot of uncertainty around that. The same thing for hurricane and other perils. So, the real complexity, I would say, in these catastrophe models is around the hazard component, and within that, even more precisely the intensity footprints. If you can get that right, you can have an accurate model.
Karen Clark:And going back to severe convective storm, that's why KCC has invested so much resource in getting that right. We will be turning to hurricane as well, because again, hurricanes for the models are still pretty much statistical approach, but we are also investing in seeing if we can apply more physics based approaches and AI to hurricanes as well. Because again, we can, I mean, the statistical approaches work pretty well, but we can do better if we had physical approaches there? So, I think it's not well understood. I think the question is good, but, it's pretty clear to us where the most complexity lies in a catastrophe model.
Chris Wesfall:I have to ask, you mentioned it, what's the issue you were seeing in the commercial risk aspect?
Karen Clark:Yes, well, yes, I said the industry has made great strides over the past couple of decades on residential property information. I mean, example, I don't want to say how long ago it was, but when I started in this, like even getting five digit zip code was a major undertaking. But now, of course, where every property is geocoded where it is, We have good replacement costs. We have construction information. AI is being used now, by the way, I didn't mention this, this is not something KCC does, but AI is being used by insurers now to get better information on roof age, roof condition, with aerial imagery combined with AI techniques.
Karen Clark:So that's, again, another valuable use of AI is getting better information on the properties that are being insured. As I said, I think on the residential side, strides have been made and continue to be made. On the commercial, for some reason, you'd have to interview the brokers too, to sort out why this is. But there still seems to be significant issues, even in getting something like a valid replacement cost. I mean, we've done analyses for companies where we've seen commercial claims higher than the estimated, the TIV values that they had, or the estimated replacement costs.
Karen Clark:So, there is a lot of room to improve, I would say, on that front to get better information. It's more complicated on commercial. You have these schedules of 500 properties scattered across the country or the world being able to get good information still seems to be challenging. But I think that's a good frontier for the future to try to improve that.
Chris Wesfall:Yeah, definitely want to pursue that down the line, understanding the commercial claims data and why that's such an issue. But you know, when you talk about AI and if I understand the paper, know, there's all this data, it's like there's atmospheric data, there's claims data. So when you're sort of thinking about an AI model or a catastrophe model, to it, what's the data moat in that? If there's all this data coming in, what what's the secret sauce in a catastrophe model tied to AI?
Karen Clark:Sure. Well, KCC, we developed what you call data moat. Data moat. One is because we started working on this almost ten years ago, and we've had our model up and running for nearly that time frame. And so we've been able to ingest, obviously, hundreds of terabytes of atmospheric data that is most of it is publicly available.
Chris Wesfall:But
Karen Clark:we've been able to add to that and to correct that data and add our own proprietary knowledge to database so that we have our own, I would say, archive of enhanced atmospheric data. And of course, we've had very you know, great, blue chip, you know, insurance company partners that have had, you know, you know, very pristine claims data that we are able to match to the exposure data. And we've had the benefits of being able to, because again, because we share our loss results and our models are completely transparent, by the way. Insurers feel through a partnership with us that if they share their claims data, they're going to get a lot of benefits back. But at the end of the day, that is their proprietary data.
Karen Clark:That is not publicly available data. It's not even KCC data. It's actually the client proprietary data. So, you know, over the years, we've been able to work to put that information together into, you know, the KCC proprietary data source that we can then use for our training purposes.
Chris Wesfall:One of the things, and I think you've mentioned it before in our conversations about updating models, know, model updates and how that flows into the ecosystem. And I guess the concept is AI could move model updates from like this occasional update or into almost real time? Or am I not thinking about it correctly? And what does what's the downstream effects of that? Of of pushing model updates on a continuous basis or what does that mean for things like rate filings, ILS triggers?
Chris Wesfall:What's your position on that?
Karen Clark:Sure. Well, we believe the industry needs to move to frequent updates to stay current because so much is changing. We're learning so fast with AI. We have climate change. We have other trends.
Karen Clark:So, you know, we believe that, you know, companies need to stay current. They need more frequent updates. And in fact, we at KCC, we've been updating our models on a two year cycle anyway, our atmospheric peril models. So going to annual updates is not a big change for us. And we've also consulted with all of our clients to make sure they're comfortable with that.
Karen Clark:And the reason KCC can do that is because of our model update process. With other models, model updates can be very disruptive and they can cause big changes in the loss estimates. And so, in that environment, it will take an insurer months to figure out what's driving the changes estimates. And that's kind of been the modus operandi for, I will say, the traditional models for quite some time. Infrequent model updates with volatile and sometimes even unexplained changes in the loss estimates.
Karen Clark:So, you can't go to annual updates if you're going to have such volatility. But with KCC models, as I've mentioned, our models are already showing a high degree of accuracy. So if we change them abruptly, our clients say, What did you do? You'd made them inaccurate. So our model updates are in the mode of refinement and fine tuning.
Karen Clark:It could be for particular subperil in a geography, or perhaps like our most recent, the credits for roof age could be refined. Or they're all in the mode of refinements. And our insurers actually anticipate our updates versus being shocked and surprised by them, because insurers are informing us as to where our models could be enhanced. So that's what we work you know, by comparing our estimates to their own claims data. And so that's where we work on, where our scientists and engineers work on to enhance the model.
Karen Clark:So the updates are anticipated and looked forward to because our clients know they're not going get a radical change, but they're going to get fine tuning and improvements just where they believe those improvements would be beneficial. And in fact, what we're working on now is having reports for each client as to exactly where their loss estimates and change and why before they even receive the update. So, again, model updates for other modeling companies tend to be a challenge because of how they conduct the model update process. So it's not just that you have AI, but you also have to have a model update process that supports smooth and efficient model updates that can be implemented quickly and painlessly.
Chris Wesfall:Does does AI at least make it more efficient or more accurate on the back end to push those sort of tweaks and continuous updates to I'm
Karen Clark:not sure AI is making the process more efficient, but AI is really improving our knowledge and we're gaining more knowledge. So it's really up to us to make the update process efficient so that we can push out the latest knowledge and information to insurers and reinsurers right away. I mean, Chris, in today's world, if your model is even two years old, you're already behind the trends. So, you know, it really behooves the industry to make sure they're staying current for their pricing, underwriting, and all of their important decisions they make based on the model results.
Chris Wesfall:One thing I wanted to get into and it's perfect timing because I know you're heading to Monte Carlo and the reinsurance rendezvous which is the big industry get together to discuss capital and risk capital, sort of the process of moving capital into renewals. So, and it's pretty unique time in that sort of dynamic where there's been, you know, there's been four straight years of global insured losses of 100,000,000,000, you know, and but there is price pressure on on reinsurance capital. So, you know, from your perspective, especially from a modeling perspective, you know, if tell me think through how if prices, you know, being strictly set by available capital, where do the models fit into that? And where is the model where are those model discussions going heading into this year's renewals?
Karen Clark:Sure. Well, first of all, I like to say $100,000,000,000 aggregate loss is not that much today. So I will say that it sounds big, but one category five hurricane hitting Miami today will cause over $200,000,000,000 just from one event. So actually, the industry has been a bit lucky over the past several years, even though some, it may not seem that way. But back to the role of the models.
Karen Clark:So the models are important no matter what phase of the, I call the underwriting cycle we're in, whether we're a soft market or a hard market. Because, so the models produce the expected losses on any contract. So if I'm going to price a contract, I need the model to tell me the expected losses, and then I'm gonna put a margin on that. So what happens is that margin may change. So in a soft market, you may be getting less profit on that contract.
Karen Clark:In a hard market, you may be getting more. But you still need the expected losses for pricing as a starting point. And then decide how going to be priced. And the other thing is that no matter what the phase of the market is, can be arbitrage opportunities. So again, if you have a better model than others, you can see in any market which are the better priced contracts or the worst priced contracts.
Karen Clark:And so there's still, it's not just pricing, but it's choosing which contracts or what your participations are on any given deal. And if you have better information, in any phase of the market, it'll enable you to construct a better portfolio and get the best metrics that you can. And also, are arbitrage opportunities. So, if you have a better model that's giving you more accuracy, you can arbitrage against others that don't have a very credible view of the risk.
Chris Wesfall:Yeah, and I think in the past, you've argued the demand for frequency barrel cover is growing while supply shrinks, if I understand it correctly. And we talked a little bit about AI, we talked a lot about AI, but does AI informed models make the frequency layer more credible in in pricing? Will that push reinsurers capital back into Frequency Perils? And what does that look like for like model loss transactions?
Karen Clark:Yeah, yeah, absolutely. I mean, the famous underwriting saying, if you can price it, you can write it.
Chris Wesfall:And
Karen Clark:the reason that a lot of reinsurers had been over the past years pulling back from the frequency layers and having insurers raise their retentions is because they weren't confident, obviously, in their ability to price. They didn't trust the models they were using and they had very little confidence. So, as the KCC models become more established, primary insurers have a lot of confidence in our models and are using them already to price and underwrite. And reinsurers need to also build up their confidence again, they're so jaded, they don't believe it. It's like, how do you convince them, yes, this model is accurate, you can have confidence.
Karen Clark:But once their confidence does improve, obviously they'll be willing to write it because they'll be more confident in their ability to price it. And the modeled loss transaction that you mentioned, that KCC was instrumental in doing for severe convective storm, really the goal of that was, again, to show how confident I mean, if a primary insurer is willing to engage in a contract where their payout is going to be based on what the KCC model says versus their actual losses, they have to have a high degree of in that model. It really showed the reinsurers and ILS community that there can be confidence in models for SCS and winter storm and wildfire. And so that was really the goal, not to replace indemnity cover. Everyone would rather have an indemnity cover than a modeled loss transaction.
Karen Clark:And so the goal of that, which I think we're achieving, is to really convince reinsurers and ILS investors that the KCC models are credible, they're highly accurate, and they can feel confident writing more of this business.
Chris Wesfall:So thank you so much for taking the time. I have one last question, and of course, you can't have a conversation anymore in Risky without talking about data centers. And, you know, last year, you know, this year was a lot of discussion about data centers. If I understand it correctly, lot of those data center construction now is like self insuring. And there's been a lot of discussion in the insurance, reinsurance market about the opportunities of insuring data set of risks.
Chris Wesfall:So what's your thoughts going into this year's renewals and this year's Monte Carlo about data set of risk and where models fit into it?
Karen Clark:Well, I think, again, it's an emerging market and it's a growing market. Insurers will, over time, gain confidence in their ability to underwrite this. And from KCC's role as a catastrophe modeling company, we've been spending a lot of resource and time in analyzing the data centers, all of the different substructures, the data halls, the power infrastructure, the cooling towers, all the components of these hyperscale data centers, so that we can have the correct vulnerability by peril, hail, earthquake, wind. So we've been investing a lot of resource in making sure that our models can credibly model the risk for these data centers. But the question will be, which is of course you mentioned earlier, the data, so what kind of data are we gonna have to feed the models?
Karen Clark:How accurate will that be? Although, it's true that there's not a lot of difference between these hyperscale data centers. I mean, fairly similar. So again, from a modeling point of view, we feel fairly confident that we can give a reasonable assessment of the risk. But it's interesting, Chris, because one of the things that CEOs are really concerned about is an EFI tornado comes through, and you have billions of dollars of assets that are going to be destroyed by one tornado.
Karen Clark:And that certainly can happen, but the probability is very low. I mean, tornado risk is very interesting. What I like to say is, the probability that some commercial building or some commercial buildings are gonna be destroyed by tornadoes in any given year is almost 100%, is pretty much 100%. You know that's gonna But the probability of any specific location being hit by an EF5 is almost zero. You have to get your head around that.
Karen Clark:And I think the way the industry will deal with that is, of course, the spread of risk. You're not going to put all your capital on one data center, but you will take a share of different ones. They're geographically spread out, which they are geographically spread out at this point. And I'm fairly confident that the industry will be able to embrace this emerging peril over time. It's a little bit getting, like even with the new SCS model, just getting comfortable with it over time.
Karen Clark:And so we have a new white paper coming out, by the way, in time for Monte Carlo, on hyperscale data centers. So we'll send you a copy of that and maybe there'll be some conversation on that later.
Chris Wesfall:Yeah, actually I'm very interested to learn what happens when an EF5 tornado hits a hyperscaler and takes out, a couple, as you said, couple billion Nvidia chips, would be a real risk. So appreciate it. Thank you so much for taking the time.
Karen Clark:Yeah, you're welcome. Thank you.