Subscribe
Share
Share
Embed
Net load uncertainty, growing markets, opportunity-cost-based resources, and agent-based simulation of electricity market design changes are the four topics at the forefront of electricity market design. In this episode of Frequency Band, hosts Paul Dockery and Becky Robinson discuss topics that two of the field’s most respected experts on market design find most interesting.
Prof. Benjamin Hobbs and Dr. Scott Harvey are two long standing members of the ISO’s Market Surveillance Committee (MSC). In this two-part series, they share wisdom drawn from decades of experience across North American markets on the “wonkier” side of challenges facing electricity markets.
Prof. Benjamin Hobbs and Dr. Scott Harvey are two long standing members of the ISO’s Market Surveillance Committee (MSC). In this two-part series, they share wisdom drawn from decades of experience across North American markets on the “wonkier” side of challenges facing electricity markets.
Creators and Guests
What is Frequency Band?
From Fort Peck, Montana to Tijuana, Mexico, the alternating current transmission network in the West oscillates in the narrow band of narrow band 59.97 and 60.02 cycles per second. On Frequency Band, the California ISO will host industry experts to talk about how the physics, economics, and governance of the grid come together to to keep the system in sync.
Thanks for tuning in to Frequency Band. I'm Paul Dockery. This episode is the second part of a conversation Becky and I had with Ben Hobbs and Scott Harvey in late April on the issues we find most interesting at the forefront of electricity market design. In the first episode, we heard Scott Harvey's enthusiasm for managing that load uncertainty and Becky's interest in collecting wisdom on the growing pains of growing markets. This episode picks up with Ben Hobbs's thoughts on what it means for electricity markets, that the power system is moving from thermal resources with marginal costs informed by fuel prices and heat rates, to a system with storage resources where opportunity costs drive pricing.
Paul Dockery:Then we discuss my topic, methods for simulating market design changes. If you haven't listened to the first part yet, I recommend starting from the beginning. Regardless, I hope you enjoy the conversation. Okay. So Ben, you're up next.
Paul Dockery:What what topic So is most intriguing to
Benjamin Hobbs:in the Let's see. I've been working in this industry since 1974, and a lot has changed in that half century, but some things haven't. So I'm gonna talk about something that has changed and hasn't changed. So when, Fred Schweppi, who was a professor at MIT, electrical engineer, conceptualized in the late seventies the idea of coordinating consumer decisions and generator decisions with locational marginal prices. So using security constrained unit commitment and dispatch and looking at the marginal costs and then settling transactions based on That was a revolution and that ultimately became the basis of FERC's standard market design of MRTU and locational marginal prices.
Benjamin Hobbs:That's the intellectual foundation. So when you put that together, and when MRTU was implemented in the CAISO on April Fool's Day, 2009, the system was mainly dispatchable thermal units that had a heat rate and a fuel cost. So you kind of knew what most generators' marginal costs are. Admittedly, there's hydro out there and that was that was difficult. But that's the world of Fred Schweppey analyzed and basically calculated prices using a static model, so that didn't have to look way ahead of time.
Benjamin Hobbs:This is an entirely different world now, now that we have what 17 gigawatts of on grid battery storage, which is roughly a third of our of the peak load here. In the evening, a big chunk of demand is is met from these batteries. They do not have a cost that you can calculate from a heat rate and a fuel cost. Their cost is an opportunity cost. If they if a battery discharges now, it can't discharge that energy later, and its cost is the revenue it gives, gives up later, which is a price that people can disagree on that you have to forecast.
Benjamin Hobbs:It's not something that, you know, accountants like to see a receipt, right? They like to see what was paid for the gas and that's the number that we'll use. We don't look at yesterday's gas prices and figure out what that is. So this means that American style market power mitigation is going to be more difficult in the future.
Dr. Scott Harvey:And I'll
Benjamin Hobbs:explain what I what I mean by that. And maybe even that it's a hopeless task and we'll have to give up doing it. Well, we'll have to see. Okay, so let me talk a little bit more about market power mitigation. So the way that the Americans do it is that we make a calculation, an estimate of every resources marginal cost.
Benjamin Hobbs:And there are challenges even just using heat rates and fuel costs, because should we use yesterday's fuel costs or future trades? What do we do about startup costs and P min costs? Because those are maybe registered once a month. And so those gas prices may get out of date quicker. So as so even just doing the heat rate times, the fuel costs can be complicated and we can get that wrong.
Benjamin Hobbs:And that has gotten the ISO in trouble when suddenly fuel costs moved up really fast and we were not paying generators their cost to starting up and they didn't want to bid. And so suddenly, your resources are drying up just when you need them. So I'm not saying that that's easy, but now it's now it's now it's even harder. So so American market power mitigation first wants to estimate the the marginal costs for all resources. And then second, assess whether the the resource has the ability to move the market.
Benjamin Hobbs:And in the Eastern markets is something called conduct and impact. Conduct is as if you're a bad boy or girl and you raise your offers well above your marginal cost. An impact is whether that actually would move the price, and so you do a market simulation with that. That's not how we do it in California. In California, the potential for exercising market power is looking at, in a sense, the supply of resources to meet demands in load pockets considering the ability to move power into a load pocket through whatever interface that you have with the rest of the system.
Benjamin Hobbs:So in a sense, there's a demand for transferring power into a load pocket. And then there's supply from folks, from residual, from excess supply outside that region, and you do a comparison and you ask, can anybody pull their supply out and say, oh, gee, I've we've got a problem. We need to do some maintenance today. We're not going, you know, an NROM sort of a thing, what is the ledge NROM did? We're gonna pull the supply out of the market.
Benjamin Hobbs:Does that create potential create a shortage, a near shortage situation, which could really push the price up? We don't actually simulate where the price might go, which would be an Eastern Impact test, but we do assess whether there's there's there's some leverage there. So if you are bidding well above and what well above means is is a judgment, your marginal cost and you have the potential to move the price in in some local areas, then you are ex ante mitigated, your offers are reduced to something called a default energy bid. There, that'll give me a wonkiness point.
Paul Dockery:Ex ante to also add a point. Yeah, that's great.
Benjamin Hobbs:I gotta I'm gonna stick in some other, Latin. Compare this to Europe, which is ex post, which is the regulators sit around. Let's say you're in Spain and the old days are two main generators in Spain. They roll out a bed in the morning, decide what they're gonna the prices they'll offer at, and lots of room to exercise market power. And if they if it's too blatant, the regulators to sort of wake up and they may take you to court, but it is very difficult to to do that.
Benjamin Hobbs:You do that very infrequently and it is proven largely ineffective. So, that's the European approach. So, we do the ex ante approach where we check before the market runs and we adjust the offers. So what's new is that with store ubiquitous storage and storage is going to be on the margin for a very important part of the day, which is now the late afternoon and the evening. And in the California ISO system, we do allow them to make offers at what they'll discharge at both in the day ahead market and in real time.
Benjamin Hobbs:I had argued, in an MSC opinion, that we should not there should not be offers day ahead of time day ahead, because the ISO has the picture the entire twenty four hours. And so with four hour batteries, the ISO can calculate the opportunity cost. You don't need an offer from the resources for that. Maybe for battery degradation costs, but and originally, that's what the ISO was going to do, not allow offers in the day ahead market, but they changed their mind. So now you so now we have this weird melange of storage can make offers, at what price will I be willing to discharge different hours, at what price am I willing to buy to charge my battery, and at the same time the ISO is keeping track of state of charge and has shadow prices, so it's neither fish nor fowl and kind kind of a mess.
Benjamin Hobbs:And in this messy system, the market monitor has to estimate the default energy bid. What is storage's marginal cost? And that is very difficult problem when folks disagree as to as people will, even two hours ahead of time, what the prices will be in in two hours. Could they leap up to a thousand, $2,000 or could they settle down to $50,150? And that's what determines the debt.
Benjamin Hobbs:And we we are learning to try to do that And as batteries grow importance, I think it's possible that we'll conclude that it's simply infeasible to have credible estimates of these opportunity costs. And we should just let the market competition in the market take care of it. And there's a problem because in many local areas in California, you have 100 megawatt, 200 megawatt, very large battery installations controlled by one owner. And the potential for exercising market power, particularly in the five minute market, can be quite significant if that's the only resource that can move move in five minutes. So I'm not going to predict what this new MSC will recommend or what the ISO will be doing about local market power mitigation in ten, twenty years.
Benjamin Hobbs:I know there'll be a lot of discussion about how how difficult it will be, and they'll have to wrestle. If they stick with American style ex ante market power mitigation, they will have to crack this estimation of opportunity cost nut. Maybe what you need is a liquid futures markets on a scale of hours to a couple days and and somehow reference that. I'm not sure. Scott, no doubt, has some ideas.
Dr. Scott Harvey:Yeah. I think the it it is an increasing challenge for mitigating storage, and I share your view that we have to think hard about whether it even makes any sense to mitigate a lot of that storage. And an important thing to to step back and think about what we're doing in California and the West is there are some days where we have to have that storage full going into the peak and we have to have it or we can't meet load. Yeah. But on a lot of days, that isn't the story.
Dr. Scott Harvey:We got enough gas that we could meet the load without the storage. What we're using the storage for is to shift solar into the night at a cost that's lower than gas, avoid the emissions cost. So it isn't like they've got the market power to raise the the price to any level. They've just got a little bit of market power versus what's the cost of burning gas. And and that's you know, we have to say how how big a deal is that.
Dr. Scott Harvey:And if there you know, some places there, a resource providing transmission because when a transmission lines out, they're what we need. But that isn't every day. That's just sometimes. And, you know, MISO has dynamic, you know, local areas for mitigation, but they're only triggered when a transmission line is out in that situation or exists. You can you can account for that.
Dr. Scott Harvey:One strength of the the conduct and impact test is that, you know, in the in the the three equivalent supplier tests, we're actually saying not just one generator pulled out some of its supply. It's like all three of the largest generators withheld all of their supply. It would be hopelessly uneconomic. The only reason that it makes any sense at all is because the test is so bad that we don't test for whether the fringe supply in the region can be delivered. It might be behind a constraint.
Dr. Scott Harvey:It might be wind that's curtailed down because it's behind a transmission line. It might be really high cost. It might be, you know, twice the price of the others. And that's what the conduct impact impact test catches. It says, can it actually be dispatched?
Dr. Scott Harvey:It isn't gonna count competition from wind that's behind a constraint, and it's not gonna count resources that are extremely high cost, like batteries that only have a very limited charge left. So, there's tensions about how we deal with these things. Yeah, we're gonna have to I'm concerned that we mitigate the storage too much and it's one thing to be mitigating them during the net load peak hours when we might need them, but you you look at we don't publish as much data as we probably should, but I see some of the data that looks like in the in real time, we're mitigating it in the the afternoon hours a lot of the time. And I I question whether that's, you know, what mitigating market power or just bad, you know, index prices, you know, bad default energy bids. It's it's I agree with Ben.
Dr. Scott Harvey:It's a it's a big challenge and probably the multipart solution. I'll I'll throw in one walkie thing. Thank you. Keep a perspective of other things being
Paul Dockery:Paul happy.
Dr. Scott Harvey:Yeah. Yeah. Is the the IESO in Canada, they switched to an LMP market this year. And one of the experiments and we don't know how this is gonna work out. It hasn't even been running a full year, but they have a product called a a software tool called ERUC Okay.
Dr. Scott Harvey:Which looks out over the entire rest of the day. And, actually, at a certain point of day, it looks out over the next day. So as Ben says, it's it it sees the whole it sees the net load peak and through the net load peak. So it can it when it's used for mitigation and applies the conduct and impact test, it says, well, we it it doesn't just say, well, if we if we dispatch this storage unit early in the day with lower prices in that hour, it's looking at it the rest of the day. And it says, wait.
Dr. Scott Harvey:If you dispatch that resource now, we have to run something really expensive at the peak or we don't have anything at the peak. And it says, no. That's not market power. That's efficient use of the of the resource. So, you know, that but we don't know how that's gonna work.
Dr. Scott Harvey:There are lot of challenges in doing that right, but it's an interesting thing to for us in in the Western EIM and CAISO to to look at and how it's doing, and we can think about you know, maybe we could do it even better than they do and and see how it plans pans out.
Paul Dockery:So we do a multi interval optimization with look aheads. Think four hours is close thereof.
Dr. Scott Harvey:So long is the stuck four hours, but most of it is RTPD, which is looking out about two hours, two hours, fifteen minutes, which is And is You you think about that in terms of the end of the net load peak at 10:00 at night. Right. Well, you don't see that until, you know, 08:00 in in RTPD or 06:00 in Stuck, which you've already been through the afternoon. So you could be blithely dispatching your storage all afternoon, and that's just the look ahead for unit commitment and interchange scheduling. The actual real time dispatch, which is what uses up energy, only looks like an hour.
Dr. Scott Harvey:Okay. So you might not see that you're running out of storage until 09:00 at night. You know? Or or, you know, it's so that's the challenge that
Paul Dockery:Is what is it IESO? Yeah. IESO. Stand for?
Dr. Scott Harvey:Yeah. Independent Electric System Operator.
Paul Dockery:Okay.
Becky Robinson:Of Ontario. Right? Of Ontario.
Paul Dockery:Right. And it do they is it a similar mechanism just with looking further or optimizing further?
Dr. Scott Harvey:Well, it is except they to to make it solvable, they look out in hourly increments, which doesn't compromise. And that's one of the things why this is an experiment. The industry needs to look at it and see how it works, but maybe for modeling storage use over the day, that's good. And then you can use information from that. If you apply a conduct and impact test in ERUC, you won't mitigate.
Dr. Scott Harvey:You don't mitigate them in the in the real time dispatch because you say those offer prices are just fine when I look out over the rest of the day. So you don't so, you know, you're you're
Paul Dockery:I don't wanna argue with the wisdom of the MSC, and I'm not. But I do wanna pull back to, I think, what you talked about, Ben, which is on the conduct side of the conduct and impact test, you do have another epistemological problem of what is their marginal cost and is it it's hard to test for conduct and a conduct and impasse with a battery.
Dr. Scott Harvey:No. But the thing is it doesn't matter what their conduct was. If it if they if there's no harm, no foul in the impact test, they're not mitigated.
Paul Dockery:But if there is a harm and a foul, what do you conduct against?
Dr. Scott Harvey:No. The yeah. There's a there's a default energy bid so that if it fails the impact test, that's mitigated just like in California. But nothing happens unless you fail the impact.
Paul Dockery:We still haven't solved the don't know what to default it to.
Dr. Scott Harvey:Right. Yeah. But Ontario has, you know, rules, you know, somewhat like California in in in in real time for the default energy bid in terms of looking at the instead of California looks at the fourth highest price in the day ahead market, Ontario looks at the highest price in the day ahead market. But it's the same concept with just, you know, their way of looking at things. And and there are lots of different institutions in Ontario.
Dr. Scott Harvey:There's lots of different resource mix. So nothing you know, their market power mitigation doesn't translate exactly the same in terms of the challenges and everything.
Paul Dockery:Be way into the
Dr. Scott Harvey:walkiness scale if I started talking
Paul Dockery:about those differences. Yeah.
Becky Robinson:Well, to make sure I'm tracking and and maybe triangulate, Paul, your question, it it sounds like, Scott, you're saying, right, we're not in this model, you're not relying on perfecting what is that reference level or that default energy bid. But you're saying the structure of the test means that, because they're looking out over the rest of the day, say they bid $100 or $200 right? Something that sounds high, right, might be higher than what you think their reference level is on that particular day for whatever reason. But you're saying that because you're looking over the rest of the day, if we were only looking at the next hour or two, you might say, well, you've got market power and and you're well, you sorry, under the conduct and impact test, you might say that looks like bad behavior, conduct. Right?
Becky Robinson:You're bidding
Dr. Scott Harvey:so hard. Raised the price at at 02:00 in the afternoon, perhaps. Right. It for by not depleting your storage. But when you look out over the rest of the day, you say, no.
Dr. Scott Harvey:It didn't raise prices over the day.
Becky Robinson:Because I didn't wanna use them, and that that's 02:00 in
Dr. Scott Harvey:the afternoon. Because if I used them at to lower prices at two in the afternoon, I would have raised prices a lot more than that later in the day. Yeah. Now in terms of the conduct test, you know, if we look down the road and say, does ERUC work really well? In ERUC, when it solves over the rest of the day, it's got a shadow price value of that energy storage.
Dr. Scott Harvey:So we could, in some future year, when we decide all of this works, there's a path, take that shadow price and use that as the default energy band.
Paul Dockery:Now we're in the land of the duels. Right. Yeah. Shadow price, land of duels. And I I wanted to tease out one thing because you mentioned that a lot of the times you have the gas resources is the next marginal resource.
Paul Dockery:In a lot of ways, it's thinking about batteries as competing with that gas. It's out competing the gas. And the competitive level is where the battery is out competing gas. Mhmm. That was nothing other than something I wrote on my card, and I was like, wanna make sure because that helped me when you were talking about, oh, the you have the gas there.
Dr. Scott Harvey:Right. The the the storage fills two roles. One is capacity, but most of the time, it's not capacity. It's reducing emissions by shifting solar or wind, to replace gas.
Paul Dockery:K.
Dr. Scott Harvey:So It's a different kind of market power. I mean, it's very limited and not something we even wanna try to fiddle with.
Paul Dockery:So I would I'm calling this, Ben, Schweppes revolution to the storage revolution, like, Schweppes revolution. Wait. Wait. Wait. What's what's this?
Benjamin Hobbs:Okay. From heat rates to opportunity costs.
Paul Dockery:From heat rates to opportunity costs. Anything else you wanna add? Because this isn't the only thing. Market power mitigation isn't the only thing in this change of heat rates to opportunity costs. Anything else?
Paul Dockery:Before we I'm give you a eager to hear You're to hear my topic. I'm flattered. Okay. What do we wanna give Ben here on the scale of kitchen table issue to the jewels and the shadow price? Scott, back
Dr. Scott Harvey:to you. An MSC level issue because we've been talking about it in last few opinions. My I my judgment is this is this a problem we've been Ben's been talking about in opinions for the last five years. It's gotta be an MSC level issue. Right?
Paul Dockery:Are you up against the duals space with this with this with this specific issues of how complicated it is?
Benjamin Hobbs:You can if you start thinking, well, we really should be putting in the the cost of battery deterioration and life shortening and so forth as a function of the state of charge. There's a lot of technical aspects of batteries that we do not model well in the market software. And that I don't not even sure that the resource owners fully understand and reflect in their offers. That would be off in dual space. Okay.
Benjamin Hobbs:And I think that would be
Paul Dockery:But you weren't getting there. That's where No. Were trying to Okay. What are you Becky? Is he a four?
Becky Robinson:I mean, I think it's you know, I heard Scott bring up, you know, there's a shadow price of this ERUC run, and maybe that's what we should use, you know,
Dr. Scott Harvey:the short term value. This is like, see what happens and how this plays out. I pull back a lot. I'm just saying this is an experiment being run. It's an interesting thing to look at.
Dr. Scott Harvey:Absolutely. If it works Yeah. There are things to use. But it might not work.
Becky Robinson:Fair enough. But an interesting place to
Paul Dockery:watch. Okay.
Becky Robinson:It's definitely at least a four.
Paul Dockery:At least a four.
Becky Robinson:You know, I think there's arguments for five, but I I know you don't wanna go have you know, you don't want you want whole numbers.
Paul Dockery:Yeah. I can't do fractions. That's just not fair. So we're
Dr. Scott Harvey:or two, five. Four. We're going with
Paul Dockery:a four. We're going with a four. You feel okay with a four? You're you're at the you're at the MSA. Keep the same.
Paul Dockery:All of us apparently are at the MSA. Okay. So I'm next. This is the last of the topics. My area of enthusiasm at the frontier of electric market design is agent based simulation of market design changes.
Paul Dockery:And I went back and looked at some history and I looked at some of Ben's papers from the February and he has this paper on strategic gaming analysis for electric power systems, where you talk about a strategic gaming model. I believe it was an equilibrium model.
Dr. Scott Harvey:Okay?
Paul Dockery:And there's characterization of the way we think about electricity and can model them. There's optimization problems. There's equilibrium models. And then there's simulation models. And where my area of enthusiasm is, is thinking about whether with the proliferation of machine learning and agents and agentic formulation and be able to deploy agents within electricity markets, you can simulate bidding behavior of these, participants in electricity markets.
Paul Dockery:One of the problems with testing new market designs is you don't know how people will respond and how they'll change their strategic bidding behavior in response to a new market design. And I think there's opportunities and there's a lot of, I think literature going on right now about how to use AI for simulating bidding behavior to think about what optimal bidding strategies are electricity markets. And I'm wondering if there's opportunity here to think about using some of those techniques to test market design before it gets implemented to test it under simulated bidding behavior of participants and stress tests around what if it's what if they're dumb agents? What if they're smart agents? What if you use machine learning techniques like reinforcement learning?
Paul Dockery:What if you use what if you can use an LLM model to to do bidding and to simulate bidding of some of this stuff? I think it's an interesting way to at the frontier of market design policy development. And that's my area of enthusiasm. And I think Ben, you've done some thinking on this obviously for a long time, maybe.
Benjamin Hobbs:So of course, I'll say that more research is always justified and here, give a grant and I will work on that. Twenty five years ago, Hong Po Chao from the Electric Power Research Institute said we needed exactly this sort of platform to test market designs before you put that. We have to have confidence that they they won't blow up. And they weren't able to do it with the research skills and computers at the time, but people have long had this holy grail of gee, it'd be great to predict how things will work out instead of what often happens now, as we say, while we think this pushes the market in the right direction, we really don't know what the right parameters are. We we maybe see some dangers, But the operators, we're just gonna have to go into it and then learn what works.
Benjamin Hobbs:And if there's some problems, make some adjustments. That's how we've pretty much always done things. And there have been a lot of cases over the years. So, you're trying to predict when things might blow up vulnerabilities. And this sort of modeling is kind of a possibility proof.
Benjamin Hobbs:It might show this is a vulnerability. I would like to see, for example, there was a market strategy of a certain market player who took advantage of the transition from one day to the next of where its resources was at eleven p. M. Or midnight, and then, bid it strategically for one a. M.
Benjamin Hobbs:Of the next day at a very high level and you had to pay that very high bid. And the market participant questioned earned some several 100 millions of dollars from this strategy. I won't name the market participant. I'm not sure I'm allowed to. But at any rate, it was something that happened in California.
Benjamin Hobbs:Would a test for such a model would be a large language model, whatever, would be able to discover that same strategy? Yeah. Right? So, if it doesn't, then we might have less confidence that passing this model test means that it's going to work. I think it's worth a try.
Benjamin Hobbs:I'm not terribly optimistic. There's so many degrees of freedoms in agent based models and you're training it on data from past behavior under a different market design, the behavior might change. Well, let's see. Another value of such a thing, it might uncover as a possibility proof possible strategies that could cause things to be very expensive blow up. Yeah.
Benjamin Hobbs:Worth its try. The other thing it might discover is where, you know, the likelihood or possibility that a new product that's proposed or market change has absolutely no value. So we're gonna be paying the software vendor for the ISO, you know, tens of millions of dollars to make software changes. We're gonna have all these stakeholder meetings and all the staff time invested and it didn't didn't make any difference whatsoever. So we often see that.
Benjamin Hobbs:There's something called mileage payments for regulation, how much you move that and created a product that has a price and folks can offer for that, and it's turned out to have no value. With flex the flexible ramp product, we're still struggling with, you know, we see very low prices for that. Is that because we're not modeling the deployment constraints right? You know, the the flexible ramp that's procured has to be able to be dispatched in in real time to meet a need, you know, if we wind up always acquiring it behind some transmission constraint, then of course it won't have any value. And, we've been working on that for, you know, several years there.
Benjamin Hobbs:Perhaps this sort of testing might have revealed that that would be the would be the case. So it's worth trying, but by no means do I think it's a panacea. And, to gain confidence in it, we'd have to test it against past. We'd want to test against past situations. Yeah.
Benjamin Hobbs:And, and then in a sense out of sample tests going forward, use it, see if it raises any red flags, and then see if those problems actually occur or not.
Paul Dockery:I ran a little nine bus network with a reinforcement learning agent and they did it explored or it found both, physical withholding and financial withholding. Some reinforcement learning, I felt good about it. You know what was wrong with it? It took like four hours to run on a nine bus network. So it's the problem is like maybe Paul's computing skills aren't that good.
Paul Dockery:It's also like it is fairly computationally challenging to test a lot of these things. Scott, what's your take on whether these types of things are at all valuable or whether Well, this is this is whether it's interesting.
Dr. Scott Harvey:This is testing gaming and market power. Yeah. And, you know, there might be some value to that, but I think a lot of our problems have not been that we didn't see the problem. We just didn't pay attention. Yeah.
Dr. Scott Harvey:And if you go back to so a lot of the going back to PJM in 1997 when they went live with the market and it blew up in June and PJM had unilaterally changed the rules. Well, what happened was for you know, Andy Yacht and I ran maps models, and it was in Bill Hogan's testimony the year before, and FERC ignored it and went ahead and was sure enough. It it didn't work in in practice just like it didn't work in theory. Yep. And they shouldn't have need to know that.
Dr. Scott Harvey:They feel like all of the gaming litigation about, you know, the up to congestion bids in in PJM, and there's probably 10 or 15 cases, well, that strategy when when FERC said they were gonna give these payments to the virtual bidders, everybody said that's what's gonna happen. There shouldn't have been any surprise in the world. But they did it and what you know, we didn't need an agent model. Right. It was there in the in the in the filings.
Dr. Scott Harvey:This is what's gonna happen, and that's what happened. I mean, sometimes it's you know, we we somehow think what, obviously isn't gonna work in theory. It's somehow gonna work in practice, and it never happens. On the other hand, I do think simulation and testing is really important. Yeah.
Dr. Scott Harvey:You know, we I think at the Kaiso and some other places, we don't do enough modeling and not just worrying about what the market participants are gonna do, but just how is our software. You know, FlexiRamp, they don't even put in a bid. There's nothing it that's all our problem. We own it when this stuff is not deliverable. And that's the and some of it's hard to test, but we I think we need to do more of that of analyzing.
Dr. Scott Harvey:And I've tried to, in the past times when we have, you know, constrained events and high prices, look back at, well, why was the FlexiRamp price zero going into this when, you know, we ran out of ramp and we're got thousand dollar prices and what happened? And it looks like, you know, we're counting all that hydro in the Pacific Northwest, so we got lots of ramp. No problem, Senor. Well, there is a problem. And now we've got the new issue of the bend raised of the storage.
Dr. Scott Harvey:You know, when we run that the ReflexiRamp test, we count the 17,000 megawatts of storage as providing ramp. But wait a minute. We if we need that for the evening peak, we really don't wanna be burning through it and dispatching it for the ramp. We might if if you know, we might wanna look at that differently in our modeling, And and that's what, you know, some of the things we've been talking about. So there's a lot of need for modeling.
Dr. Scott Harvey:And and as things get more complicated with the the way we're modeling energy usage for for the ramp product and energy usage for regulation, and it's not like we're being unnecessarily complicated in creating it. This is the world we're in with and going back to what we talked about at the beginning, intermittent resources and net load uncertainty, that's where we are whether or not we like it. And, you know, we can't just say we wanna be simple and set the lights go out or build 40,000 megawatts of generation we don't need. We we need we need to deal with the complexity, and that's hard. But I think simulation and some of the ISOs that there there is more simulation early on and during the stakeholder process iterating back and forth about how it works and and and and even the the concept changing in the course of that discussion and the course of the simulations.
Dr. Scott Harvey:And I think for some of the complicated stuff, almost everything ISO do does is really complicated these days. We need to we need to do that.
Paul Dockery:Yeah. Do have anything to add, Becky?
Becky Robinson:Well, I I I think those are those are great insights. And but I I do wanna you know, for our audience, on the FlexiRamp within California ISO and the challenges that you're highlighting, and those are totally valid challenges. I think they they remain challenges for us even as we have tried to make incremental progress with that product, with the deployment scenarios, you know, to test if something's behind a constraint or not, trying to roll, trying to apply those more across the board, right, not just for our base flows, but also for nomograms and contingencies. That's something that we're, again, trying to make progress on, but it's challenging from a self time perspective. But there's one instance I know where you've asked the questions about, yeah, why didn't we see a higher FlexiRamp price, you know, in this tight system condition?
Becky Robinson:And, my colleague Guillermo looked into it and his team, and and they found that, well, even if we had identified that, which maybe we missed in the in the model, we wouldn't have procured it because the demand curve would have cut it off. So it's like the challenges are coming
Dr. Scott Harvey:from the Right. You know, I'll I'll go off on that wonky thing because I think there's a problem in the way we do the banter. You know? And and decisions are made to maybe keep it simple, but I don't think they're the right decisions. And that's that's something that we should be testing.
Dr. Scott Harvey:We should be modeling that and understanding the implications of the way we set up the demand the demand curve, in particular that when we relax the demand curve inside a local area, we also take that out of the aggregate demand curve. So if we can't deliver it, we we put out net load uncertainty to somewhere in some you know, up in Humboldt, and then we can't deliver it there. We don't procure it anywhere else. And if the load net load uncertainty doesn't materialize in Humboldt, but it materializes somebody else somewhere else, we're short. And so I think that's where, you know, some of this we have to step back and think about the way we're modeling some of these things and do simulations to understand that.
Dr. Scott Harvey:Because sometimes the real world is so complicated, it's hard to see what's going on. And sometimes simulations, like in the New York ISO when they developed the dynamic reserve product, we went through multiple levels of simulations of different levels of complexity. And, like, we started out with a load pocket model that was basically New York City with a couple of transmission lines coming in. Yeah. But we worked that and did lots of simulations and models off it.
Dr. Scott Harvey:And the NISHA RANs, you know, solve the you know, a linear programming, and then I took it into and we and our model, we analyzed all the settlement effects on TCCs and other things and worked things out. Then they built the prototype and tested that, and now they're building the final one. And and the the product changed hugely in the courses of that iteration and study. And it was a zonal model at when it started, and now it's an it's a nodal reserve pricing and scheduling model now. Yeah.
Dr. Scott Harvey:Because that's what we found we needed. And, you know, that's, I think, important to do. The MISO with their FlexiRamp product to get it started. I did very simple simulations. They gave me all the data for all the units in in hours where where we were studying because of spikes.
Dr. Scott Harvey:We drew some you know, we learned from that. Like, okay. These are things we need to to nail down. Then they built up, you know, a a a simulation tool, and then they built and ran more complicated stuff and then a prototype. And you learn from that.
Becky Robinson:All those steps.
Dr. Scott Harvey:Yeah. All those steps and starting with something that maybe is simple enough to understand and get the big picture right and then testing it at more levels of complexity. I like that. I think that's important as we get
Paul Dockery:The staircase of complexity.
Dr. Scott Harvey:Yeah. And so many interactions, you know, because storage affects the FlexiRamp product, affects all you know, it's iterative complexity and some and there are four or five pieces of FlexiRamp that
Becky Robinson:Yeah. No shortage of interesting things to think about.
Dr. Scott Harvey:Exactly. Lots of PhD topics.
Paul Dockery:Yeah. And lots of PhD. I think going on right now, I think this is an area where there's a lot of active research around it. And Ben, like, so and I think your paper was on an equilibrium model to do some of this strategic gaming. I'm not doing agent based person.
Paul Dockery:You are?
Benjamin Hobbs:No. No. It's it's nice to be it's hard to prove theorems with agent based ones. And I like theorems. I guess that means if you scratch me, I'm a little bit of an economist.
Benjamin Hobbs:That's easier to do with equilibrium models. Okay. But that's just a personal taste.
Paul Dockery:Okay. So I what I heard out of this was modeling. Yes. We should do more modeling. Agent based simulation, like, sure.
Paul Dockery:Why not try it? Mhmm. But but generally, endorsement for modeling.
Benjamin Hobbs:The agents can be, grad students fed by pizza. I mean, you can have real people. So that's something that folks have also done is experiments. It's just they're a little difficult to duplicate and so forth. But sometimes you get insights and, it's amazing how quickly students can break a system.
Benjamin Hobbs:Yeah. So that that's a possibility too, in addition to letting the the computer rip.
Paul Dockery:Yep. Okay. And where do you rate me on our wonkiness rubric from kitchen table to dual space?
Benjamin Hobbs:Oh gosh. That's that's out in the extreme.
Paul Dockery:That's got
Benjamin Hobbs:to be a five.
Dr. Scott Harvey:Yeah. That's a good point.
Paul Dockery:Okay. Becky, my mind?
Becky Robinson:I think especially because I think in discussing your topic, we also got back to the first topic of net load uncertainty because it's just it it brings in all of these Mhmm. Brings in lots of things in discussing it. So I'd give you a five.
Paul Dockery:Okay. I made it I made it as a five. Broke up the scores a little bit. Scott was at a four with managing that load uncertainty still at the MSC. Becky also at the MSC with a four with growing pains of a new market.
Paul Dockery:Ben also at the MSC, but like MSC plus with Wisdom for dealing with system of opportunity cost based instead of heat rate based resources. And I am in dual space out of five playing electric market design games with agents. That sound right?
Dr. Scott Harvey:Alright. That's right.
Paul Dockery:Okay. Well, this is a wonderful conversation. I appreciated it. Market runs end with market awards and we end Frequency Band with a segment we call Frequent Awards. Where you get a niche, industry award for your contributions to the podcast.
Paul Dockery:Scott, I'm happy to convey Frequency Band's current overcurrent protection relay award for keeping us current with understanding what's going on in other markets, but also protecting us from the cutting edge by tripping us offline or tripping us to make sure we don't fall into a abyss below. About to explode with all those metaphors. It's okay.
Becky Robinson:Prior hour of conversation. Yeah.
Paul Dockery:Right? Scott, any touching speech you wanna leave us with at the end of this episode?
Dr. Scott Harvey:It's been fun, and I think the the dominant theme is how complicated the challenges the Kaizo and Western EIM have. All these things are interrelated. And but we're moving in the right direction, but it's complicated in every step. There are many steps.
Paul Dockery:Well, thank you. And thank you for all your years on the MSC and for all your contributions to the ISO over the years. I hope you feel seen, heard, valued, appreciated. Of course. Good.
Paul Dockery:Ben, I'm happy to convey to you the Synchronizer Award. I've got a Synchroscope trophy for you. Synchroscope 60 Hertz or 50 Hertz? It's a 60 Hertz That's right. Showing LPN sensibility.
Paul Dockery:You chaired the MSC for a long time. You helped synchronize perspectives and made sure you were herding together, Jim and Scott, and use perspectives into a single opinion. So thank you for that. It's a good synchronizing perspective. We always need somebody on the grid making sure we are oscillating to the same frequency.
Paul Dockery:Any any closing thoughts, any speech you'd like to give?
Benjamin Hobbs:I wish the new MSC luck. We allowed to mention names maybe. They will
Paul Dockery:be approved before this episode goes up.
Benjamin Hobbs:Was chaired by Jim Bushnell and joined by Ross Baldic and Jacob Mays, all extremely smart people. The challenges they will face over the next ten, twenty, well in my case twenty five years are similar to but different than what we did. I mean we're always gonna have to deal with uncertainty and variability which will simply get greater in the short run with renewable resources. With, you know, figuring out how much charge, how much gas is gonna remain in the tank of these batteries at the end of the day when the batteries are also providing regulation and flexi ramp. That's a risky thing that we don't model using probability distributions.
Benjamin Hobbs:But the the market software is moving in the direction of having multiple possible realizations whether for deployment of flexi ramp, different possible net loads or the use of batteries for regulation. So the stochastic programming camel is sneaking its nose under the tent. That means more complicated software, perhaps less transparent prices. We know market participants won't like that. So that's an issue that the MSC will have to deal with.
Benjamin Hobbs:Long term uncertainty in building infrastructure, as we know, we don't know where load is going in the future with data centers, how much flexibility they'll have. What are the implications of this for resource adequacy? Right now, the Eastern ISOs are changing the resource adequacy systems to something that looks more like New York's kind of prompt market. New England is is mulling over heading in that direction. PJM possibly also.
Benjamin Hobbs:So the old days, we used to think, oh, you have to procure capacity three years ahead of time in an RA market. That's what we did in PJM. And now people wondering whether that's necessary or are we gonna learn something the hard way find ourselves in a capacity shortage situation because we didn't look far enough ahead of time. So these are issues that the new MSCs will have to deal with. Allocation of congestion rents: a big issue at the start of our markets with LMP, you know, there's this money that the ISO gets because consumers pay more than what is paid to generators because of congestion, and there's a lot of debate over how and whether that and how much of that should be returned to consumers, and whether the allocations will greatly change as we as we must expand the geographic scope of markets, as well as the temporal scope looking ahead several days as has emerged in the discussions preceding day one of EDAM, there's going to be potentially big shifts of congestion rents among balancing authorities as perhaps in the past people caused congestion that they didn't pay for and now they're gonna perhaps have to pay for it.
Benjamin Hobbs:Those will be issues that the present embassy has started talking about and will not go away after the beginning of EDEM. And so I wish the new embassy all the luck I'll be watching and enjoying learning about what they do, And and no doubt that your podcast will will help and make my watching things from the other coast, the East Coast, more enjoyable as as things continue to develop in California.
Paul Dockery:Dude, do we did we recruit you? Are you gonna be a listener? Oh, absolutely.
Benjamin Hobbs:Oh, glad.
Paul Dockery:I'm flattered.
Benjamin Hobbs:I I listened to them when I when I bicycled to work.
Paul Dockery:That's awesome. Thank you very much. I and thank you for sharing your wisdom, both of you, during this. Ben, I hope you feel seen, heard, valued, appreciated.
Benjamin Hobbs:Thanks. Good. Yes.
Paul Dockery:Good. Becky, we we we don't get awards, but I do hope you feel seen, heard, valued, and appreciated. Thank you for the conversation and joining and doing this with me.
Becky Robinson:Yeah. No. Today's been great. It's been such a pleasure to have our two special guests today, and, I do just wanna say, you know, that your the imprint you have on the design of markets here and and more broadly as well, I know, but it's huge. Right?
Becky Robinson:And I I've heard from folks on my team that, you know, the best part of their job is getting to work with the MSC. And I think that's in no small part because of who we have had on the MSC and and just the the great insights and personalities and and willingness to, you know, highlight these issues and wrestle with them and, and share your share the how you think about these issues. And, that's just always been, I think, incredibly inspiring, and we'll live on through your opinions. So thank you.
Benjamin Hobbs:Well, Scott and I have seen staff members wince when one of us raises our voice. You know, once in a while, you know, when a when a stakeholder says this is fundamentally flawed, once in while, we actually say that person has a point. We didn't always agree with staff. And that's We good stuff think that the result were often changes that maybe even put some of the time were actually worth making. Yeah.
Benjamin Hobbs:And so it's it's it's not always been comfortable, the news that we've delivered. A staff member would like to talk about us vomiting all over.
Becky Robinson:Certainly, there's lots of different vantage points to view these market design challenges, and sometimes a fresh perspective is helpful. So
Benjamin Hobbs:Yeah. We wonder whether, there's a reason why no other ISO has established such a committee. I like to think that California's problem I mean, California's problem's political situation is unique and you're getting, you know, more careful vetting and I think that was needed given what it went through with the crisis. But hopefully other ISOs will establish some. And of course we know the WEAM has a market advisor and that's a good step, Susan Pope.
Benjamin Hobbs:And and I I like to think I do think that we've we've been very helpful over the years.
Paul Dockery:Yes. Absolutely. Well, thank you. And thank you to our listeners. While you aren't seen or heard, you are valued and appreciated.
Paul Dockery:Please subscribe and share the show that so that other electricity market enthusiasts like us can find us. Roll on, enthusiasts. Roll on.
Becky Robinson:Frequency Band is a production of the California ISO. It is produced and directed by Paul Dockery Paul Koliatich and Jeremy Lipps with writing by Paul Dockery and Becky Robinson. It is mixed, edited, published by Paul Koliatich with graphics by Stacy Gibbs and Annabel DeGraff. Jamie Ackman is its editor in chief.
Paul Dockery:Its executive producers include Crystal Ball, Jacob Mays, Nicole Hughes, Aaron Bloom, Deborah Smith, Monica Gaddis, and Pam Sporbord.
Becky Robinson:The views expressed during today's recording are our own and not the official views of the California ISO or the organization of the guests also appearing on Frequency Band today. Any aggregation, quotation, or references to opinions shared in today's episode should be ascribed to the individual participants and not their respective organizations.
Paul Dockery:You can find additional information in today in the show notes of today's show, including where to subscribe. Frequency Band, celebrating the wonky charm of electricity markets.
Becky Robinson:Frequency Band, staying in sync at 60 Hertz.
Paul Dockery:Thank you. That was great.
Benjamin Hobbs:Okay. Yeah. I'm looking at the clock. How long is this podcast? It's, they went Okay.
Benjamin Hobbs:To two