Subscribe
Share
Share
Embed
Enterprising Investor is the flagship podcast of CFA Institute and the definitive program for the investment management industry. As stewards of the investment industry, Enterprising Investor will feature intimate conversations with some of the most influential people from the world of finance about the topics that matter most to investment professionals.
Kevin Max (00:02.981)
We eventually get to the core question if AI can eventually do most of what portfolio managers do today, what remains distinctively human, and is that enough to sustain the profession? Enter the idea of the self driving portfolio, a network of AI agents that operates continuously, generates capital market assumptions, constructs portfolios, even challenges its own conclusions, monitors outcomes, and modifies itself when you get things wrong.
Hello and welcome to the Enterprising Investor, the flagship investment podcast for the CFA Institute. I'm Kevin Max, editor of Enterprising Investor blog, and I'm joined today by Dr. Irina Bebza. Irina is head of the quantitative so yeah, the quantitative solutions at Finaco Asset Management in Dublin, and she is a board member of the CFA Society in Ireland and a research fellow at Trinity Business School, Trinity College Dublin. Her research focuses on qualitative
sorry, quantitative investment strategies and ETF market structure. Most recently, Arena did a tremendous amount of work in assessing the latest human thinking around the topic of AI driven investment portfolios, and at work became the blog, the self-driving portfolio, promise pitfalls, and the practitioner GAT. You can find this blog post at cfanstitute.org, then search for my guess surname, Bebza B E V Z A. Welcome, Arena.
Irina Bevza (01:32.406)
Hi Kevin, thank you very much for having me here. It's my pleasure.
Kevin Max (01:35.67)
Thanks. Thanks for coming on the podcast. So so let's start with the proposition before we get into the pitfalls. When you say self driving portfolio, what are we actually talking about?
Irina Bevza (01:48.313)
Sure. So
The concept is actually not new. It's something that was derived from when quantitative and systematic strategies came to be. So the idea there behind it is just you encode investment decisions into rules and models. However, AI adds an additional layer of orchestration and I think it's great and interesting. What it does is previously humans had to connect analysis, research, risk management together, so now there's analysis.
element of AI contributing to it. However, so that's where like this is the feature of the concept. However, I don't see it as a completely new new concept because as I said it has been around for quite a while.
Kevin Max (02:34.169)
Okay. Well, you know, the proposed architecture creates this investment basically a whole investment organization around built on software. So, you know, includes basically the software version of researchers, portfolio instructors, constructors, risk managers, critics, even investment its own investment committee. So what does that tell us about where the future of AI is headed?
Irina Bevza (03:00.456)
Yeah, yes, sure. So
I would say before we go there, let's just take a pause for a second and think what we need what we needed. So the concept itself, what it means that like you would create all these agents, right, and they would have to collaborate and work very well together. But what we currently experiencing for say average investment firm is that we still are very far behind in terms of the data preparation. So a lot of
a huge amount of work needs to be done to in terms of like data governance and data cleaning, data preparation just for AI to be able to function very efficiently. Unfortunately fa finance, maybe investment industry is a little bit ahead of it, is still not fully capable not fully capable to do it. However, what AI does very well already right is
the speed of some of the processing so where you have it enabled is doing it very very well but what I'm seeing I think for now is that you can use AI on localized tasks rather than actually fully automate in like use it at the broader like automation of the of an investment from level.
Kevin Max (04:22.533)
Well, it it's interesting and we'll get to this in a little bit, but in given those drawbacks and those insufficiencies, we're already seeing self driving portfolios that are in retail markets right now. Like I can go online and invest in a self driving portfolio today for thirty dollars a share. and and you know, i I realise there are tons of caveats that still have to be met, but
We'll get to that in a moment. but the concept is that these AI agents are automating all these different processes and and doing those to different degrees. And so I guess the question is, like, why wouldn't we do this? if AI can do some of these things and many of these things better than humans, what's the drawback right now?
Irina Bevza (05:19.722)
I think
So I cannot say that AI is doing a lot of processes better than humans yet. it does things faster, it can be rational, it can think, but in a lot of cases I think we're still at the stage where it is more enabling and collaborating with the human rather than replacing the human. I think that's probably what the h the kind of the big question is about, about the future of portfolio management and overall.
And say for example yeah, sorry, I think c can we detour for example? I I think I need to think of how to answer this question.
Kevin Max (06:04.133)
Well let me let me ask you this. Some humans that you seem you seem okay with them replacing might be the governance council or the committees. So the so in your blog for Enterprising Investor, you led with how slow investment committees are and human g and human governments is deliberately slow. You know, we've all worked alongside some of these councils where they only meet once a month or maybe they'll meet quarterly.
Irina Bevza (06:15.383)
Yeah.
Irina Bevza (06:22.508)
Yeah.
Kevin Max (06:32.259)
to decide on the merits of g what the investments they'll make and and and what governance looks like. So that's not the way that AI works though. AI works in markets work daily, you know, daily if not continuously. So is what we call good governance sometimes simply institutional latency or le lethargy.
Irina Bevza (06:45.42)
Yeah.
Irina Bevza (06:58.656)
Yeah, let's
break it into a few things for a moment. yes, human committees are taking a long time and it's it is annoying. And working in those committees and being present of those committees it it is frustrating. however, there's a bit where f okay, first of all like what AI can do here, right, is help bring the analysis faster to make this
committees operating faster because what usually delays these committees is multiple stakeholders preparing and doing analysis and deliberating on a task and try to understand and come come to conclusion. Where AI could be really efficient is to bring all this information to the committee faster so committee can work more efficiently. In this way you actually already narrowed the gap where that that frustrating gap that people cannot make
decision all the time. The second part that you mentioned is yes, you can have your system of agents that function like a committee and they are communicating and discussing and talking nonstop. However, there's a big part of ownership that's missing here. We're still not fully comfortable to outsource investment decision making making to a machine. And I think even when you're yourself looking at that 30 euro investment you might you
won't put your pension there, right? You must likely put pension in more traditional fund. So that is going to be a challenge for for a while because we are still not yet comfortable to fully trust the machine. And honestly, re from the perspective of the regulation and like the you know all the way to protect investor, we are not there yet, and it won't won't be there yet. So that's where actually the function of the human portfolio manager or
Irina Bevza (08:59.906)
person who sits on investment committee whether it's a risk manager or stakeholder there where it would evolve people would be able to make decisions faster because with a proper like data infrastructure AI will be more operational, more fit more efficient. So the time to market what we call it it could happen faster, it's just all these levels of preparing the information will be apparently taking a lot of time so that would be replaced.
So I think what I see in terms of where it is going is we will see more collaboration. It's not like AI replacing portfolio managers at the stage or replacing investment committees at the stage. It's more them working together and our roles evolve, maybe becoming more efficient and more sophisticated.
Kevin Max (09:52.196)
I played devil's advocate here for a second. So on one side you see like that notion hanging on by a thread, which is once governance catches up and maybe we're more comfortable with AI making these decisions, you know, if I'm s if I'm KKR and I'm going around to pension funds, I'm gonna go to the ones that really are able to make decisions on a rolling basis and do it quickly, rather than those who I have to schedule three months out.
before they get around to it and then maybe they'll make the decision that day and maybe it'll be another month. I I kind of see market forces playing into this, sadly, rather than just good old fashioned governance.
Irina Bevza (10:33.762)
you're right but at the same time what you will see in s after some period is you won't need to schedule a meeting three months in advance so those people will become faster and you will be debating between instant decisions that you probably don't need and just the faster decisions. So to to pay to play devil's ad advocate back at you
Kevin Max (10:56.377)
Good. That's a good point. Yeah. Right. Too shy. Okay. All right. So, you know, we t we're talking about these investment committees, and you know, there's some alert, some level of lethargy behind them, maybe, and it takes time. But you know and there is this improvement on AI side that can also be offered. But humans also have these failure modes. We'll talk about the AI failure modes that
you get to in your in your paper. But let's talk about the humor human failure modes, which are things that there are AI counterparts for, like group think, confirmation bias, career risk well, maybe not not for AI, but career risk, recency bias, and even personal politics. So are we now are we going to hold AI to a standard of reliability that human portfolios themselves can't match?
Irina Bevza (11:55.451)
I think there's some hu
kind of the human bias of been applied to AI is that we're trying to compare AI to a perfect human, right, that makes amazing decisions, is very, very smart and you know, like almost like an ideal prompt that you could write, you know, in one line, just imagine is the best investor in the world that that's making decisions. Unfortunately, like we like it is with life, we know humans have biases as you mentioned and is extensive
body of research that has been studying those biases and I think the current products have been adjusted to those biases. AI will also have its own biases and it's could be either part of you know the fault of the prompt engineering or the fault of being I think what I mentioned in the pay in the paper is being overly positive and tend to have these responses. What we the system we want to really get
Kevin Max (12:51.237)
Yeah.
Irina Bevza (12:58.228)
To is the collaborative system between humans and AI not replacing each other because that would be the stronger system. Rather the so where you can have AI offsetting some of the human biases, right? Being stronger there, and I think part of it has been the role of quantitative investing, right? To get the emotion out of the question, and I think that's where it is going. And human on the other side can be more.
diligent and understanding AI like biases coming off out of like LLM models and modifying them. But I would say we're still in quite we're in a very interesting environment at the moment because every couple of weeks we have a new LLM model right with a different modification with different fine tuning. So what we're seeing is like it's growing and it's been monetized over time. So but and the new roles are new new skills have
been developed right and it will continue. Like this is the part of the growth and that that's the direction we're going. So we will never eliminate the biases but the best model is the combination is to complement each other.
Kevin Max (14:15.819)
That sounds that sounds pretty amazing. now I'd like to kind of go in a slightly different angle here, which is your article suggests that the answer isn't simply that AI makes mistakes. It's that AI can make mistakes in a different way, different and potentially systematic ways. and so and there are a number of there are a number of kind of angles that you look at this in the in the blog post.
But let's take a deeper dive into the failure modes of AI that you note and let's start with the cascade problem. So how does one relatively small error cascade into a portfolio lever level failure in this multi agent system?
Irina Bevza (14:59.286)
Yes, absolutely. So or say when you mentioned the like self driving portfolio, right? Potentially behind it you would have like a different a system of agents, right? And naturally sim some in somewhat similar to the an organization, right? Because that's what we currently shape it of, right? they would have different functions. And say there was a news flow and it was I know it
tariff news and an agent misinterpreted it would have misinterpreted it right and so that's where the cascade problem would start right it would say something positive became negative and then would would feed down upstream to say portfolio manager agent or the risk agent and they would take the opposite side of the portfolio while say a follow-up news would have c cleared it up. So that's where you have a cascade problem right
Right, you have a data point that is arriving very down the s stream and impacting it all the way going all the way up, leading to a a material impact from a position that was misunderstood. Similarly could happen in human, but because in the human system, but because we do take longer decisions, there's a a time for news to come back and flow back and be interpreted multiple times by news agencies and have a level
level of seniority of the managers also participating in this decision and being c being corrected. So that's kind of one of the problems to the cascade effect. It could happen at different levels at different scales. So what I described is could be some some small material impact but it could go all the way up. So we could like you you understand the flow and like the impact could be could really be different.
Kevin Max (16:40.463)
Yeah.
Kevin Max (16:57.785)
I I worry that five or ten years down the road that we won't have those senior people who still understand and could disintermediate and step in and say, Huh, that doesn't seem right. Based on what experience that I have, I have no experience because I've been using AI for everything.
Irina Bevza (17:13.368)
Ha ha ha ha.
I think you underestimate the p the commitment of the social structures we have. I don't think we will see the the c moving away of the seniority or people like I think if something what AI adds is in a way is a creativity and run in different scenarios so I'm actually very positive about it. I think it could give a lot of learning experience to people who might might have not experienced it but
But maybe yeah, go ahead, sorry.
Kevin Max (17:46.415)
I really hope I hope I hope the sky is is that blue and that your world is is the one that went out.
Irina Bevza (17:51.671)
Ha ha.
Maybe in your scenario all people would have invested the entire pension in the successful robot driven portfolio and, you know, retired somewhere on a nice island and enjoying the martinius in the i in the afternoon. So
Kevin Max (18:04.836)
Love that. okay. one of the other failures she talked about is quote unquote the false consensus trap. Can you tell me about that?
Irina Bevza (18:17.538)
False consensus is related to have a lot of similarity and I think it's actually very human as well. But the idea behind it is if you have very similar models of of prompts, I think n now kinda we move down to the prompt engineering is more common, where you basically have like a system of agents and they two dribble with each other. It's means you've given them the settings that are very common and you just
effectively have the system that agrees to itself all the time, it doesn't create any diversification or of opinions. It's also I think one of the papers I was referencing, it was a common thing just overall with the a like some of the LLMs being too agreeable in the settings so and when one one talks with another one they tend to just reinforce each other by each other's biases. Like it's it happens when people are too polite as well, I think.
Kevin Max (19:16.901)
Right. Maybe maybe these models should all start with the question, should I short sell this? And if not, why?
Irina Bevza (19:17.402)
Okay.
Irina Bevza (19:23.512)
Yeah, exactly. I think I think it's a good start, yes.
Kevin Max (19:27.197)
okay, so now one of the other interesting things you talk about as a failure, is this ghost in the machine quality, which is the meta agent. So that AI evaluates mistakes and rewrites instructions or code to improve the system. So isn't that also where things become genuinely difficult to govern? And realizing that governance, you know, is maybe different in Europe than it is in the US, at least it in the
Irina Bevza (19:49.944)
Mm, yeah.
Kevin Max (19:56.845)
At this moment we're talking.
Irina Bevza (19:58.827)
Yes, absolutely. And actually maybe have to add that my blog is based on a paper about self-driven portfolio and the concept of the meta agents from there. And I think it's really clever, to be honest, because you effectively have this like meta agent that oversees all other agents and based on the mistakes adjust the prompt. So it's a well very elegant decision to operate a system like this. However, unfortunately we are not in a
w working with the precise science. the financial markets they're social science. So like it's a lot of unpredictable things happening and especially when we are in a environment of market stress, there's not enough information or data to draw from. So if you have a system that has been trained a lot on their calm markets, it might have very difficult time adjusting it. And the same happens to the meta engine because
It would try to reorganize the agent system but wouldn't really know how, but it would not even have time to relearn because the volatility is so high. so that's where it's quite challenging for the the system to exist, but so is for a lot of say systematic strategies. So it has been quite common and the same is with humans as well. So that's where we have our most
pressure to pull out of the position or to the pressures to make a like a different decision. But yeah, so going back to the meta agent, that's kind of the challenging part that we need to work more and pro and test more like and be more robust in building up any s a system like this because that that's where it's crucial.
Kevin Max (21:56.762)
Yeah. And you know, one one of the problems that keeps coming up again and again across my desk is the problem of can you audit this black box and the accountability behind that, right? Because let's say in your blue sky world where we're working alongside AI and in harmony, the human part still has to be able to interpret for its clients or regulators or whomever what's going on in this black box next to me, right? So do you see
Like, do you see this as a big issue going forward? the accountability, who's accountability, who's accountable to clients and who's accountable to regulators?
Irina Bevza (22:37.759)
Yes, I think
It's a very good question and it's it will take a bit of time for us to answer it. Not to me I mean at the moment but our overall as we evolve in and adopting this new technology, the distributing of accountability, like and whether you can give it to machine, that's a big question. I think there will be a lot of additional regular driven directions to bring it to to find
tune it but for the time being actually I wanna say a few years ago there's a big trend with machine learning and I think the black box is really coming from there because this notion of okay we're now going to run like a neural network on something and we do what really know how how it makes decisions so in a way LLM is a step ahead in terms of give it a little giving a little bit of comfort to people using this technology because and you probably
know from your experience of using an Chin GPT or Claude is it kind of tends to write down for yourself to you to read all its thoughts but go imagine like going back and auditing it all so you have the multi-agent system that is debating the decision making and it it's not a very like not the most brief it's not working with a quant person who like usually operates in one sentence you work with like LLM who likes to produce
a Bible for decision making. So imagine going and auditing it after after it as a human.
Kevin Max (24:09.113)
Yeah. Yeah.
Yeah. R right. No, I know. In in some ways I feel like on the personal level I'm working with a crazy person. because I could ask a big question like, Hey, if I'm buying a car, what do you think about these cars? And it'll be and it'll say, These are all great and s and you know, extol the virtues of all of them. And then I drill down on one of and you know, Claude may say, n hell no. I would never ever there's so many problems with this car, I would never buy even think about buying that.
so but you know, back to kind of the accountability thing, I come from a big family. And so this is just a quick analogy. I come from a big family and my parents worked, you know, all hour all hours a day, and you know, when they would come home and you know an heirloom may be broken. And so there was no one ever accountable for that because you had four kids just standing there looking at each other and no one was gonna say say who did it. and yet there was
there was an heirloom pregnant, and so that heirloom could be, you know, a complete portfolio and billions of dollars w worth of investment. And so that's kind of where my fears go on this question of black box and humans working alongside black boxes they don't make they don't understand.
Irina Bevza (25:24.053)
Yeah.
Irina Bevza (25:29.168)
And I think that's where the role of human would evolve a lot. I think if you talk a little bit over all of like of the role of a portfolio manager and how you grow to become a portfolio manager because of the different steps you you go through before you become like say start you get to the level where you take that much responsibility. And I think what will happen is we will
learn or we will take ownership much early in a career, because there will be less demand for say what AI is now doing, like PPT making or like quick analysis or consolidating of you know database creating charts. That's what's usually like when I started my career, right? my first like few years it was mainly it was like t twenty percent maybe actually their growing work, right? Manificent
By the CFA, by the way, because I had to take it almost very very early in my career. But a lot of time as a junior you pick up like you know this random job that no one wants to do because for them it's your learning curve, but for you it's like, yeah, I just have to do it. So that's what AI will take away. But the positive thing in it is that junior person will be more trained to take over the ownership much faster to learn how to be accountable to.
Kevin Max (26:49.402)
Mm-hmm.
Irina Bevza (26:58.798)
decision how to audit and validate those decisions and I think it's great from the perspective of critical thinking and how we grow overall and involved as humans. So sorry I'm going back to I'm very positive on on all the developments.
Kevin Max (27:12.451)
Yeah. Okay. But this is this is good and I wanna I want to stay in this moment and kind of switch to the human side of things now. So just I just recently read a piece from an esteemed group of PhDs and CFAs and they're from UC Berkeley, Stanford, IBM Research and more, and they said, quote, the first large scale study of AI agents in production finds that successful deployments are simple, tightly constrained,
And continuously supervised. In other words, AI agents today are neither autonomous nor casually intelligent. That was a year ago. And yet today, as was started the segment with, we have an ETF that has to ticker AINTAINT, which is great. And it's an all AI driven, dollar neutral portfolio of SP large caps.
It has been trading since February and has significantly outpaced its benchmarks, although it's hard to say what a benchmark is for that. and so this is this is really the genie is not just out of the bottle. The genie is already granting wishes to investors.
Irina Bevza (28:26.84)
And I would say would you put your money into it and how would you put your pension into it?
Kevin Max (28:34.197)
Sorry, would I put my yeah. so for me for me I'm thinking about actually playing money right now until some of these issues have been resolved we're talking about. But then a after that, who knows?
Irina Bevza (28:36.928)
Like would you be would yes
Irina Bevza (28:46.722)
So you would put your dis like some maybe discretionary, like a little bit, but you wouldn't trust your pension, like say. So I think that's where the bias is. That's and part of it is I think the fund you mentioned has only been traded from February and going back to our earlier conversation you really need to see through different stress periods over time, to see how it performs. And with the long short strategies I think it's even more common because
Kevin Max (28:49.966)
Yes.
No, no. Good done now.
Irina Bevza (29:16.666)
so if I think for especially for r retail investors when they try to look at so sorry I d I like I I don't know how well what's your background in terms of investing so I don't I don't want to assume but
Kevin Max (29:30.277)
So yeah, the the launch art strategy, yeah, I I was interested what you were going to say next there.
Irina Bevza (29:36.661)
Yeah, so I kinda have dealt with different type of investors, through my time like through my career, right? And i like
Yeah, it's a great story, but for retail investors when you come and tell them okay you're going to buy a long short portfolio, it becomes very confusing because people are more kinda cobbling to to do log only S P and overall, especially like in a world where S P just go grows like it grows, like it's also quite a different difficult thing to sell to the retail investors. I saw the performance, it's fantastic, it's a really good
fund output I would also buy it to be honest. Just from the perspective of also s supporting it as well because seeing innovation in the space is great. but yeah I think the problem is we wouldn't trust our pension in it in it and that would be the bias and the thing to address.
Kevin Max (30:20.899)
Yeah. It's lunch money for now, but yeah.
Kevin Max (30:33.507)
Right. Yeah.
Okay, so then we wouldn't trust to put our pension in right now, but what would have to happen for you to be able to trust that and to, you know, put a significant amount of money into something that's AI driven?
Irina Bevza (30:55.318)
I think for now we know that it's not like
technology is very fast and I know where it's much better than me, right? But there's we need to get to the level of governance and being like autonomous, being auditable, all these pillars that make it act like independently and trust trustworthy. Right? currently I don't know who's going to take responsibility if something's happening to my investment. I don't know
what whether like what would be the parameters for it to act or how it's controlled because even like from my personal experience right I use ai extensively for coding and we also trying to figure out how to create our agent systems a lot of time like I'm not
Like I think it's very already trustworthy in sense like it checks in with you, like it makes sure like you doesn't we don't like give access to like aca occasional folders that's not supposed to be again access but it still hallucinates a lot, it still go in and creates random stuff and it's part way we're still learning how to do proper like prompt engineering or organizing it.
Kevin Max (32:17.333)
Right. Okay, Ren, now I'm really going to put your sky your blue skies to test here, your blue sky optimism to test here, with kind of a closing question. All right, so I want you to complete this sentence. Ten years from now, maybe even five years from now, the human portfolio manager will most likely be blank.
Irina Bevza (32:25.08)
No.
Irina Bevza (32:41.792)
still there.
Kevin Max (32:44.501)
Still there in what capacity?
Irina Bevza (32:47.6)
I wanna say that like with every technology
the roles of people and the skill set of people evolved. So it won't be the same set skills as the current portfolio manager but the responsibility is going to gonna still be there and the way it acts will be s slightly different. So what I want to say is we're not going to be replaced, right? It's gonna be just different and evolved.
And I think also on the part of the portfolio manager, it depends what happens to the investment industry. So five years from now, ten years from now, like you kinda I wanna hope that there won't be huge change in terms of the demand, right? Because you see so many interesting things apart from AI happening in the space with like ETF, like faster trading, like faster settlement. So the industry is really becoming much more interesting.
and diversified and there's so many creative ideas like the ETF you mentioned that coming. So this is not going to go away, but with the delegation of the boring tasks, I hope the creativity is going to come to emerge and move the industry in a more interesting direction.
Kevin Max (34:13.657)
Well, I hope you're right, and I hope all the I hope all the portfolio managers listening in on this will agree with you as well. My my guest today has been Dr. Irina Bebza and she is the head of quantitative solutions at FinECO Asset Management in Dublin. She is also a board member for the CFA Society of Ireland and a research fellow at Trinity Business School at Trinity College Dublin. And Irina, thank you so much for joining me. It was fun.
Fascinating topic and a great conversation.
Irina Bevza (34:45.314)
Fantastic, thank you, Kevin.